arXiv subjects

Anton Vakhrushev

Publications and source records attributed to Anton Vakhrushev.

5 recordsLinked to original sources

arXiv2026

Exploring New Frontiers in Vertical Federated Learning: the Role of Saddle Point Reformulation

The objective of Vertical Federated Learning (VFL) is to collectively train a model using features available on different devices while sharing the same users. This paper focuses on the saddle point reformulation of the VFL problem via the classical Lagrangian function. We first demonstrate how this formulation can be solved using deterministic methods. More importantly, we explore various stochastic modifications to adapt to practical scenarios, such as employing compression techniques for efficient information transmission, enabling partial participation for asynchronous communication, and utilizing coordinate selection for faster local computation. We show that the saddle point reformulation plays a key role and opens up possibilities to use mentioned extension that seem to be impossible in the standard minimization formulation. Convergence estimates are provided for each algorithm, demonstrating their effectiveness in addressing the VFL problem. Additionally, alternative reformulations are investigated, and numerical experiments are conducted to validate performance and effectiveness of the proposed approach.

math.OC↗

arXiv2024

ProtBoost: protein function prediction with Py-Boost and Graph Neural Networks -- CAFA5 top2 solution

Predicting protein properties, functions and localizations are important tasks in bioinformatics. Recent progress in machine learning offers an opportunities for improving existing methods. We developed a new approach called ProtBoost, which relies on the strength of pretrained protein language models, the new Py-Boost gradient boosting method and Graph Neural Networks (GCN). The ProtBoost method was ranked second best model in the recent Critical Assessment of Functional Annotation (CAFA5) international challenge with more than 1600 participants. Py-Boost is the first gradient boosting method capable of predicting thousands of targets simultaneously, making it an ideal fit for tasks like the CAFA challange. Our GCN-based approach performs stacking of many individual models and boosts the performance significantly. Notably, it can be applied to any task where targets are arranged in a hierarchical structure, such as Gene Ontology. Additionally, we introduced new methods for leveraging the graph structure of targets and present an analysis of protein language models for protein function prediction task. ProtBoost is publicly available at: https://github.com/btbpanda/CAFA5-protein-function-prediction-2nd-place.

q-bio.QM↗

arXiv2024

Cross-Domain Latent Factors Sharing via Implicit Matrix Factorization

Data sparsity has been one of the long-standing problems for recommender systems. One of the solutions to mitigate this issue is to exploit knowledge available in other source domains. However, many cross-domain recommender systems introduce a complex architecture that makes them less scalable in practice. On the other hand, matrix factorization methods are still considered to be strong baselines for single-domain recommendations. In this paper, we introduce the CDIMF, a model that extends the standard implicit matrix factorization with ALS to cross-domain scenarios. We apply the Alternating Direction Method of Multipliers to learn shared latent factors for overlapped users while factorizing the interaction matrix. In a dual-domain setting, experiments on industrial datasets demonstrate a competing performance of CDIMF for both cold-start and warm-start. The proposed model can outperform most other recent cross-domain and single-domain models. We also provide the code to reproduce experiments on GitHub.

cs.IR↗

arXiv2022

SketchBoost: Fast Gradient Boosted Decision Tree for Multioutput Problems

Gradient Boosted Decision Tree (GBDT) is a widely-used machine learning algorithm that has been shown to achieve state-of-the-art results on many standard data science problems. We are interested in its application to multioutput problems when the output is highly multidimensional. Although there are highly effective GBDT implementations, their scalability to such problems is still unsatisfactory. In this paper, we propose novel methods aiming to accelerate the training process of GBDT in the multioutput scenario. The idea behind these methods lies in the approximate computation of a scoring function used to find the best split of decision trees. These methods are implemented in SketchBoost, which itself is integrated into our easily customizable Python-based GPU implementation of GBDT called Py-Boost. Our numerical study demonstrates that SketchBoost speeds up the training process of GBDT by up to over 40 times while achieving comparable or even better performance.

cs.LG↗

arXiv2022

LightAutoML: AutoML Solution for a Large Financial Services Ecosystem

We present an AutoML system called LightAutoML developed for a large European financial services company and its ecosystem satisfying the set of idiosyncratic requirements that this ecosystem has for AutoML solutions. Our framework was piloted and deployed in numerous applications and performed at the level of the experienced data scientists while building high-quality ML models significantly faster than these data scientists. We also compare the performance of our system with various general-purpose open source AutoML solutions and show that it performs better for most of the ecosystem and OpenML problems. We also present the lessons that we learned while developing the AutoML system and moving it into production.

cs.LG↗