arXiv subjects

Anton Vorontsov

Publications and source records attributed to Anton Vorontsov.

5 recordsLinked to original sources

arXiv2025

Systems and Algorithms for Convolutional Multi-Hybrid Language Models at Scale

We introduce convolutional multi-hybrid architectures, with a design grounded on two simple observations. First, operators in hybrid models can be tailored to token manipulation tasks such as in-context recall, multi-token recall, and compression, with input-dependent convolutions and attention offering complementary performance. Second, co-designing convolution operators and hardware-aware algorithms enables efficiency gains in regimes where previous alternative architectures struggle to surpass Transformers. At the 40 billion parameter scale, we train end-to-end 1.2 to 2.9 times faster than optimized Transformers, and 1.1 to 1.4 times faster than previous generation hybrids. On H100 GPUs and model width 4096, individual operators in the proposed multi-hybrid StripedHyena 2 architecture achieve two-fold throughput improvement over linear attention and state-space models. Multi-hybrids excel at sequence modeling over byte-tokenized data, as demonstrated by the Evo 2 line of models. We discuss the foundations that enable these results, including architecture design, overlap-add blocked kernels for tensor cores, and dedicated all-to-all and point-to-point context parallelism strategies.

cs.LG↗

arXiv2024

BioNeMo Framework: a modular, high-performance library for AI model development in drug discovery

Artificial Intelligence models encoding biology and chemistry are opening new routes to high-throughput and high-quality in-silico drug development. However, their training increasingly relies on computational scale, with recent protein language models (pLM) training on hundreds of graphical processing units (GPUs). We introduce the BioNeMo Framework to facilitate the training of computational biology and chemistry AI models across hundreds of GPUs. Its modular design allows the integration of individual components, such as data loaders, into existing workflows and is open to community contributions. We detail technical features of the BioNeMo Framework through use cases such as pLM pre-training and fine-tuning. On 256 NVIDIA A100s, BioNeMo Framework trains a three billion parameter BERT-based pLM on over one trillion tokens in 4.2 days. The BioNeMo Framework is open-source and free for everyone to use.

cs.LG↗

arXiv2024

Chiral superfluid helium-3 in the quasi-two-dimensional limit

Anisotropic pair breaking close to surfaces favors the chiral A phase of the superfluid $^3$He over the time-reversal invariant B phase. Confining the superfluid $^3$He into a cavity of height $D$ of the order of the Cooper pair size characterized by the coherence length $\xi_0$ - ranging between 16 nm (34 bar) and 77 nm (0 bar) - extends the surface effects over the whole sample volume, thus allowing stabilization of the A phase at pressures $P$ and temperatures $T$ where otherwise the B phase would be stable. In this Letter, the surfaces of such a confined sample are covered with a superfluid $^4$He film to create specular quasiparticle scattering boundary conditions, preventing the suppression of the superfluid order parameter. We show that the chiral A phase is the stable superfluid phase under strong confinement over the full $P$-$T$ phase diagram down to a quasi-two-dimensional limit $D / \xi_0 = 1$ , where $D = 80$ nm. The planar phase, which is degenerate with the chiral A phase in the weak-coupling limit, is not observed. The gap inferred from measurements over the wide pressure range from 0.2 to 21.0 bar leads to an empirical ansatz for temperature-dependent strong-coupling effects. We discuss how these results pave the way for the realization of the fully gapped two-dimensional $p_x + ip_y$ superfluid under more extreme confinement.

cond-mat.supr-con↗

arXiv2016

Spin susceptibility of Andreev bound states

We calcuate electronic spin susceptibility and spin-lattice relaxation rate in singlet superconductor near a pairbreaking surface, or in a domain wall of the order parameter. We directly link presence of high-density Andreev bound states in the inhomogeneous region, combined with coherence factors, to enhancement of the susceptibility above the normal state's value for certain $\bf q$ vectors. Beside the dominant peak at ferromagnetic vector $q=0$, we find significant enhancement of antiferromagnetic correlations at vectors $q\lesssim 2 k_f$, with $\bf q$ $along$ the domain wall in $S$-wave superconductor, and $across$ domain wall in $D$-wave (nodes along the wall). These features are destroyed by applying moderate Zeeman field that splits the zero-energy peak. We solve Bogoliubov-de Gennes equations in momentum space and our results deviate from the lattice models investigated previously. Large enhancement of the spin-lattice relaxation rate $T_1^{-1}$ at the domain wall provides clear signature of the quasiparticle bound states, and is in good agreement with recent experiment in organic superconductor $κ$-(BEDT-TTF)$_2$Cu(NCS)$_2$.

cond-mat.supr-con↗

arXiv2014

Enhancement of Electronic Spin Susceptibility in Pauli Limited Unconventional Superconductors

We calculate the wave-vector dependent electronic spin susceptibility $χ_{αβ}(\mathbf q, \mathbf H_0)$ of a d-wave superconductor in uniform magnetic field $\mathbf{H}_0$ with Pauli pair-breaking. We find that the transverse component of the susceptibility tensor can be greater than its normal state value; the longitudinal component also slightly increases but in a very limited range of $q$s. We identify several wave vectors $\{\mathbf q_\perp,\mathbf q_\parallel\}$, that correspond to the maxima of either $χ_\perp$ or $χ_\parallel$. We compare our results with available data on the high-field phase in heavy-fermion CeCoIn$_5$.

cond-mat.supr-con↗