Searcharxiv⌕ Search

arXiv subjects

Ekaterina Kochetkova

Publications and source records attributed to Ekaterina Kochetkova.

7 recordsLinked to original sources
arXiv2026

Towards Tight Bounds for Streaming Attention

The attention mechanism is a cornerstone of modern transformer architectures. However, its expressive power comes at the cost of quadratic runtime and linear space usage. In particular, the classical transformer architecture explicitly stores all previously seen input elements (tokens) in order to generate the next one. The problem of implementing a transformer in limited space, known as KV cache compression, has received much interest over the past few years, spurring the development of powerful heuristics. Recent works of Haris et al, COLT'25 and Kochetkova et al, NeurIPS'25, formalized KV cache compression as the streaming attention approximation problem, providing both upper bounds (based on discrepancy theory) and information theoretic lower bounds. However, those papers left open a significant gap between the upper and lower bounds. For example, the space usage of their algorithms increases with the precision parameter, but the lower bound does not get stronger. In this work, we revisit the streaming attention approximation problem and provide nearly tight bounds on its space complexity. On the algorithmic side, we achieve the result through a surprisingly tight interplay between three distinct methods for kernel density estimation: discrepancy-based coreset constructions (e.g., Charikar-Kapralov-Waingarten'24), the polynomial method (e.g., Greengard-Rokhlin'87, Alman-Song'23), and space partitioning (e.g., Andoni-Laarhoven-Razenshteyn-Waingarten'17, Charikar-Kapralov-Nouri-Siminelakis'20). On the lower bound side, our main technical contribution is a new technique for using the INDEX problem with a large amount of side information that we hope will prove useful in other high dimensional geometric estimation problems.

cs.DS↗
arXiv2026

Streaming Attention Approximation via Discrepancy Theory

Large language models (LLMs) have achieved impressive success, but their high memory requirements present challenges for long-context token generation. In this paper we study the streaming complexity of attention approximation, a key computational primitive underlying token generation. Our main contribution is BalanceKV, a streaming algorithm for $ε$-approximating attention computations based on geometric process for selecting a balanced collection of Key and Value tokens as per Banaszczyk's vector balancing theory. We complement our algorithm with space lower bounds for streaming attention computation. Besides strong theoretical guarantees, BalanceKV exhibits empirically validated performance improvements over existing methods, both for attention approximation and end-to-end performance on various long context benchmarks.

cs.LG↗
arXiv2026

Engineering photomagnetism in collinear van der Waals antiferromagnets

Achieving efficient ultrafast optical control of antiferromagnetic spin dynamics is a central goal for next-generation high-speed THz spintronic and magnonic devices. Resonant optical pumping of crystal-field-split d-d orbital multiplets in magnetic TM ions directly modulates exchange and spin-orbit interactions, inducing large-amplitude coherent spin precession. However, such effects are limited to a handful of systems and there is no general strategy to enhance d-d photomagnetism in antiferromagnets. Here, we demonstrate the engineering of photomagnetism via TM-ion doping in collinear van der Waals antiferromagnets. In Mn$_{1-x}$Ni$_x$PS$_3$, small amounts of Ni$^{2+}$ activate a strong photomagnetic response while largely preserving the Néel ground state. Even 10% Ni boosts the response by more than an order of magnitude compared to pure MnPS$_3$, with resonant pumping of Ni$^{2+}$ d-d transitions driving large-amplitude coherent spin precession and providing helicity-dependent phase control. Tuning the pump energy across the full Mn$_{1-x}$Ni$_x$PS$_3$ composition range shows that Ni excitations remain effective across competing Néel and zig-zag antiferromagnetic states while supporting tunable-frequency coherent spin precession. These results establish TM-ion doping as a versatile strategy to harness orbital multiplet excitations for ultrafast, low-dissipation spin control in van der Waals antiferromagnets.

cond-mat.mtrl-sci↗
arXiv2026

Spectral Clustering in Birthday Paradox Time

Given a vertex in a $(k, φ, ε)$-clusterable graph, i.e. a graph whose vertex set can be partitioned into a disjoint union of $φ$-expanders of size $\approx n/k$ with outer conductance bounded by $ε$, can one quickly tell which cluster it belongs to? This question goes back to the expansion testing problem of Goldreich and Ron'11. For $k=2$ a sample of $\approx n^{1/2+O(ε/φ^2)}$ logarithmic length walks from a given vertex approximately determines its cluster membership by the birthday paradox: two vertices whose random walk samples are `close' are likely in the same cluster. The study of the general case $k>2$ was initiated by Czumaj, Peng and Sohler [STOC'15], and the works of Chiplunkar et al. [FOCS'18], Gluch et al. [SODA'21] showed that $\approx \text{poly}(k)\cdot n^{1/2+O(ε/φ^2)}$ random walk samples suffice for general $k$. This matches the $k=2$ result up to polynomial factors in $k$, but creates a conceptual inconsistency: if the birthday paradox is the guiding phenomenon, then the query complexity should decrease with the number of clusters $k$! Since clusters have size $\approx n/k$, we expect to need $\approx (n/k)^{1/2+O(ε/φ^2)}$ random walk samples, which decreases with $k$. We design a novel representation of vertices in a $(k, φ, ε)$-clusterable graph by a mixture of logarithmic length walks. This representation uses the optimal $\approx (n/k)^{1/2+O(ε/φ^2)}$ walks per vertex, and allows for a fast nearest neighbor search: given $k$ vertices representing the clusters, we can find the cluster of a given query vertex $x$ using nearly linear time in the representation size of $x$. This gives a clustering oracle with query time $\approx (n/k)^{1/2+O(ε/φ^2)}$ and space complexity $k\cdot (n/k)^{1/2+O(ε/φ^2)}$, matching the birthday paradox bound.

cs.DS↗
arXiv2025

Spectral Clustering with Side Information

In the graph clustering problem with a planted solution, the input is a graph on $n$ vertices partitioned into $k$ clusters, and the task is to infer the clusters from graph structure. A standard assumption is that clusters induce well-connected subgraphs (i.e. $Ω(1)$-expanders), and form $ε$-sparse cuts. Such a graph defines the clustering uniquely up to $\approx ε$ misclassification rate, and efficient algorithms for achieving this rate are known. While this vanilla version of graph clustering is well studied, in practice, vertices of the graph are typically equipped with labels that provide additional information on cluster ids of the vertices. For example, each vertex could have a cluster label that is corrupted independently with probability $δ$. Using only one of the two sources of information leads to misclassification rate $\min\{ε, δ\}$, but can they be combined to achieve a rate of $\approx εδ$? In this paper, we give an affirmative answer to this question and present a sublinear-time algorithm in the number of vertices $n$. Our key algorithmic insight is a new observation on ``spectrally ambiguous'' vertices in a well-clusterable graph. While our sublinear-time classifier achieves the nearly optimal $\approx \widetilde O(εδ)$ misclassification rate, the approximate clusters that it outputs do not necessarily induce expanders in the graph $G$. In our second result, we give a polynomial-time algorithm that reweights edges of the original $(k, ε, Ω(1))$-clusterable graph to transform it into a $(k, \widetilde O(εδ), Ω(1))$-clusterable one (for constant $k$), improving sparsity of cuts nearly optimally and preserving expansion properties of the communities - an algorithm for refining community structure of the input graph.

cs.DS↗
arXiv2023

Intermixing-driven surface and bulk ferromagnetism in the quantum anomalous Hall candidate MnBi$_6$Te$_{10}$

The recent realizations of the quantum anomalous Hall effect (QAHE) in MnBi$_2$Te$_4$ and MnBi$_4$Te$_7$ benchmark the (MnBi$_2$Te$_4$)(Bi$_2$Te$_3$)$_n$ family as a promising hotbed for further QAHE improvements. The family owes its potential to its ferromagnetically (FM) ordered MnBi$_2$Te$_4$ septuple layers (SL). However, the QAHE realization is complicated in MnBi$_2$Te$_4$ and MnBi$_4$Te$_7$ due to the substantial antiferromagnetic (AFM) coupling between the SL. An FM state, advantageous for the QAHE, can be stabilized by interlacing the SL with an increasing number $n$ of Bi$_2$Te$_3$ layers. However, the mechanisms driving the FM state and the number of necessary QLs are not understood, and the surface magnetism remains obscure. Here, we demonstrate robust FM properties in MnBi$_6$Te$_{10}$ ($n = 2$) with $T_C \approx 12$ K and establish their origin in the Mn/Bi intermixing phenomenon by a combined experimental and theoretical study. Our measurements reveal a magnetically intact surface with a large magnetic moment, and with FM properties similar to the bulk. Our investigation thus consolidates the MnBi$_6$Te$_{10}$ system as perspective for the QAHE at elevated temperatures.

cond-mat.mtrl-sci↗