arXiv subjects

Yimu Zhang

Publications and source records attributed to Yimu Zhang.

12 recordsLinked to original sources

arXiv2026

PRAC: Principal-Random Subspace for LLM Activation Compression and Memory-Efficient Training

Activations have become the primary memory bottleneck in large-batch LLM training. However, existing compression methods fail to exploit the spectral structure of activations, resulting in slow convergence or limited compression. To address this, we bridge the relationship between the algorithm's fast convergence and the requirements for subspace projection, and show that an effective compression should yield an unbiased estimate of the original activation with low variance. We propose Principal-Random Subspace for LLM Activation Compression (PRAC), which novelly decomposes activations into two components: a principal subspace captured via SVD to retain dominant information, and a random subspace sampled from the orthogonal complement to approximate the tail. By introducing a precise scaling factor, we prove that PRAC yields an unbiased gradient estimator with minimum variance under certain conditions. Extensive experiments on pre-training and fine-tuning tasks demonstrate that PRAC achieves up to 36% total memory reduction with negligible performance degradation and minimal computational cost.

cs.LG↗

arXiv2025

SimSort: A Data-Driven Framework for Spike Sorting by Large-Scale Electrophysiology Simulation

Spike sorting is an essential process in neural recording, which identifies and separates electrical signals from individual neurons recorded by electrodes in the brain, enabling researchers to study how specific neurons communicate and process information. Although there exist a number of spike sorting methods which have contributed to significant neuroscientific breakthroughs, many are heuristically designed, making it challenging to verify their correctness due to the difficulty of obtaining ground truth labels from real-world neural recordings. In this work, we explore a data-driven, deep learning-based approach. We begin by creating a large-scale dataset through electrophysiology simulations using biologically realistic computational models. We then present SimSort, a pretraining framework for spike sorting. Trained solely on simulated data, SimSort demonstrates zero-shot generalizability to real-world spike sorting tasks, yielding consistent improvements over existing methods across multiple benchmarks. These results highlight the potential of simulation-driven pretraining to enhance the robustness and scalability of spike sorting in experimental neuroscience.

q-bio.NC↗

arXiv2025

AdaPM: a Partial Momentum Algorithm for LLM Training

In the training of large language models, momentum is widely used and often demonstrated to achieve significant acceleration. However, storing momentum typically presents memory challenges. In this paper, we propose AdaPM, an adaptive training strategy that leverages partial momentum to implement a memory-efficient optimizer. To this end, AdaPM utilizes a non-uniform momentum design: for most blocks, full momentum is not necessary to preserve the performance of the optimization. In the momentum design of AdaPM, to mitigate the bias and performance loss caused by partial momentum, we enhance the partial momentum by a bias correction technique. Empirically, we verify that our approach reduces memory by over $90\%$ in momentum while maintaining both efficiency and performance for pretraining various language models ranging from 60M to 1.5B, as well as for supervised fine-tuning and RLHF. AdaPM can further reduce memory by up to $95\%$ in optimizer states by combining the memory-efficient technique on the second-order statistic, saving over $30\%$ GPU hours for pretraining GPT-2 1.5B.

cs.LG↗

arXiv2025

A remark on the counterexample to the unknotting number conjecture

By using Snappy, M. Brittenham and S. Hermiller discovered a very surprising example that $u(7_1\#\overline{7_1})\leq 5<6=u(7_1)+u(\overline{7_1})$, where $7_1$ is the $(2,7)$-torus knot and $\overline{7_1}$ is its mirror image. Based on their work, we give a direct verification of this fact.

math.GT↗

arXiv2017

Graphs in the 3--sphere with maximum symmetry

We consider the orientation-preserving actions of finite groups $G$ on pairs $(S^3, Γ)$, where $Γ$ is a connected graph of genus $g>1$, embedded in $S^3$. For each $g$ we give the maximum order $m_g$ of such $G$ acting on $(S^3, Γ)$ for all such $Γ\subset S^3$. Indeed we will classify all graphs $Γ\subset S^3$ which realize these $m_g$ in different levels: as abstract graphs and as spatial graphs, as well as their group actions. Such maximum orders without the condition "orientation-preserving" are also addressed.

math.GT↗

arXiv2017

Bordered surfaces in the 3-sphere with maximum symmetry

We consider orientation-preserving actions of finite groups $G$ on pairs $(S^3, Σ)$, where $Σ$ denotes a compact connected surface embedded in $S^3$. In a previous paper, we considered the case of closed, necessarily orientable surfaces, determined for each genus $g>1$ the maximum order of such a $G$ for all embeddings of a surface of genus $g$, and classified the corresponding embeddings. In the present paper we obtain analogous results for the case of bordered surfaces $Σ$ (i.e. with non-empty boundary, orientable or not). Now the genus $g$ gets replaced by the algebraic genus $α$ of $Σ$ (the rank of its free fundamental group); for each $α> 1$ we determine the maximum order $m_α$ of an action of $G$, classify the topological types of the corresponding surfaces (topological genus, number of boundary components, orientability) and their embeddings into $S^3$. For example, the maximal possibility $12(α- 1)$ is obtained for the finitely many values $α= 2, 3, 4, 5, 9, 11, 25, 97, 121$ and $241$.

math.GT↗

arXiv2017

Embedding compact surfaces into the 3-dimensional Euclidean space with maximum symmetry

The symmetries of surfaces which can be embedded into the symmetries of the 3-dimensional Euclidean space $\mathbb{R}^3$ are easier to feel by human's intuition. We give the maximum order of finite group actions on $(\mathbb{R}^3, Σ)$ among all possible embedded closed/bordered surfaces with given geometric/algebraic genus $>1$ in $\mathbb{R}^3$. We also identify the topological types of the bordered surfaces realizing the maximum order, and find simple representative embeddings for such surfaces.

math.GT↗

arXiv2016

Embedding surfaces into $S^3$ with maximum symmetry

We restrict our discussion to the orientable category. For $g > 1$, let $OE_g$ be the maximum order of a finite group $G$ acting on the closed surface $Σ_g$ of genus $g$ which extends over $(S^3, Σ_g)$, where the maximum is taken over all possible embeddings $Σ_g\hookrightarrow S^3$. We will determine $OE_g$ for each $g$, indeed the action realizing $OE_g$. In particular, with 23 exceptions, $OE_g$ is $4(g+1)$ if $g\ne k^2$ or $4(\sqrt{g}+1)^2$ if $g=k^2$, and moreover $OE_g$ can be realized by unknotted embeddings for all $g$ except for $g=21$ and $481$.

math.GT↗

arXiv2015

Maximum Orders of Cyclic and Abelian Extendable Actions on Surfaces

Let $Σ_g (g>1)$ be a closed surface embedded in $S^3$. If a group $G$ can acts on the pair $(S^3, Σ_g)$, then we call such a group action on $Σ_g$ extendable over $S^3$. In this paper we show that the maximum order of extendable cyclic group actions is $4g+4$ when $g$ is even and $4g-4$ when $g$ is odd; the maximum order of extendable abelian group actions is $4g+4$. We also give results of similar questions about extendable group actions over handlebodies.

math.GT↗

arXiv2015

Alternating Heegaard diagrams and Williams solenoid attractors in 3--manifolds

We find all Heegaard diagrams with the property "alternating" or "weakly alternating" on a genus two orientable closed surface. Using these diagrams we give infinitely many genus two 3--manifolds, each admits an automorphism whose non-wondering set consists of two Williams solenoids, one attractor and one repeller. These manifolds contain half of Prism manifolds, Poincaré's homology 3--sphere and many other Seifert manifolds, all integer Dehn surgeries on the figure eight knot, also many connected sums. The result shows that many kinds of 3--manifolds admit a kind of "translation" with certain stability.

math.GT↗

arXiv2013

Embedding periodic maps on surfaces into those on $S^3$

Call a periodic map $h$ on the closed orientable surface $Σ_g$ extendable if $h$ extends to a periodic map over the pair $(S^3, Σ_g)$ for possible embeddings $e: Σ_g\to S^3$. We determine the extendabilities for all periodical maps on $Σ_2$. The results involve various orientation preserving/reversing behalves of the periodical maps on the pair $(S^3, Σ_g)$. To do this we first list all periodic maps on $Σ_2$, and indeed we exhibit each of them as a composition of primary and explicit symmetries, like rotations, reflections and antipodal maps, which itself should be an interesting piece. A by-product is that for each even $g$, the maximum order periodic map on $Σ_g$ is extendable, which contrasts sharply to the situation in orientation preserving category.

math.GT↗

arXiv2012

Extending finite group actions on surfaces over $S^3$

Let $OE_g$ (resp. $CE_g$ and $AE_g$) and resp. $OE^o_g$ be the maximum order of finite (resp. cyclic and abelian) groups $G$ acting on the closed orientable surfaces $Σ_g$ which extend over $(S^3, Σ_g)$ among all embeddings $Σ_g\to S^3$ and resp. unknotted embeddings $Σ_g\to S^3$. It is known that $OE^o_g\le 12(g-1)$, and we show that $12(g-1)$ is reached for an unknotted embedding $Σ_g \to S^3$ if and only if $g = 2$, 3, 4, 5, 6, 9, 11, 17, 25, 97, 121, 241, 601. Moreover $AE_g$ is $2g+2$; and $CE_g$ is $2g+2$ for even $g$, and $2g-2$ for odd $g$. Efforts are made to see intuitively how these maximal symmetries are embedded into the symmetries of the 3-sphere.

math.GT↗