arXiv subjects

Tianqi Shen

Publications and source records attributed to Tianqi Shen.

11 recordsLinked to original sources

arXiv2026

Escaping Local Minima Provably in Non-convex Matrix Sensing: A Deterministic Framework via Simulated Lifting

Low-rank matrix sensing is a fundamental yet challenging nonconvex problem whose optimization landscape typically contains numerous spurious local minima, making it difficult for gradient-based optimizers to converge to the global optimum. Recent work has shown that over-parameterization via tensor lifting can convert such local minima into strict saddle points, an insight that also partially explains why massive scaling can improve generalization and performance in modern machine learning. Motivated by this observation, we propose a Simulated Oracle Direction (SOD) escape mechanism that simulates the landscape and escape direction of the over-parametrized space, without resorting to actually lifting the problem, since that would be computationally intractable. In essence, we designed a mathematical framework to project over-parametrized escape directions onto the original parameter space to guarantee a strict decrease of objective value from existing local minima. To the best of our knowledge, this represents the first deterministic framework that could escape spurious local minima with guarantee, especially without using random perturbations or heuristic estimates. Numerical experiments demonstrate that our framework reliably escapes local minima and facilitates convergence to global optima, while incurring minimal computational cost when compared to explicit tensor over-parameterization. We believe this framework has non-trivial implications for nonconvex optimization beyond matrix sensing, by showcasing how simulated over-parameterization can be leveraged to tame challenging optimization landscapes.

cs.LG↗

arXiv2026

Towards Understanding the Power and Limits of the Muon Optimizer: A River-Valley Perspective

Recently, Muon has gained substantial attention as an appealing alternative to Adam-like optimizers, with many works highlighting its advantages through spectral normalization and improved conditioning. Yet this positive theoretical narrative contrasts with its empirical performance in large language model (LLM) training, where Muon's gains over Adam/AdamW are often mixed, schedule-sensitive, and not uniformly superior. To address this gap, we develop a trajectory-level theory characterizing both the strengths and limitations of Muon. We introduce a mixed-spiked matrix sensing model whose sensing operator decomposes into signal, spike, and bulk components, capturing a mixture of anisotropic structure and long-tail information reminiscent of LLM training. On top of it, we adopted a river-valley perspective in which we view the landscape as composed of a river direction flowing to the desired solution and hill directions encoding nuisance or task-irrelevant information. In the momentum-free setting, we show that Muon moves faster along the information-bearing river direction during early optimization, but can converge much more slowly near the river bottom than gradient descent. We then extend the river-valley perspective to general nonconvex objectives with momentum by studying points on the spectral river. There, while Muon converges faster early on, its orthogonalized update removes residual scale information, making it prone to overshooting and oscillation near the target solution. Together, these results suggest that our characterizations extend beyond spiked matrix sensing and motivate switching to GD-like refinement optimizers in the final phase, rather than relying only on a fixed learning-rate schedule for Muon. We also provide preliminary evidence supporting this two-stage approach in language model training experiments.

cs.LG↗

arXiv2026

Not All Thoughts Need HBM: Semantics-Aware Memory Hierarchy for LLM Reasoning

Reasoning LLMs produce thousands of chain-of-thought tokens whose KV cache must reside in scarce GPU HBM. The dominant response -- permanently evicting low-importance tokens -- is catastrophic for reasoning: accuracy collapses to 0-2.5% when half the cache is removed. We ask a different question: must every token live in HBM, or can some live elsewhere? We introduce a semantics-aware memory hierarchy that sorts tokens into four tiers -- HBM, DDR, compressed, and evicted -- using cumulative attention scoring. Low-importance tokens are moved to CPU memory rather than destroyed; before each attention step they are prefetched back at full precision, contributing exactly the same terms as if they had never left the GPU. We formalize this as zero-approximation-error offloading and derive our central finding: accuracy depends solely on how many tokens are permanently discarded (the eviction ratio), not on how many remain in HBM. A controlled 3x3 grid over HBM and eviction ratios confirms this across three model scales (7B-32B) and four benchmarks. With only 3% eviction, the hierarchy retains 91% of full-cache accuracy on GSM8K and 71% on MATH-500 (n=200); at 14B scale it matches the uncompressed baseline (90% vs. 86%) while halving HBM occupancy. A head-to-head reproduction of R-KV -- the current SOTA eviction method -- on our setup achieves only 0-32% at comparable budgets. A system prototype with real GPU-CPU data movement shows that the price of this preservation is modest -- 5-7% transfer overhead -- and scaling analysis projects 2-48 GB HBM savings at production batch sizes.

cs.CL↗

arXiv2026

AcademiClaw: When Students Set Challenges for AI Agents

Benchmarks within the OpenClaw ecosystem have thus far evaluated exclusively assistant-level tasks, leaving the academic-level capabilities of OpenClaw largely unexamined. We introduce AcademiClaw, a bilingual benchmark of 80 complex, long-horizon tasks sourced directly from university students' real academic workflows -- homework, research projects, competitions, and personal projects -- that they found current AI agents unable to solve effectively. Curated from 230 student-submitted candidates through rigorous expert review, the final task set spans 25+ professional domains, ranging from olympiad-level mathematics and linguistics problems to GPU-intensive reinforcement learning and full-stack system debugging, with 16 tasks requiring CUDA GPU execution. Each task executes in an isolated Docker sandbox and is scored on task completion by multi-dimensional rubrics combining six complementary techniques, with an independent five-category safety audit providing additional behavioral analysis. Experiments on six frontier models show that even the best achieves only a 55\% pass rate. Further analysis uncovers sharp capability boundaries across task domains, divergent behavioral strategies among models, and a disconnect between token consumption and output quality, providing fine-grained diagnostic signals beyond what aggregate metrics reveal. We hope that AcademiClaw and its open-sourced data and code can serve as a useful resource for the OpenClaw community, driving progress toward agents that are more capable and versatile across the full breadth of real-world academic demands. All data and code are available at https://github.com/GAIR-NLP/AcademiClaw.

cs.AI↗

arXiv2026

Search Multilayer Perceptron-Based Fusion for Efficient and Accurate Siamese Tracking

Siamese visual trackers have recently advanced through increasingly sophisticated fusion mechanisms built on convolutional or Transformer architectures. However, both struggle to deliver pixel-level interactions efficiently on resource-constrained hardware, leading to a persistent accuracy-efficiency imbalance. Motivated by this limitation, we redesign the Siamese neck with a simple yet effective Multilayer Perception (MLP)-based fusion module that enables pixel-level interaction with minimal structural overhead. Nevertheless, naively stacking MLP blocks introduces a new challenge: computational cost can scale quadratically with channel width. To overcome this, we construct a hierarchical search space of carefully designed MLP modules and introduce a customized relaxation strategy that enables differentiable neural architecture search (DNAS) to decouple channel-width optimization from other architectural choices. This targeted decoupling automatically balances channel width and depth, yielding a low-complexity architecture. The resulting tracker achieves state-of-the-art accuracy-efficiency trade-offs. It ranks among the top performers on four general-purpose and three aerial tracking benchmarks, while maintaining real-time performance on both resource-constrained Graphics Processing Units (GPUs) and Neural Processing Units (NPUs).

cs.CV↗

arXiv2025

Topology-Aware 3D Gaussian Splatting: Leveraging Persistent Homology for Optimized Structural Integrity

Gaussian Splatting (GS) has emerged as a crucial technique for representing discrete volumetric radiance fields. It leverages unique parametrization to mitigate computational demands in scene optimization. This work introduces Topology-Aware 3D Gaussian Splatting (Topology-GS), which addresses two key limitations in current approaches: compromised pixel-level structural integrity due to incomplete initial geometric coverage, and inadequate feature-level integrity from insufficient topological constraints during optimization. To overcome these limitations, Topology-GS incorporates a novel interpolation strategy, Local Persistent Voronoi Interpolation (LPVI), and a topology-focused regularization term based on persistent barcodes, named PersLoss. LPVI utilizes persistent homology to guide adaptive interpolation, enhancing point coverage in low-curvature areas while preserving topological structure. PersLoss aligns the visual perceptual similarity of rendered images with ground truth by constraining distances between their topological features. Comprehensive experiments on three novel-view synthesis benchmarks demonstrate that Topology-GS outperforms existing methods in terms of PSNR, SSIM, and LPIPS metrics, while maintaining efficient memory usage. This study pioneers the integration of topology with 3D-GS, laying the groundwork for future research in this area.

cs.CV↗

arXiv2021

Foresight of Graph Reinforcement Learning Latent Permutations Learnt by Gumbel Sinkhorn Network

Vital importance has necessity to be attached to cooperation in multi-agent environments, as a result of which some reinforcement learning algorithms combined with graph neural networks have been proposed to understand the mutual interplay between agents. However, highly complicated and dynamic multi-agent environments require more ingenious graph neural networks, which can comprehensively represent not only the graph topology structure but also evolution process of the structure due to agents emerging, disappearing and moving. To tackle these difficulties, we propose Gumbel Sinkhorn graph attention reinforcement learning, where a graph attention network highly represents the underlying graph topology structure of the multi-agent environment, and can adapt to the dynamic topology structure of graph better with the help of Gumbel Sinkhorn network by learning latent permutations. Empirically, simulation results show how our proposed graph reinforcement learning methodology outperforms existing methods in the PettingZoo multi-agent environment by learning latent permutations.

cs.LG↗

arXiv2014

The statistics of frictional families

We develop a theoretical description for mechanically stable frictional packings in terms of the difference between the total number of contacts required for isostatic packings of frictionless disks and the number of contacts in frictional packings, $m=N_c^0-N_c$. The saddle order $m$ represents the number of unconstrained degrees of freedom that a static packing would possess if friction were removed. Using a novel numerical method that allows us to enumerate disk packings for each $m$, we show that the probability to obtain a packing with saddle order $m$ at a given static friction coefficient $μ$, $P_m(μ)$, can be expressed as a power-series in $μ$. Using this form for $P_m(μ)$, we quantitatively describe the dependence of the average contact number on friction coefficient for static disk packings obtained from direct simulations of the Cundall-Strack model for all $μ$ and $N$.

cond-mat.soft↗

arXiv2012

Rods are less fragile than spheres: Structural relaxation in dense liquids composed of anisotropic particles

We perform extensive molecular dynamics simulations of dense liquids composed of bidisperse dimer- and ellipse-shaped particles in 2D that interact via repulsive contact forces. We measure the structural relaxation times obtained from the long-time decay of the self-part of the intermediate scattering function for the translational and rotational degrees of freedom (DOF) as a function of packing fraction ϕ, temperature T, and aspect ratio α. We are able to collapse the ϕand T-dependent structural relaxation times for disks, and dimers and ellipses over a wide range of α, onto a universal scaling function {\cal F}_{\pm}(|ϕ-ϕ_0|,T,α), which is similar to that employed in previous studies of dense liquids composed of purely repulsive spherical particles in 3D. {\cal F_{\pm}} for both the translational and rotational DOF are characterized by the α-dependent scaling exponents μand δand packing fraction ϕ_0(α) that signals the crossover in the scaling form {\cal F}_{\pm} from hard-particle dynamics to super-Arrhenius behavior for each aspect ratio. We find that the fragility at ϕ_0, m(ϕ_0), decreases monotonically with increasing aspect ratio for both ellipses and dimers. Moreover, the results for the slow dynamics of dense liquids composed of dimer- and ellipse-shaped particles are qualitatively the same, despite the fact that zero-temperature static packings of dimers are isostatic, while static packings of ellipses are hypostatic.

cond-mat.soft↗

arXiv2011

The contact percolation transition

Typical quasistatic compression algorithms for generating jammed packings of athermal, purely repulsive particles begin with dilute configurations and then apply successive compressions with relaxation of the elastic energy allowed between each compression step. It is well-known that during isotropic compression athermal systems with purely repulsive interactions undergo a jamming transition at packing fraction $ϕ_J$ from an unjammed state with zero pressure to a jammed, rigid state with nonzero pressure. Using extensive computer simulations, we show that a novel second-order-like transition, the contact percolation transition, which signals the formation of a system-spanning cluster of mutually contacting particles, occurs at $ϕ_P < ϕ_J$, preceding the jamming transition. By measuring the number of non-floppy modes of the dynamical matrix, and the displacement field and time-dependent pressure following compression, we find that the contact percolation transition also heralds the onset of complex spatiotemporal response to applied stress. Thus, highly heterogeneous, cooperative, and non-affine particle motion occurs in unjammed systems significantly below the jamming transition for $ϕ_P < ϕ< ϕ_J$, not only for jammed systems with $ϕ> ϕ_J$.

cond-mat.soft↗

arXiv2008

First-principles study of the spin-lattice coulpling in spin frustrated DyMn$_2$O$_5$

The lattice dynamic properties and spin-phonon coupling in DyMn$_2$O$_5$ are studied by using the density-functional theory. The calculated phonon frequencies are in very good agreement with experiments. We then compare the phonon modes calculated from different spin configurations. The results show that the phonon frequencies change substantially in different spin configurations, suggesting that the spin-phonon coupling in this material is very strong. Especially, the short range spin ordering has drastic effects on the highest Raman and IR phonon modes that might be responsible for the observed phonon anomalies near and above the magnetic phase transitions.

cond-mat.mtrl-sci↗