arXiv subjects

Zhen Zhang

Publications and source records attributed to Zhen Zhang.

At least 163 records · Page 9Linked to original sources

arXiv2026

TTrace: Lightweight Error Checking and Diagnosis for Distributed Training

Distributed training is essential for scaling the training of large neural network models, such as large language models (LLMs), across thousands of GPUs. However, the complexity of distributed training programs makes them particularly prone to silent bugs, which do not produce explicit error signals but lead to incorrect training outcomes. Effectively detecting and localizing such silent bugs in distributed training is challenging. Common debugging practices based on monitoring training loss or gradient norm curves are indirect, inefficient, and provide no way to localize bugs. To address those challenges, we design and implement TTrace, the first systematic differential testing system for detecting and localizing silent bugs in distributed training. TTrace aligns intermediate tensors from distributed training with those from a trusted reference implementation. To properly compare the floating-point values in the corresponding tensors, we propose a novel mathematical analysis that provides a guideline for setting tolerances, enabling TTrace to distinguish bug-induced errors from numerical errors. Experimental results demonstrate that TTrace effectively detects 11 existing bugs and 3 new bugs in the widely used Megatron-LM framework, while requiring fewer than 10 lines of code changes. TTrace is effective in various training recipes, including low-precision recipes involving BF16 and FP8. Notably, a popular open-source training framework has already adopted the method proposed by TTrace in its development workflow.

cs.DC↗

arXiv2026

MMFormalizer: Multimodal Autoformalization in the Wild

Autoformalization, which translates natural language mathematics into formal statements to enable machine reasoning, faces fundamental challenges in the wild due to the multimodal nature of the physical world, where physics requires inferring hidden constraints (e.g., mass or energy) from visual elements. To address this, we propose MMFormalizer, which extends autoformalization beyond text by integrating adaptive grounding with entities from real-world mathematical and physical domains. MMFormalizer recursively constructs formal propositions from perceptually grounded primitives through recursive grounding and axiom composition, with adaptive recursive termination ensuring that every abstraction is supported by visual evidence and anchored in dimensional or axiomatic grounding. We evaluate MMFormalizer on a new benchmark, PhyX-AF, comprising 115 curated samples from MathVerse, PhyX, Synthetic Geometry, and Analytic Geometry, covering diverse multimodal autoformalization tasks. Results show that frontier models such as GPT-5 and Gemini-3-Pro achieve the highest compile and semantic accuracy, with GPT-5 excelling in physical reasoning, while geometry remains the most challenging domain. Overall, MMFormalizer provides a scalable framework for unified multimodal autoformalization, bridging perception and formal reasoning. To the best of our knowledge, this is the first multimodal autoformalization method capable of handling classical mechanics (derived from the Hamiltonian), as well as relativity, quantum mechanics, and thermodynamics. More details are available on our project page: MMFormalizer.github.io

cs.CL↗

arXiv2025

AutoForge: Automated Environment Synthesis for Agentic Reinforcement Learning

Conducting reinforcement learning (RL) in simulated environments offers a cost-effective and highly scalable way to enhance language-based agents. However, previous work has been limited to semi-automated environment synthesis or tasks lacking sufficient difficulty, offering little breadth or depth. In addition, the instability of simulated users integrated into these environments, along with the heterogeneity across simulated environments, poses further challenges for agentic RL. In this work, we propose: (1) a unified pipeline for automated and scalable synthesis of simulated environments associated with high-difficulty but easily verifiable tasks; and (2) an environment level RL algorithm that not only effectively mitigates user instability but also performs advantage estimation at the environment level, thereby improving training efficiency and stability. Comprehensive evaluations on agentic benchmarks, including tau-bench, tau2-Bench, and VitaBench, validate the effectiveness of our proposed method. Further in-depth analyses underscore its out-of-domain generalization.

cs.CL↗

arXiv2025

The Quest for Winning Tickets in Low-Rank Adapters

The Lottery Ticket Hypothesis (LTH) suggests that over-parameterized neural networks contain sparse subnetworks ("winning tickets") capable of matching full model performance when trained from scratch. With the growing reliance on fine-tuning large pretrained models, we investigate whether LTH extends to parameter-efficient fine-tuning (PEFT), specifically focusing on Low-Rank Adaptation (LoRA) methods. Our key finding is that LTH holds within LoRAs, revealing sparse subnetworks that can match the performance of dense adapters. In particular, we find that the effectiveness of sparse subnetworks depends more on how much sparsity is applied in each layer than on the exact weights included in the subnetwork. Building on this insight, we propose Partial-LoRA, a method that systematically identifies said subnetworks and trains sparse low-rank adapters aligned with task-relevant subspaces of the pre-trained model. Experiments across 8 vision and 12 language tasks in both single-task and multi-task settings show that Partial-LoRA reduces the number of trainable parameters by up to 87\%, while maintaining or improving accuracy. Our results not only deepen our theoretical understanding of transfer learning and the interplay between pretraining and fine-tuning but also open new avenues for developing more efficient adaptation strategies.

cs.LG↗

arXiv2025

Decomposing Task Vectors for Refined Model Editing

Large pre-trained models have transformed machine learning, yet adapting these models effectively to exhibit precise, concept-specific behaviors remains a significant challenge. Task vectors, defined as the difference between fine-tuned and pre-trained model parameters, provide a mechanism for steering neural networks toward desired behaviors. This has given rise to large repositories dedicated to task vectors tailored for specific behaviors. The arithmetic operation of these task vectors allows for the seamless combination of desired behaviors without the need for large datasets. However, these vectors often contain overlapping concepts that can interfere with each other during arithmetic operations, leading to unpredictable outcomes. We propose a principled decomposition method that separates each task vector into two components: one capturing shared knowledge across multiple task vectors, and another isolating information unique to each specific task. By identifying invariant subspaces across projections, our approach enables more precise control over concept manipulation without unintended amplification or diminution of other behaviors. We demonstrate the effectiveness of our decomposition method across three domains: improving multi-task merging in image classification by 5% using shared components as additional task vectors, enabling clean style mixing in diffusion models without generation degradation by mixing only the unique components, and achieving 47% toxicity reduction in language models while preserving performance on general knowledge tasks by negating the toxic information isolated to the unique component. Our approach provides a new framework for understanding and controlling task vector arithmetic, addressing fundamental limitations in model editing operations.

cs.LG↗

arXiv2025

Data-Driven Nonconvex Reachability Analysis using Exact Multiplication

This paper addresses a fundamental challenge in data-driven reachability analysis: accurately representing and propagating non-convex reachable sets. We propose a novel approach using constrained polynomial zonotopes to describe reachable sets for unknown LTI systems. Unlike constrained zonotopes commonly used in existing literature, constrained polynomial zonotopes are closed under multiplication with constrained matrix zonotopes. We leverage this property to develop an exact multiplication method that preserves the non-convex geometry of reachable sets without resorting to approximations. We demonstrate that our approach provides tighter over-approximations of reachable sets for LTI systems compared to conventional methods.

eess.SY↗

arXiv2025

A nanopore-gated sub-attoliter silicon nanocavity for single-molecule trapping and analysis

Biomolecules exhibit dynamic conformations critical to their functions, yet observing these processes at the single-molecule level under native conditions remains a formidable challenge. While surface immobilization has been widely used to extend observation times, it could disrupt molecular dynamics and impede biological function. Moreover, the study of weak molecular interactions requires high local concentrations, often leading to problems with signal saturation in fluorescence-based approaches. Recent advancements in single-molecule trapping techniques have addressed some limitations, but achieving precise, controllable, long-term trapping in a molecularly crowded environment without external forces remains difficult. Here, we introduce a nanopore-gated sub-attoliter silicon nanocavity that enables precise, non-perturbative trapping of individual biomolecules for extended observation times, eliminating the need for external forces. Using nucleosomes as model systems, we demonstrate single-molecule Förster resonance energy transfer (smFRET) to monitor relative distances. Our data also show that an applied electric field can modulate the conformational properties of macromolecules, emphasizing a key advantage of our device: it does not require an electric field to retain trapped molecules. We envision this nanocavity platform as a powerful tool for interrogating molecular dynamics in physiologically relevant environments, offering unperturbed access to weak and transient interactions that are central to biological regulation.

physics.ins-det↗

arXiv2025

Prefix Trees Improve Memory Consumption in Large-Scale Continuous-Time Stochastic Models

Highly-concurrent system models with vast state spaces like Chemical Reaction Networks (CRNs) that model biological and chemical systems pose a formidable challenge to cutting-edge formal analysis tools. Although many symbolic approaches have been presented, transient probability analysis of CRNs, modeled as Continuous-Time Markov Chains (CTMCs), requires explicit state representation. For that purpose, current cutting-edge methods use hash maps, which boast constant average time complexity and linear memory complexity. However, hash maps often suffer from severe memory limitations on models with immense state spaces. To address this, we propose using prefix trees to store states for large, highly concurrent models (particularly CRNs) for memory savings. We present theoretical analyses and benchmarks demonstrating the favorability of prefix trees over hash maps for very large state spaces. Additionally, we propose using a Bounded Model Checking (BMC) pre-processing step to impose a variable ordering to further improve memory usage along with preliminary evaluations suggesting its effectiveness. We remark that while our work is motivated primarily by the challenges posed by CRNs, it is generalizable to all CTMC models.

cs.DS↗

arXiv2025

Revisit the periodicity of SGR J1935+2154 bursts with updated sample

Since FRB 200428 has been found to be associated with an X-ray burst from the Galactic magnetar SGR J1935+2154, it is interesting to explore whether the magnetar bursts also follow the similar active periodic behavior as some repeating FRBs. Previous studies show that there is possible period about 230 day in SGR J1935+2154 bursts. Here, we collected an updated burst sample from SGR J1935+2154, including all bursts reported by Fermi/GBM and GECAM till 2022 January. We also developed a targeted search pipeline to reveal more bursts from SGR J1935+2154 in the Fermi/GBM data from 2008 August to 2014 December (i.e. before the first burst detected by Swift/BAT). With this burst sample, we re-analyzed the possible periodicity of SGR J1935+2154 bursts using the Period Folding and Lomb-Scargle Periodogram methods. Our results show that the periodicity $\sim$238 day reported in literature is probably fake and the observation effects may introduce false periods (i.e. 55 day) according to simulation tests. We find that, for the current burst sample, the most probable period is 126.88$\pm$2.05 day, which could be interpreted as the precession of the magnetar. However, we note that the whole burst history is very complicated and difficult to be perfectly accommodated with any period reported thus far, therefore more monitoring observations of SGR J1935+2154 are required to test any periodicity hypothesis.

astro-ph.HE↗

arXiv2025

Probing formation and epitaxy of ultrathin Titanium Silicide using low and medium energy ion scattering

Titanium silicide is a key contact material in advanced three-dimensional semiconductor device architectures. Here, we examine the formation of ultrathin Ti-silicide on Si(100) using a combination of non-destructive in-situ and ex-situ ion scattering techniques capable of resolving composition and structure at the nanoscale. In-situ Time-of-Flight Low-Energy Ion Scattering (ToF-LEIS) indicates intermixing after annealing at 350 °C, with further compositional changes after annealing at 500 °C, including the emergence of a Si terminating layer at the surface. Consecutive ex-situ Time-of-Flight Medium-Energy Ion Scattering (ToF-MEIS) reveals a Ti-rich polycrystalline surface layer and a Si-rich interface layer exhibiting strong ordering along the Si [100] axis. High-Resolution Transmission Electron Microscopy (HR-TEM) images confirm these findings, revealing a $\approx$1.5 nm thick epitaxial silicide layer at the interface. The presence of an epitaxial interface is particularly promising for minimizing contact resistivity in ultrathin contact layers, where interfacial order can dominate electronic performance. In addition, both ToF-MEIS and HR-TEM unveil significant variations in the thickness of the silicide layer, with a substantial interface roughness but no translation of this roughness to the surface.

cond-mat.mtrl-sci↗

arXiv2025

Effects of mean-field momentum dependence on pion production in intermediate-energy heavy-ion collisions

Pion production in heavy-ion collisions at intermediate energies provides an important probe of the collision dynamics and nuclear matter equation of state, especially the high-density behavior of the symmetry energy. Using the lattice Boltzmann-Uehling-Uhlenbeck transport model with a recently developed nuclear effective interaction based on the so-called N$5$LO Skyrme pseudopotential, we investigate the effects of the momentum dependence of nucleon mean-field potentials on the pion production in Au+Au collisions at a beam energy of $1.23$~GeV/nucleon. We find that a stronger momentum dependence, for which the nucleon mean-field potentials increase faster with momentum, generally suppresses pion production. This feature can be understood in terms of the mean-field-induced modification of nucleon high-momentum phase space during the compression stage: a stronger momentum dependence can reduce the relative fraction of high-momentum nucleons in heavy-ion collisions, thereby suppressing the production of $Δ$ resonances and pions.

nucl-th↗

arXiv2025

EcomBench: Towards Holistic Evaluation of Foundation Agents in E-commerce

Foundation agents have rapidly advanced in their ability to reason and interact with real environments, making the evaluation of their core capabilities increasingly important. While many benchmarks have been developed to assess agent performance, most concentrate on academic settings or artificially designed scenarios while overlooking the challenges that arise in real applications. To address this issue, we focus on a highly practical real-world setting, the e-commerce domain, which involves a large volume of diverse user interactions, dynamic market conditions, and tasks directly tied to real decision-making processes. To this end, we introduce EcomBench, a holistic E-commerce Benchmark designed to evaluate agent performance in realistic e-commerce environments. EcomBench is built from genuine user demands embedded in leading global e-commerce ecosystems and is carefully curated and annotated through human experts to ensure clarity, accuracy, and domain relevance. It covers multiple task categories within e-commerce scenarios and defines three difficulty levels that evaluate agents on key capabilities such as deep information retrieval, multi-step reasoning, and cross-source knowledge integration. By grounding evaluation in real e-commerce contexts, EcomBench provides a rigorous and dynamic testbed for measuring the practical capabilities of agents in modern e-commerce.

cs.AI↗

arXiv2025

Tetragonal Fe2O: the stable iron oxide at Earth's core conditions

The Fe-O system is fundamental to understanding the composition and properties of the Earth's core. Recent studies have suggested the possible existence of stable, iron-rich FenO compounds at around 215 GPa. Here, we performed crystal-structure searches and fully anharmonic free-energy calculations to investigate the Fe-FeO system under inner-core conditions. We identified Fe2O as a stable phase and constructed its high P-T phase diagram. Fe2O undergoes a hexagonal-to-tetragonal transition with increasing pressure and temperature. It remains thermodynamically stable against decomposition into Fe and FeO from 200 to 400 GPa and at high temperatures. Although oxygen has been considered nearly absent in the inner core due to its limited solubility, these results suggest that oxygen can, in fact, be incorporated into the solid inner core in the form of an Fe+Fe2O mixture, and can match PREM densities for 53 mol% Fe2O. Our work has the potential to lead to a significant revision of the current understanding of the core's structure and composition.

cond-mat.mtrl-sci↗

arXiv2025

Probabilistic Verification for Modular Network-on-Chip Systems (extended version)

Quantitative verification can provide deep insights into reliable Network-On-Chip (NoC) designs. It is critical to understanding and mitigating operational issues caused by power supply noise (PSN) early in the design process: fluctuations in network traffic in modern NoC designs cause dramatic variations in power delivery across the network, leading to unreliability and errors in data transfers. Further complicating these challenges, NoC designs vary widely in size, usage, and implementation. This case study paper presents a principled, systematic, and modular NoC modeling approach using the Modest language that closely reflects the standard hierarchical design approach in digital systems. Using the Modest Toolset, functional and quantitative correctness was established for several NoC models, all of which were instantiated from a generic modular router model. Specifically, this work verifies the functional correctness of a generic router, inter-router communication, and the entire NoC. Statistical model checking was used to verify PSN-related properties for NoCs of size up to 8x8.

cs.LO↗

arXiv2025

A novel phase-field model for $N$-phase problems: modeling, asymptotic analysis and numerical simulations

The classical phase-field modeling approaches for multiphase problems represent each phase using a regularized characteristic function, which necessarily introduces a simplex constraint for the phase-field variables. Additionally, the consistency requirement for phase-field modeling brings difficulties to the construction of nonlinear potentials in the energy functionals, posing significant challenges for classical phase-field modeling and its numerical methods for problems involving many phases. In this work, by adopting a dichotomic approach to represent multiphase, we propose a novel phase-field modeling framework without simplex constraint,in which the free energy is interpolated from the classical two-phase Ginzburg-Landau free energies. We systematically establish the interpolation rules and explicitly construct the interpolation functions, rendering the consistency properties of the model. The proposed model enjoys an energy dissipation property and is shown to be asymptotically consistent with its sharp interface limit, with the Neumann triangle condition recovered at the triple junction.Based on a mobility operator splitting technique, we develop a linear, decoupled, and energy stable scheme for efficiently solving the system of phase-field equations. The numerical stability and accuracy, as well as the consistency properties of the model, are validated through a large number of numerical examples. In particular, the model demonstrates its success in several benchmark simulations for multiphase problems, such as the formation of liquid lenses between two stratified fluids, the generation of double emulsions and Janus emulsions, showing good agreement with experimental observations.

math.NA↗

arXiv2025

Beam Halo Formation via Longitudinal-Transverse Coupling in Continuous-Wave Photoinjectors

Beam halo formation poses a critical challenge for high-repetition-rate continuous-wave (CW) free-electron lasers (FELs), directly affecting beam quality and machine protection, as observed during the LCLS-II commissioning. We identify and experimentally validate a previously unrecognized three-step mechanism for halo generation in the photoinjector, arising from coupled longitudinal-transverse dynamics in the low-energy beam. Theoretical analysis reveals that (i) the RF buncher induces an energy-radius correlation, (ii) velocity bunching transforms this correlation into hollowed density structures in the bunch head and tail, and (iii) differential overfocusing of these hollowed regions by downstream focusing forms the observed halo. This mechanism is confirmed by particle-in-cell simulations and direct experimental measurements, including controlled formation of a core-ring profile via solenoid tuning. The results establish the physical origin of the halo and demonstrate a mitigation via buncher compression tuning that reduces halo and downstream loss, supporting sustained high-repetition-rate FEL operation.

physics.acc-ph↗

arXiv2025

Differentiable Fast Top-K Selection for Large-Scale Recommendation

Cascade ranking is a widely adopted paradigm in large-scale information retrieval systems for Top-K item selection. However, the Top-K operator is non-differentiable, hindering end-to-end training. Existing methods include Learning-to-Rank approaches (e.g., LambdaLoss), which optimize ranking metrics like NDCG and suffer from objective misalignment, and differentiable sorting-based methods (e.g., ARF, LCRON), which relax permutation matrices for direct Top-K optimization but introduce gradient conflicts through matrix aggregation. A promising alternative is to directly construct a differentiable approximation of the Top-K selection operator, bypassing the use of soft permutation matrices. However, even state-of-the-art differentiable Top-K operator (e.g., LapSum) require $O(n \log n)$ complexity due to their dependence on sorting for solving the threshold. Thus, we propose DFTopK, a novel differentiable Top-K operator achieving optimal $O(n)$ time complexity. By relaxing normalization constraints, DFTopK admits a closed-form solution and avoids sorting. DFTopK also avoids the gradient conflicts inherent in differentiable sorting-based methods. We evaluate DFTopK on both the public benchmark RecFLow and an industrial system. Experimental results show that DFTopK significantly improves training efficiency while achieving superior performance, which enables us to scale up training samples more efficiently. In the online A/B test, DFTopK yielded a +1.77% revenue lift with the same computational budget compared to the baseline. To the best of our knowledge, this work is the first to introduce differentiable Top-K operators into recommendation systems and the first to achieve theoretically optimal linear-time complexity for Top-K selection. We have open-sourced our implementation to facilitate future research in both academia and industry.

cs.LG↗

arXiv2025

Competing Magnetic Phases in Li-Fe-Ge Kagome Systems

Competing interlayer magnetic interactions in kagome magnets can lead to diverse magnetic phases, which enable various promising topological or quantum material properties. Here, the electronic structure and magnetic properties have been studied using first-principles calculations for the LiFe$_6$Ge$_6$, LiFe$_6$Ge$_4$, and LiFe$_6$Ge$_5$ compounds sharing the kagome Fe$_3$Ge layer motif but with different interlayer arrangements. For LiFe$_6$Ge$_4$ and LiFe$_6$Ge$_5$, the predicted magnetic ground states are collinear antiferromagnetic (AFM) states involving a mix of ferromagnetic (FM) and AFM interlayer orientations. Whereas for LiFe$_6$Ge$_6$, an incommensurate cycloidal spin spiral is stabilized as a ground state, being close to a collinear A-type AFM state. The analysis of magnetic RKKY exchange coupling confirms the results of electronic structure calculations. The values of atomic magnetic moments are in good agreement with existing experimental estimations. Our experiments on LiFe$_6$Ge$_6$ single crystals have observed AFM ordering at ~540 K and transition to another magnetic phase, with a small FM component (possibly with spin canting), below ~270 K. Thus, our theory and experiment suggest the existence and sequence of collinear and noncollinear magnetic states in kagome LiFe$_6$Ge$_6$. Our findings provide a platform for exploring various novel magnetic phases and associated unconventional or topological magnetism.

cond-mat.mtrl-sci↗