arXiv subjects

Yining Jiang

Publications and source records attributed to Yining Jiang.

7 recordsLinked to original sources

arXiv2026

AGZO: Activation-Guided Zeroth-Order Optimization for LLM Fine-Tuning

Zeroth-Order (ZO) optimization has emerged as a promising solution for fine-tuning LLMs under strict memory constraints, as it avoids the prohibitive memory cost of storing activations for backpropagation. However, existing ZO methods typically employ isotropic perturbations, neglecting the rich structural information available during the forward pass. In this paper, we identify a crucial link between gradient formation and activation structure: the gradient of a linear layer is confined to the subspace spanned by its input activations. Leveraging this insight, we propose Activation-Guided Zeroth-Order optimization (AGZO). Unlike prior methods, AGZO extracts a compact, activation-informed subspace on the fly during the forward pass and restricts perturbations to this low-rank subspace. We provide a theoretical framework showing that AGZO optimizes a subspace-smoothed objective and provably yields update directions with higher cosine similarity to the true gradient than isotropic baselines. Empirically, we evaluate AGZO on Qwen3 and Pangu models across various benchmarks. AGZO consistently outperforms state-of-the-art ZO baselines and significantly narrows the performance gap with first-order fine-tuning, while maintaining almost the same peak memory footprint as other ZO methods.

cs.LG↗

arXiv2025

Quantum-classical gravity distinction in reservoir-engineered massive quantum system

Massive quantum systems have emerged as compelling tabletop interface-systems for testing the quantum nature of gravity. However, conventional schemes that focus on directly using gravity to induce entanglement suffer from overwhelming environmental decoherence: maintaining entanglement between two oscillators requires an impractically high mechanical quality factor. In this work, we put forward an alternative reservoir-engineered scheme, whose core function is to quantify how gravity modifies (rather than prepares) the steady-state entanglement. Compared to quantum gravity, classical gravity introduces additional dissipative channels, which in turn give rise to distinct entanglement characteristics and thus enable the discrimination between the two types of gravity. Notably, this entanglement difference can still be maintained even when the mechanical quality factor is far below the threshold required by conventional schemes. Moreover, it demonstrates significant robustness against non-gravitational couplings, specifically, those like Casimir and Coulomb forces that are inherent in experimental setups. Our scheme relaxes the experimental requirements for verifying quantum gravity, thereby paving a new path toward its near-term realization.

quant-ph↗

arXiv2025

RepLLM: Toward Automatically Reproducing Network Research Results

Result reproduction of computer networking research is challenging as the scarcity of open-source implementations and the complexity of heterogeneous system architectures. Even though Large Language Models have demonstrated potential in code generation, existing code generation frameworks often fail to address the long-context constraints and intricate logical dependencies, which are vital in reproducing network systems from academic papers. Thus, we introduce RepLLM, an end-to-end multi-agent framework designed to automate code reproduction from paper content. RepLLM features a collaborative architecture comprising four specialized agents -- Content Parsing, Architecture Design, Code Generation, and Audit&Repair, which are coordinated through Shared Memory mechanism to ensure global context consistency. With the enhancement of Structured Chain-of-Thought LLM reasoning and a sandbox-isolated static-dynamic debugging methodology, our framework effectively resolves semantic discrepancies and runtime errors, thereby improving reliable reproductions. Extensive evaluations on representative papers in top conferences demonstrate that RepLLM outperforms state-of-the-art system-level LLM frameworks in generating compile-ready and logically correct systems. Our results show that, with the aid of RepLLM, we can reproduce 95% of the original benchmarks within approximately two hours while reducing token consumption by up to 10% compared with state-of-the-art baselines.

cs.NI↗

arXiv2025

DESign: Dynamic Context-Aware Convolution and Efficient Subnet Regularization for Continuous Sign Language Recognition

Current continuous sign language recognition (CSLR) methods struggle with handling diverse samples. Although dynamic convolutions are ideal for this task, they mainly focus on spatial modeling and fail to capture the temporal dynamics and contextual dependencies. To address this, we propose DESign, a novel framework that incorporates Dynamic Context-Aware Convolution (DCAC) and Subnet Regularization Connectionist Temporal Classification (SR-CTC). DCAC dynamically captures the inter-frame motion cues that constitute signs and uniquely adapts convolutional weights in a fine-grained manner based on contextual information, enabling the model to better generalize across diverse signing behaviors and boost recognition accuracy. Furthermore, we observe that existing methods still rely on only a limited number of frames for parameter updates during training, indicating that CTC learning overfits to a dominant path. To address this, SR-CTC regularizes training by applying supervision to subnetworks, encouraging the model to explore diverse CTC alignment paths and effectively preventing overfitting. A classifier-sharing strategy in SR-CTC further strengthens multi-scale consistency. Notably, SR-CTC introduces no inference overhead and can be seamlessly integrated into existing CSLR models to boost performance. Extensive ablations and visualizations further validate the effectiveness of the proposed methods. Results on mainstream CSLR datasets (i.e., PHOENIX14, PHOENIX14-T, CSL-Daily) demonstrate that DESign achieves state-of-the-art performance.

cs.CV↗

arXiv2025

Breaking Bad Molecules: Are MLLMs Ready for Structure-Level Molecular Detoxification?

Toxicity remains a leading cause of early-stage drug development failure. Despite advances in molecular design and property prediction, the task of molecular toxicity repair, generating structurally valid molecular alternatives with reduced toxicity, has not yet been systematically defined or benchmarked. To fill this gap, we introduce ToxiMol, the first benchmark task for general-purpose Multimodal Large Language Models (MLLMs) focused on molecular toxicity repair. We construct a standardized dataset covering 11 primary tasks and 660 representative toxic molecules spanning diverse mechanisms and granularities. We design a prompt annotation pipeline with mechanism-aware and task-adaptive capabilities, informed by expert toxicological knowledge. In parallel, we propose an automated evaluation framework, ToxiEval, which integrates toxicity endpoint prediction, synthetic accessibility, drug-likeness, and structural similarity into a high-throughput evaluation chain for repair success. We systematically assess 43 mainstream general-purpose MLLMs and conduct multiple ablation studies to analyze key issues, including evaluation metrics, candidate diversity, and failure attribution. Experimental results show that although current MLLMs still face significant challenges on this task, they begin to demonstrate promising capabilities in toxicity understanding, semantic constraint adherence, and structure-aware editing.

cs.AI↗

arXiv2025

A Systematic Evaluation of On-Device LLMs: Quantization, Performance, and Resources

Deploying Large Language Models (LLMs) on edge devices enhances privacy but faces performance hurdles due to limited resources. We introduce a systematic methodology to evaluate on-device LLMs, balancing capability, efficiency, and resource constraints. Through an extensive analysis of models (0.5B-14B) and seven post-training quantization (PTQ) methods on commodity hardware, we demonstrate that: 1) Heavily quantized large models consistently outperform smaller, high-precision models, with a performance threshold at ~3.5 effective bits-per-weight (BPW); 2) Resource utilization scales linearly with BPW, though power and memory footprints vary by quantization algorithm; and 3) With a reduction in model size, the primary constraint on throughput transitions from communication overhead to computational latency. We conclude by offering guidelines for optimizing LLMs in resource-constrained edge environments. Our codebase is available at https://anonymous.4open.science/r/LLMOnDevice/.

cs.LG↗

arXiv2024

Cavity-QED Simulation of a Maser beyond the Mean-Field Approximation

Based on the well-known Tavis-Cummings (TC) model of cavity quantum electrodynamics (QED), we introduce a method for quantum-mechanically simulating the dynamics of experimental masers beyond the mean-field approximation (MFA) that takes into account the spatial variation of the a.c. magnetic field of the maser's amplified microwave mode across its gain medium. The distribution in the coupling between the amplified mode and the medium's very large number (typically $10^{17}$) of spatially distributed quantum emitters can be determined straightforwardly for a given geometry and composition using an electromagnetic-field solver. Upon discretising this distribution as a histogram over a small finite number of bins, we assign -- as an approximation -- the same coupling to all emitters that fall within the same bin, where the value of this coupling equals the center value of the bin's range. With our approximate Hamiltonian arranged as a weighted sum over these bins, we generate expressions for expectation values of operators in the Heisenberg picture to second order in cumulant expansion, using the publicly available QuantumCumulants.jl package in Julia. For ten evenly spaced bins, our model, which can be run on a laptop computer, is used to simulate the recorded output from an experimental maser with a pentacene-doped para-terphenyl gain medium. We find that it replicates the quantum-mechanical features of the measured maser's dynamics, in particular its damped collective Rabi oscillations, more closely than the standard TC model under the MFA can, with an R$^2$ value of 0.774, as opposed to 0.265. Our model should thus aid the quantitative engineering of improved, optimised maser designs.

quant-ph↗