arXiv subjects

Zhenyu Zhang

Publications and source records attributed to Zhenyu Zhang.

At least 37 records · Page 2Linked to original sources

arXiv2026

Accelerating Dense LLMs via L0-regularized Mixture-of-Experts

Large language models (LLMs) achieve strong performance but suffer from slow and costly inference. Existing acceleration methods often lead to noticeable performance degradation, while Mixture-of-Experts (MoE) models require extensive computational resources. In this paper, we propose L0-MoE, a lightweight MoE approach using L0-regularization to accelerate dense LLMs nearly without performance loss. Our method introduces a cluster confusion matrix for domain-aware dataset curation and applies dynamic batching for efficient training. Experiments show that L0-MoE achieves up to 2.5x speedup over dense models while maintaining competitive performance, outperforming existing LLM acceleration baselines.

cs.AI↗

arXiv2026

HintMiner: Automatic Question Hints Mining From Q&A Web Posts with Language Model via Self-Supervised Learning

Users often need ask questions and seek answers online. The Question - Answering (QA) forums such as Stack Overflow cannot always respond to the questions timely and properly. In this paper, we propose HintMiner, a novel automatic question hints mining tool for users to help them find answers. HintMiner leverages the machine comprehension and sequence generation techniques to automatically generate hints for users' questions. It firstly retrieve many web Q\&A posts and then extract some hints from the posts using MiningNet that is built via a language model. Using the huge amount of online Q\&A posts, we design a self-supervised objective to train the MiningNet that is a neural encoder-decoder model based on the transformer and copying mechanisms. We have evaluated HintMiner on 60,000 Stack Overflow questions. The experiment results show that the proposed approach is effective. For example, HintMiner achieves an average BLEU score of 36.17\% and an average ROUGE-2 score of 36.29\%. Our tool and experimental data are publicly available.

cs.LG↗

arXiv2026

POSPAN: Position-Constrained Span Masking for Language Model Pre-training

Span-level masked language modeling (MLM) has shown to be advantageous to pre-trained language models over the original single-token MLM, as entities/phrases and their dependencies are critical to language understanding. Previous works only consider span length with some discrete distributions, while the dependencies among spans are ignored, i.e., assuming that the positions of masked spans are uniformly distributed. In this paper, we present POSPAN, a general framework to allow diverse position-constrained span masking strategies via the combination of span length distribution and position constraint distribution, which unifies all existing span-level masking methods. To verify the effectiveness of POSPAN in pre-training, we evaluate it on the datasets from several NLU benchmarks. Experimental results indicate that the position constraint is capable of enhancing span-level masking broadly, and our best POSPAN setting consistently outperforms its span-length-only counterparts and vanilla MLM. We also conduct theoretical analysis for the position constraint in masked language models to shed light on the reason why POSPAN works well, demonstrating the rationality and necessity of POSPAN.

cs.LG↗

arXiv2026

Thermodynamics of Kerr-Newman-Bertotti-Robinson black holes

In this work, we extend the thermodynamic analyses of the neutral and specially charged Kerr-Bertotti-Robinson black holes to the general Kerr-Newman-Bertotti-Robinson family, in which the electric and external-field parameters are independent. Using covariant surface charges and canonical integrability methods, we determine the total angular momentum and the canonical mass. The angular momentum follows analytically from the combined gravitational and electromagnetic surface charges. However, the infinitesimal charge associated with coordinate time translations is not integrable in solution space, so the mass must be associated with a more general symmetry generator. Imposing the canonical integrability conditions on this generator, together with the Kerr-Newman mass as the zero-field boundary condition, selects the Christodoulou-Ruffini mass and determines the associated thermodynamic potentials. The first law and Smarr formula take the same form as the ones in usual Kerr-Newman case, and the two previously studied Kerr-Bertotti-Robinson cases are recovered as special limits.

gr-qc↗

arXiv2026

MedQA-MM: Shortcuts Behind Medical Visual Reasoning

A benchmark score credits final answers, but not the route by which an item can be answered. In medical multimodal multiple-choice questions (MCQs), this distinction matters because a correct answer can be supported by the intended image finding or by benchmark-preserved cues in the wording of answers, non-visual clinical text, visible image text, artificial annotations, or device/context artifacts. We call the resulting score-level overinterpretation reasoning inflation. Here, a route is an observable input path that can support answer selection, not a claim about the model's hidden cognition. Across six medical multimodal MCQ datasets, we separate candidate cues from behavioral evidence through prompt- and image-side audits, modality ablations, and matched repairs that preserve the medical target and answer key. In a 13-configuration open-model panel, full-input accuracy is 62.63%, while text-only and options-only settings achieve 53.96% and 29.71%, respectively. Removing length-gap, absolute/conspicuous, and spatial/prepositional cues lowers accuracy by 6.58, 3.50, and 4.77 percentage points. We also construct MedQA-MM, a 1,000-item shortcut-mitigated subset, where text-only and options-only accuracy fall to 5.21% and 12.33%. This does not imply that models never use images; it shows that medical image-reasoning claims require route-level evidence.

cs.CV↗

arXiv2026

Shadows and Thin-Disk Images of Kerr-Newman Black Holes in a Bertotti-Robinson Magnetic Field

In this paper, we investigate the optical properties of Kerr-Newman-Bertotti-Robinson (KN-BR) black holes. We use the separability of null geodesics to analyze unstable spherical photon orbits and determine the radial extent of the photon shell. Because the spacetime is not asymptotically flat, we construct the critical curve on the screen of a finite-distance zero-angular-momentum observer. We then perform backward ray tracing for a geometrically thin and optically thin disk that extends from the outer region to the event horizon, and examine the resulting images, intensity profiles, critical-curve areas, and inner-shadow areas. We find that the genuine neutral Kerr-BR$_0$ and specially charged Kerr-BR$_s$ configurations have nearly identical optical appearances. It is remarkable that for the KN-BR black holes increasing the electric charge reduces the characteristic image size in the Kerr-Newman limit but enlarges it in the magnetized configurations considered here. We also find that the external magnetic field strongly increases the apparent image scale, while the observer inclination affects the inner-shadow area more significantly than the critical-curve area. These results may provide useful theoretical insight for future observations aimed at identifying such exotic magnetized black holes.

gr-qc↗

arXiv2026

ChainSWE: Benchmarking Coding Agents on Multi-Bug Software Maintenance

Language model (LM) agents are increasingly deployed to maintain codebases over extended periods, fixing streams of related defects while carrying context from one fix to the next. Yet existing software engineering (SWE) benchmarks evaluate models one bug at a time: the repository is reset, the codebase is re-read, and a single self-contained issue is graded in isolation. This setting collapses a continuous maintenance workflow into a series of independent sessions, ignoring the cumulative dependencies that make real-world bug fixing challenging. To bridge this gap, we introduce ChainSWE, the first benchmark for evaluating agents on sequential, dependent bug fixes within a shared codebase. We collect chronological chains of 304 issues across 54 Python projects, mined from six SWE-bench-family datasets. Our evaluation across a range of agents and models reveals a consistent performance drop by up to 70% as the chain length increases.

cs.SE↗

arXiv2026

Machine Learning Unveils Finite-volume Energy Shifts in Three-body System

Finite-volume extrapolation (FVE) is essential for extracting physical observables in the lattice calculation. While rigorous FVE formulations are well established for short-range potentials in both two- and three-body systems, long-range interactions with force ranges comparable to the lattice size $L$ remain challenging. Extending a previous data-driven scheme for two-body systems, we apply symbolic regression (PySR) to uncover universal three-body FVE formulae. For short-range potentials, we reproduce the two limiting cases, i.e. $κ_3\ggκ_2$ and $κ_3\simκ_2$. For pure long-range potentials, we obtain a dedicated analytic expression, and after incorporating short-range contributions, we uncover a unified formula consistent with the original PySR solution, which performs excellently in the intermediate force range around 1 fm. This work demonstrates that combining machine learning with physical constraints can yield novel analytical results inaccessible to conventional theoretical tools, advancing data-driven methodologies in hadron physics.

hep-lat↗

arXiv2026

Memory Augmentation Unlocks Efficient Chain-of-Thought Reasoning

Large language models often rely on Chain-of-Thought (CoT) reasoning to solve complex tasks, but verbose reasoning traces introduce substantial inference overhead. CoT compression shortens generation, yet aggressive compression may disrupt logical coherence and degrade performance. We formalize this trade-off as the Context-Generation Substitution Law, where explicit reasoning context substitutes for part of decode-time generation. Based on this principle, we propose Memory-Augmented Compression, a training-free framework that constructs reusable reasoning memories from historical traces and retrieves them as prefill-side scaffolds. Rather than using raw demonstrations, these memories summarize reusable reasoning patterns, key constraints, and critical operations to compensate for information lost during compression. Experiments show that Memory consistently improves prompt-based Chain-of-Draft (CoD) compression across mathematical reasoning, complex reasoning, and science question answering tasks, yielding accuracy gains of 21.4, 28.0, 29.5, and 6.61 points over CoD on GSM8K, MATH, BBH, and MMLU-Sci, while achieving a 1.14-1.49x latency speedup latency speedup over standard CoT. Memory is also compatible with token-level, reasoning-trace-level, and inference-state compression mechanisms.

cs.CL↗

arXiv2026

Advantageous Parameter Expansion Training Makes Better Large Language Models

Although scaling up the number of trainable parameters can effectively improve the training performance of large language models, it also leads to increased computational overhead. When delving into the parameter difference, we find that a subset of parameters, termed advantageous parameters, plays a crucial role in determining model performance. Further analysis reveals that stronger models tend to possess more such parameters. In this paper, we propose Advantageous Parameter EXpansion Training (APEX), a method that progressively expands advantageous parameters into the space of disadvantageous ones, thereby increasing their proportion and enhancing training effectiveness, while keeping the total parameter count unchanged. Extensive experiments on both instruction tuning and continued pre-training across five base models demonstrate that, in instruction tuning, APEX outperforms full-parameter tuning while using only 52% of the trainable parameters. In continued pre-training, APEX achieves the same perplexity level as conventional training with only approximately 30% of the training data, and yields significant improvements on downstream tasks.

cs.CL↗

arXiv2026

Ask, Condition or Abstain: Reinforcement Learning for Missing-Premise Reasoning

Answer-only reinforcement learning (RL) trains reasoning models to solve fully specified problems, but many realistic queries omit a premise needed for a unique answer. In this setting, the useful response is not always refusal: the model should ask for the missing premise, condition its answer on the unknown quantity, or abstain when no informative conditional response is available. We present \emph{Ask-Condition-Abstain Reinforcement Learning} (ACA-RL), a data-augmented RL framework for this setting. Its reasoning-graph-guided pipeline converts well-posed problems into missing-premise training instances with localized gap annotations; ACA-RL then trains on these instances with a structured reward over five observable response behaviors. We also introduce the \emph{Missing-Premise Benchmark} (MPB), a 274-instance human-verified benchmark spanning mathematical, logical, and real-world word problems. Across Qwen3 and Llama models, ACA-RL consistently improves on MPB while preserving competitive performance on well-posed reasoning tasks. Together with the released code, MPB, and training data, this work supports a new mission for NLP evaluation: measuring whether models can recognize when a task is underdetermined and handle uncertainty, not only whether they can answer fully specified questions.

cs.CL↗

arXiv2026

HD3C: Efficient Medical Data Classification for Edge Devices

Efficient medical data classification is essential for modern disease screening, particularly in resource-constrained environments where power budgets and computing capabilities are limited. We present HD3C, a lightweight classification framework designed for low-power edge devices. HD3C encodes data into high-dimensional hypervectors, aggregates them into multiple cluster prototypes, and performs classification through similarity search in hyperspace. We evaluate HD3C across three medical classification tasks; on heart sound classification, HD3C is 350x more energy-efficient than Bayesian ResNet with less than 1% difference in accuracy. Moreover, HD3C demonstrates exceptional robustness to noise, limited training data, and hardware error, supported by both theoretical analysis and empirical results, highlighting its potential for reliable deployment in real-world settings.

cs.LG↗

arXiv2026

Topological Flat Minibands and Fractional Chern Insulators in Rashba Systems with Tunable Superlattice Potentials

Fractional Chern insulators are interaction-driven topological states that realize the lattice analogue of the fractional quantum Hall effect without external magnetic fields. Here we propose a programmable platform for engineering topological flat bands by combining a Zeeman field with a tunable electrostatic superlattice potential in a Rashba spin-orbit-coupled thin film. The Zeeman field generates a nearly flat segment of the Rashba band, which is isolated by the superlattice potential into a flat band with Chern number $\mathcal{C}=1$. Using many-body exact diagonalization, we show that this band supports fractional Chern insulators at filling factors $n = 1/3$ and $2/3$, both of which remain robust over broad parameter ranges. We further identify realistic material candidates and the corresponding device conditions that enable experimental realization. These results establish a versatile and experimentally accessible platform for engineering topological flat bands and exploring correlated topological phases.

cond-mat.mes-hall↗

arXiv2026

STAGE: Controlled Objective Admission for Multi-Preference LLM Alignment

Multi-preference alignment is often framed as scalarization: combine reward dimensions, then optimize. This leaves a temporal decision underspecified: when should each preference dimension enter policy optimization? We propose \methodname, a stability-guided active-set controller for controlled objective admission. \methodname starts from a small active set, retains admitted objectives, and expands when reward-deviation gates indicate low recent deviation or a patience budget is exhausted. A probing phase estimates a hard-to-easy order, and adaptive weighting emphasizes underperforming active dimensions. Automatic evaluations with 15 training preferences and 16 held-out benchmark columns show that \methodname obtains higher averages than simultaneous scalarization and shared-budget adapted baselines. Component ablations and expansion dynamics further support cumulative retention, gated admission, and probing-derived ordering as useful design choices in this setting. These results position objective-entry timing as a concrete control variable in reward-vector RLHF.

cs.CL↗

arXiv2026

AISPA: User-Centric System Prompt Auditing for Large Language Model Applications

System prompts are instructions configured by developers to govern the behaviors of foundation models in AI applications. They are used throughout commercial AI products, but are rarely disclosed to the public or regulators, creating a serious trust and accountability gap in the wide deployment of AI systems. In this paper, we introduce Artificial Intelligence System Prompt Assurance (AISPA), a user-centric framework for systematically auditing system prompts in AI systems. AISPA examines specific parts of a system prompt and evaluates them along eight dimensions that matter to users. We then use this framework to review 3,249 instructions from system prompts in 88 commercial AI products, classifying each instruction as either protective (of users) or problematic. Our audit surfaces four core findings. First, system prompt design varies substantially across products and developers, with some organizations averaging over 60 protective instructions per product while others average fewer than 5. Second, protective instructions are widely adopted but shallow in scope: 98.9% of products contain at least one, yet only 24% cover all eight dimensions of the AISPA taxonomy. Third, system prompts have grown steadily longer and more protective of users, suggesting that user protection is becoming a more visible concern in commercial prompt design. Fourth, despite this progress, problematic instructions remain pervasive: roughly 40% of products contain at least one instruction that works against user interests, and protective and problematic instructions frequently coexist within the same prompt. Our findings highlight the need for greater transparency, standardization, and independent oversight for system prompts in commercial AI products.

cs.AI↗

arXiv2026

TokTier: Exact Stateful CPU+GPU Tokenization for Agentic LLM Serving

LLM serving caches prompt KV state, yet most front ends still re-tokenize the full request on every call. Coding agents pay most: sessions repeatedly submit a long transcript after a small append, which can shift token boundaries near the end of the prior sequence. Across 153,951 calls the median append is ~1.4K characters; only 1.0-3.6% of calls start or rebuild a session, yet those carrymulti-million-character contexts. Fleet prompt-cache hit rate is 94.1%, and as it approaches 0.99, tokenization grows from 10% to 64% of time to first token (TTFT) in component measurements. TokTier is a stateful CPU+GPU tokenization service for this two-mode workload, under one contract: emitted token IDs are always identical to full reference tokenization. For session continuations it re-tokenizes a small window around the append and splices only when a per-request check finds a stable pre-tokenization boundary; failed checks widen the window or fall back to full reference tokenization. For calls without a reusable prefix it runs exact GPT-family regex pre-tokenization and BPE on a GPU. A sampled shadow verifier re-checks live traffic. Across 17 production tokenizer families, differential campaigns cover 1.5x10^10 split checks, a 12.4 TB real-text corpus, and 93,000+ replayed agent steps, with zero divergence. Incremental repair takes 0.5-1.1 ms from 100K to 3M characters, up to 437x faster than HF tokenization and 2.1x faster at 1M characters than the strongest cache-based baseline (Gigatoken) fully prewarmed. GPU tokenization encodes a 1M-character request in 0.87 ms, up to 491x below HF and 23.4x below the fastest published CPU method on the same protocol. With vLLM, median TTFT drops 16-34% and P99 TTFT 23% under recorded bursts. Under a 50 ms P99 objective, a four-core repair pool plus one GPU sustains 1,821 requests/s, where a 16-core stateless front end saturates at 40 requests/s.

cs.CL↗

arXiv2026

Dilated coordinate method for solving nuclear lattice effective field theory

We introduce a dilated coordinate method to address computational challenges in nuclear lattice effective field theory (NLEFT) for weakly-bound few-body systems. The approach employs adaptive mesh refinement via analytic coordinate transformations, dynamically adjusting spatial resolution to resolve short-range nuclear interactions with fine grids while efficiently capturing long-range wave function tails with coarse grids. Numerical demonstrations for two- and three-body systems confirm accelerated convergence towards infinite-volume limit compared to uniform lattices, particularly beneficial for accessing highly excited states and shallow bound states near the continuum threshold. This method establishes a foundation for \textit{ab initio} studies of exotic nuclear systems near the dripline and light hypernuclei, with direct extensions to scattering and reaction processes.

nucl-th↗

arXiv2026

Can Text-to-Image Models Draw from the Right Frame of Reference?

Spatial instruction following has become a crucial requirement for text-to-image (T2I) generation. A common challenge arises when directional expressions are interpreted under different frames of reference. For example, ``the left of'' may refer to the viewer's image coordinates or to the intrinsic orientation of an object, leading to different expected layouts. Existing T2I benchmarks reveal important layout failures, yet they rarely isolate whether models can follow a specified frame of reference when it differs from camera view. To mitigate this gap, we introduce FoR-T2I, a benchmark for evaluating this distinction with 1,200 prompt pairs built from controlled spatial layouts. In each pair, the camera-view (Cam) prompt states the target relation in camera view, while the frame-of-reference (FoR) prompt describes the same target placement through an oriented anchor object. Across 22 closed-source and open-source T2I models, mean final accuracy is 41.8\% lower on FoR prompts than on matched Cam prompts; even the best-performing model achieves only 44.3\% FoR accuracy. This suggests that current models struggle more when the same layout is described through an object's orientation rather than directly in image coordinates. We further analyze this gap by relation type and camera view, compare several training-free prompting and feedback-based mitigation strategies, and propose a VLM-gated rewriting approach that selects rewritten prompts using visual feedback, improving average FoR accuracy from 25.0\% to 29.2\% under the same generation budget.

cs.CV↗