arXiv subjects

Zijun Wei

Publications and source records attributed to Zijun Wei.

At least 19 recordsLinked to original sources

arXiv2026

Athanor: An Agent-Native System for Autonomous Post-Training Research

Recursive self-improvement, in which AI systems build stronger AI systems, is one promising route to superintelligence. We study a weaker form that can be measured today: frontier agents post-training a small raw base model with public data, as a test of their research potential. We build Athanor, an open agent-native system for autonomous post-training research (AutoPostTrain) on real GPUs. We give the agents full research autonomy, low-friction engineering tools, and efficient starter code. The agents carry out the research themselves: they read datasets and evaluation results, build their training data, write and modify training algorithms, tune hyperparameters, and analyze the results to decide what to try next. They work in a sandbox and cannot use external teacher models. Athanor's services supply frozen evaluation, managed jobs, decontaminated data, a persistent research record, and a held-out suite the agents are unaware of. In a 104-hour campaign, a team of four Claude Opus 5.5 agents, each on an eight-GPU node and given no research advice, improves Qwen3-4B-Base from 15.9 to 67.3 on a six-task development suite and from 9.2 to 48.2 on a held-out suite, against 66.5 and 47.9 for the official Qwen3-4B-Instruct-2507. To our knowledge, this is the first demonstration that a team of frontier agents can autonomously post-train a raw base model into a cross-domain generalist model that matches or surpasses the official instruct version on average. The campaign's full record shows that the agents reach this level by recombining known methods and each other's models: SFT on public reasoning data, RL with a rollout change one agent introduced, weight merges across agents, and rounds of self-distillation. With execution handled, today's agents already have the engineering and research ability to conquer this task. We hope Athanor provides a foundation for scaling autonomous AI research.

cs.AI↗

arXiv2026

Factorization-Guided Training and Inference for Diffusion-Drafted Speculative Decoding

Speculative decoding accelerates large language model inference through parallel verification of tokens proposed by a drafter. Diffusion drafters further parallelize proposal generation, but greater proposal capacity or higher mean accepted tokens (MAT) need not yield higher throughput. Because verification accepts only a consecutive prefix, early rejections can render later correct predictions ineffective, limiting MAT gains, while even higher MAT may not improve throughput if per-round latency increases. We introduce DScout, which uses two complementary factorizations to guide diffusion drafter training and inference for higher throughput. For training, DScout factorizes position-wise loss into inclusion, weighting, and prediction loss. This guides the design and combination of mixed-horizon training, intra-block unmasking, and acceptance-weighted loss to align supervision with prefix acceptance and improve MAT. For inference, DScout factorizes throughput as the ratio of MAT to per-round latency, guiding target-conditioning depth, drafter depth, and inference block size selection to identify a high-throughput configuration for each workload. Across six benchmarks, DScout achieves $3.84\times$ and $4.14\times$ mean speedups over autoregressive decoding on Qwen3-4B and Qwen3-8B, respectively, improving mean speedup over the state-of-the-art baseline by 9.1% and 7.1%. Across models and concurrency settings, our inference configuration policy provides up to 9.75% additional mean speedup over DScout's fixed configuration.

cs.LG↗

arXiv2026

Diagnosing Training Inference Mismatch in LLM Reinforcement Learning via a Zero-Mismatch Reference

Modern LLM RL systems separate rollout generation from policy optimization. These two stages are expected to produce token probabilities that match exactly. However, implementation differences can make them assign different values to the same sequence under the same model weights, inducing Training-Inference Mismatch (TIM). TIM is difficult to inspect because it is entangled with off-policy drift and common stabilization mechanisms. In this work, we isolate TIM in a zero-mismatch diagnostic setting (VeXact), and show that small token-level numerical disagreements can independently cause training collapse. We further show that TIM changes the effective optimization problem, and identify a set of remedies that could mitigate TIM. Our results suggest that TIM is not benign numerical noise, but a systems-level perturbation that should be treated as a first-order factor in analyzing LLM RL stability.

cs.LG↗

arXiv2026

Where Does Staleness Accumulate? Pool Aware Effective Staleness Control for Asynchronous RL in LLM Post-Training

Fully asynchronous reinforcement learning (RL) improves resource utilization in large language model post-training by overlapping rollout generation with policy optimization, but it also introduces policy lag as trajectories are generated and queued while the trainer continues to update. We study how this lag accumulates over a trajectory's lifetime and how it can be controlled without sacrificing the wall-clock benefits of asynchronous execution. We decompose trajectory staleness into Generation Staleness, accumulated before rollout completion, and Waiting Staleness, accumulated after a completed trajectory enters the pool. Motivated by this decomposition, we introduce PACE (Pool-Aware Control of Effective Staleness). PACE converts excess pool occupancy into an adaptive rejection budget and ranks completed trajectories using an effective-staleness score that combines Waiting Staleness with prefix-aware Generation Staleness. This avoids penalizing long or interrupted rollouts solely because they span multiple policy versions. In single-turn mathematical reasoning, PACE improves the six-benchmark average validation accuracy by 18.7\% over unfiltered asynchronous RL at the same wall-clock budget and matches synchronous RL performance with 47.1\% less GPU time. PACE also improves validation performance in multi-turn tool-integrated reasoning, outperforming both synchronous and unfiltered asynchronous RL. Further experiments with the mixture-of-experts model and an alternative RL algorithm support its applicability across model architectures and training algorithms.

cs.AI↗

arXiv2026

LPNSR: Learnable Noise Prediction for Diffusion-Based Image Super-Resolution

Diffusion-based image super-resolution (SR) aims to reconstruct high-resolution (HR) images from low-resolution (LR) observations. A key property of diffusion models is that, once the starting point of the reverse chain and the denoising network are fixed, the quality of the generated image is dictated by the noise maps sampled at the intermediate steps, which are drawn from an unconstrained standard Gaussian distribution in conventional pipelines. This property raises a natural question: does there exist a noise sampling distribution better than the standard Gaussian that improves the quality of the generated images? To this end, we use a parameterized deep neural network to predict the mean and the variance of the Gaussian noise sampling distribution at each intermediate step, and design two training schemes, one supervised by the quality of the final generated image and the other by aligning each reverse step with the forward posterior. Experiments show that, compared with the standard Gaussian distribution, the learnable noise sampling distribution improves the quality of the generated images. The source code of our method can be found at https://github.com/Faze-Hsw/LPNSR.

cs.CV↗

arXiv2026

Strong Quantum Mpemba Effect from Exact Slow-Mode Selection in Constrained Rydberg Chains

CStrong quantum Mpemba acceleration requires suppressing the slowest visible Liouvillian relaxation channel, but a robust many-body mechanism for enforcing such suppression remains challenging. We identify such a mechanism in locally dephased constrained Rydberg chains through exact slow-mode selection. For constrained single-spin-flip Hamiltonians, local dephasing turns the Hamiltonian itself into an exact left Liouvillian eigenmode, $\mathcal L^\dagger(H)=-γH$. A finite-temperature reference state generically overlaps with this $H$-like slow mode, whereas translationally invariant states with $\mathrm{Tr}(Hρ_0)=0$ remove it and are confined to the $Q=0$ operator sector. When the next visible $Q=0$ mode decays faster, these selected states exhibit a strong quantum Mpemba effect. We demonstrate this mechanism in the PXP chain for a zero-energy scar eigenstate, the all-zero product state, and a translation-invariant $Z_2$ cat state, and show that it persists in the $(2,3)$ model and the longer-range blockade family. Our results identify Liouvillian mode visibility, rather than special scar wave functions, as the organizing principle for anomalously fast relaxation in constrained open quantum systems.

quant-ph↗

arXiv2026

Symmetry-Induced Relaxation Comb and Strong Quantum Mpemba Effect in Long-Range XXZ Spin Chains

We uncover a symmetry-filtered mechanism for anomalous dissipative relaxation in a long-range XXZ spin chain subject to local dephasing. At the isotropic point, the coherent Hamiltonian has global $SU(2)$ symmetry, whereas the full Liouvillian retains only the $U(1)$ symmetry associated with total magnetization. This structure pins a family of spatially uniform zero-$U(1)$-charge left eigenoperators with exact eigenvalues $λ=-2q$, forming a Liouvillian relaxation comb. For the ferromagnetic Dicke ground state, the overlap envelope on this comb is known exactly at finite size and becomes Gaussian in the large-$S$ limit. Since higher-$q$ components decay rapidly, the $q=1$ comb tooth controls the long-time dynamics and yields universal $D(t)\sim e^{-2t}$ relaxation independent of system size and interaction range. This mode-accessibility filtering realizes a spectral strong quantum Mpemba effect: an initially farther state relaxes faster than closer thermal states because slow non-steady Liouvillian modes are inaccessible. Weak breaking of the Hamiltonian $SU(2)$ symmetry restores slow-mode overlap and suppresses this acceleration.

quant-ph↗

arXiv2026

Lavida-O: Elastic Large Masked Diffusion Models for Unified Multimodal Understanding and Generation

We propose Lavida-O, a unified Masked Diffusion Model (MDM) for multimodal understanding and generation. Unlike existing multimodal MDMs such as MMaDa and Muddit which only support simple image-level understanding tasks and low-resolution image generation, Lavida-O presents a single framework that enables image-level understanding, object grounding, image editing, and high-resolution (1024px) text-to-image synthesis. Lavida-O incorporates a novel Elastic Mixture-of-Transformers (Elastic-MoT) architecture that couples a lightweight generation branch with a larger understanding branch, supported by token compression, universal text conditioning and stratified sampling for efficient and high-quality generation. Lavida-O further incorporates planning and iterative self-reflection in image generation and editing tasks, seamlessly boosting generation quality with its understanding capabilities. Lavida-O achieves state-of-the-art performance on a wide range of benchmarks including RefCOCO object grounding, GenEval text-to-image generation, and ImgEdit image editing, outperforming existing autoregressive models and continuous diffusion models such as Qwen2.5-VL and FluxKontext-dev, while offering considerable speedup at inference. These advances establish Lavida-O as a new paradigm for scalable multimodal reasoning and generation.

cs.CV↗

arXiv2026

Sparse-LaViDa: Sparse Multimodal Discrete Diffusion Language Models

Masked Discrete Diffusion Models (MDMs) have achieved strong performance across a wide range of multimodal tasks, including image understanding, generation, and editing. However, their inference speed remains suboptimal due to the need to repeatedly process redundant masked tokens at every sampling step. In this work, we propose Sparse-LaViDa, a novel modeling framework that dynamically truncates unnecessary masked tokens at each inference step to accelerate MDM sampling. To preserve generation quality, we introduce specialized register tokens that serve as compact representations for the truncated tokens. Furthermore, to ensure consistency between training and inference, we design a specialized attention mask that faithfully matches the truncated sampling procedure during training. Built upon the state-of-the-art unified MDM LaViDa-O, Sparse-LaViDa achieves up to a 2x speedup across diverse tasks including text-to-image generation, image editing, and mathematical reasoning, while maintaining generation quality.

cs.CV↗

arXiv2026

Observation of Switchable Chiral Magnons in an Altermagnet

Chiral magnons, the quanta of handed spin waves, transport spin angular momentum without energy loss due to Joule heating. The recently discovered altermagnets were proposed to host chiral magnons arising from a non-relativistic exchange mechanism, similar to that in ferromagnets but without net magnetization, offering a stray-field-free platform for efficient magnon spin-current manipulation. In this work, we directly observed chiral magnons in the altermagnetic prototype MnTe using polarized inelastic neutron scattering. Furthermore, the magnon chirality was found to be reversibly switched by magnetic-field control, establishing a robust foundation for functional altermagnetic magnonics.

cond-mat.str-el↗

arXiv2025

VGent: Visual Grounding via Modular Design for Disentangling Reasoning and Prediction

Current visual grounding models are either based on a Multimodal Large Language Model (MLLM) that performs auto-regressive decoding, which is slow and risks hallucinations, or on re-aligning an LLM with vision features to learn new special or object tokens for grounding, which may undermine the LLM's pretrained reasoning ability. In contrast, we propose VGent, a modular encoder-decoder architecture that explicitly disentangles high-level reasoning and low-level bounding box prediction. Specifically, a frozen MLLM serves as the encoder to provide untouched powerful reasoning capabilities, while a decoder takes high-quality boxes proposed by detectors as queries and selects target box(es) via cross-attending on encoder's hidden states. This design fully leverages advances in both object detection and MLLM, avoids the pitfalls of auto-regressive decoding, and enables fast inference. Moreover, it supports modular upgrades of both the encoder and decoder to benefit the whole system: we introduce (i) QuadThinker, an RL-based training paradigm for enhancing multi-target reasoning ability of the encoder; (ii) mask-aware label for resolving detection-segmentation ambiguity; and (iii) global target recognition to improve the recognition of all the targets which benefits the selection among augmented proposals. Experiments on multi-target visual grounding benchmarks show that VGent achieves a new state-of-the-art with +20.6% F1 improvement over prior methods, and further boosts gIoU by +8.2% and cIoU by +5.8% under visual reference challenges, while maintaining constant, fast inference latency.

cs.CV↗

arXiv2025

Refer to Any Segmentation Mask Group With Vision-Language Prompts

Recent image segmentation models have advanced to segment images into high-quality masks for visual entities, and yet they cannot provide comprehensive semantic understanding for complex queries based on both language and vision. This limitation reduces their effectiveness in applications that require user-friendly interactions driven by vision-language prompts. To bridge this gap, we introduce a novel task of omnimodal referring expression segmentation (ORES). In this task, a model produces a group of masks based on arbitrary prompts specified by text only or text plus reference visual entities. To address this new challenge, we propose a novel framework to "Refer to Any Segmentation Mask Group" (RAS), which augments segmentation models with complex multimodal interactions and comprehension via a mask-centric large multimodal model. For training and benchmarking ORES models, we create datasets MaskGroups-2M and MaskGroups-HQ to include diverse mask groups specified by text and reference entities. Through extensive evaluation, we demonstrate superior performance of RAS on our new ORES task, as well as classic referring expression segmentation (RES) and generalized referring expression segmentation (GRES) tasks. Project page: https://Ref2Any.github.io.

cs.CV↗

arXiv2025

Quantum Mpemba Effect in Dissipative Spin Chains at Criticality

The Quantum Mpemba Effect (QME) is the quantum counterpart of the classical Mpemba effect--a counterintuitive phenomenon in which a system initially at a higher temperature relax to thermal eauilibrium faster than one at a lower temperature. In this work, we investigate the QME in one-dimensional quantum spin chains coupled to a Markovian environment. By analyzing the full relaxation dynamics governed by the Lindblad master equation, we reveal the emergence of a strong quantum Mpemba effect at quantum critical points. Our findings reveal that criticality enhances the non-monotonic dependence of relaxation times on the initial temperature, leading to anomalously accelerated equilibration. This phenomenon is directly linked to the structure of the Liouvillian spectrum at criticality and the associated overlaps with the initial states. These findings demonstrate that quantum phase transitions could provide a natural setting for realizing and enhancing non-equilibrium phenomena in open quantum systems.

quant-ph↗

arXiv2025

Expedited thermalization dynamics in incommensurate systems

We study the thermalization dynamics of a quantum system embedded in an incommensurate potential and coupled to a Markovian thermal reservoir. The dephasing induced by the bath drives the system toward an infinite-temperature steady state, erasing all initial information-including signatures of localization. We find that initially localized states can relax to the homogeneous steady state faster than delocalized states. Moreover, low-temperature initial states thermalize to infinite temperature more rapidly than high-temperature states -- a phenomenon reminiscent of the Mpemba effect, in which hotter liquids freeze faster than colder ones. The slowest relaxation mode in the Liouvillian spectrum plays a critical role in the expedited thermalization for localized or cold initial states. Our results reveal that the combination of disordered structure and environmental dissipation may lead to non-trivial thermalization behavior, which advances both the conceptual framework of the Mpemba effect and the theoretical understanding of nonequilibrium processes in dissipative disordered systems.

quant-ph↗

arXiv2025

Dissipation induced localization-delocalization transition in a flat band

The interplay between dissipation and localization in quantum systems has garnered significant attention due to its potential to manipulate transport properties and induce phase transitions. In this work, we explore the dissipation-induced extended-localized transition in a flat band model, where the system's asymptotic state can be controlled by tailored dissipative operators. By analyzing the steady-state density matrix and dissipative dynamics, we demonstrate that dissipation is able to drive the system to states dominated by either extended or localized modes, irrespective of the initial conditions. The control mechanism relies on the phase properties of the dissipative operators, which selectively favor specific eigenstates of the Hamiltonian. Our findings reveal that dissipation can be harnessed to induce transitions between extended and localized phases, offering a novel approach to manipulate quantum transport in flat band systems. This work not only deepens our understanding of dissipation-induced phenomena in flat band systems but also provides a new avenue for controlling quantum states in open systems.

quant-ph↗

arXiv2025

Dissipation induced transition between delocalization and localization in the three-dimensional Anderson model

We investigate the probable delocalization-localization transition in open quantum systems with disorder. The disorder can induce localization in isolated quantum systems and it is generally recognized that localization is fragile under the action of dissipations from the external environment due to its interfering nature. Recent work [Y. Liu, et al, Phys. Rev. Lett. 132, 216301 (2024).] found that a one-dimensional quasiperiodic system can be driven into the localization phase by a tailored local dissipation where a dissipation-induced delocalized-localized transition is proposed. Based on this, we consider a more realistic system and show that a dissipation-induced transition between delocalization and localization appears in the three-dimensional (3D) Anderson model. By tuning local dissipative operators acting on nearest neighboring sites, we find that the system can relax to localized states dominated steady state instead of the choice of initial conditions and dissipation strengths. Moreover, we can also realize a delocalized states predominated steady-state from a localized initial state by using a kind of dissipation operators acting on next nearest neighboring sites. Our results enrich the applicability of dissipation-induced localization and identify the transition between delocalized and localized phases in 3D disordered systems.

cond-mat.dis-nn↗

arXiv2024

Reminding Multimodal Large Language Models of Object-aware Knowledge with Retrieved Tags

Despite recent advances in the general visual instruction-following ability of Multimodal Large Language Models (MLLMs), they still struggle with critical problems when required to provide a precise and detailed response to a visual instruction: (1) failure to identify novel objects or entities, (2) mention of non-existent objects, and (3) neglect of object's attributed details. Intuitive solutions include improving the size and quality of data or using larger foundation models. They show effectiveness in mitigating these issues, but at an expensive cost of collecting a vast amount of new data and introducing a significantly larger model. Standing at the intersection of these approaches, we examine the three object-oriented problems from the perspective of the image-to-text mapping process by the multimodal connector. In this paper, we first identify the limitations of multimodal connectors stemming from insufficient training data. Driven by this, we propose to enhance the mapping with retrieval-augmented tag tokens, which contain rich object-aware information such as object names and attributes. With our Tag-grounded visual instruction tuning with retrieval Augmentation (TUNA), we outperform baselines that share the same language model and training data on 12 benchmarks. Furthermore, we show the zero-shot capability of TUNA when provided with specific datastores.

cs.CV↗

arXiv2024

Predicting Visual Attention in Graphic Design Documents

We present a model for predicting visual attention during the free viewing of graphic design documents. While existing works on this topic have aimed at predicting static saliency of graphic designs, our work is the first attempt to predict both spatial attention and dynamic temporal order in which the document regions are fixated by gaze using a deep learning based model. We propose a two-stage model for predicting dynamic attention on such documents, with webpages being our primary choice of document design for demonstration. In the first stage, we predict the saliency maps for each of the document components (e.g. logos, banners, texts, etc. for webpages) conditioned on the type of document layout. These component saliency maps are then jointly used to predict the overall document saliency. In the second stage, we use these layout-specific component saliency maps as the state representation for an inverse reinforcement learning model of fixation scanpath prediction during document viewing. To test our model, we collected a new dataset consisting of eye movements from 41 people freely viewing 450 webpages (the largest dataset of its kind). Experimental results show that our model outperforms existing models in both saliency and scanpath prediction for webpages, and also generalizes very well to other graphic design documents such as comics, posters, mobile UIs, etc. and natural images.

cs.CV↗