Searcharxiv⌕ Search

arXiv subjects

Yao Zhou

Publications and source records attributed to Yao Zhou.

At least 19 recordsLinked to original sources
arXiv2026

KaLM-Embedding-V2: Superior Training Techniques and Data Inspire A Versatile Embedding Model

Recent advancements in Large Language Models (LLMs)-based text embedding models primarily focus on data scaling or synthesis, yet limited exploration of training techniques and data quality, thereby constraining performance. In this work, we propose KaLM-Embedding-V2 from the Lychee-KaLM team, a series of versatile and compact embedding models, systematically incentivizing advanced embedding capability in LLMs by superior training techniques and high-quality data. For model architecture, we implement the models in a 0.5B compact size with simple mean-pooling to produce fixed-length embeddings and remove the causal attention mask to enable fully bidirectional representation learning. For training techniques, we propose a progressive multi-stage training pipeline: pre-training on weakly supervised large-scale datasets, fine-tuning with supervised high-quality datasets, and contrastive distillation with fine-grained soft signals, integrated with focal-style reweighting and online hard-negative mixing to emphasize difficult samples and enrich hard negatives, respectively. For training data, we curate over 20 categories for pre-training and 100 categories for fine-tuning and contrastive distillation to improve both performance and generalization, leveraging task-specific instructions, hard-negative mining, and example-based multi-class labeling to ensure high quality. Combining these techniques, our KaLM-Embedding-V2 series achieves state-of-the-art performance on the Massive Text Embedding Benchmark, outperforming models of comparable size and rivaling models 3--26x larger, setting a new standard for versatile and compact embedding models under 1B parameters. The code, data, and models are available at https://kalm-embedding.github.io/.

cs.CL↗
arXiv2026

HypRQ-VAE: Hyperbolic Item Indexing for Long-Tail-Aware Generative Recommender Systems

Sequential recommender systems model user behavior as item ID sequences, while recent generative methods cast recommendation as a language modeling task using large language models (LLMs). While this paradigm incorporates rich textual semantics, it introduces a fundamental mismatch: LLMs operate on text tokens, whereas recommender systems depend on discrete item indices. This misalignment often leads to hallucinations in generative recommendations. Existing methods attempt to bridge this gap by learning item vocabularies in Euclidean space, but they struggle to model the inherent long-tail distribution of real-world catalogs, where a small number of head items dominate, and a vast number of tail items reflect users' niche preferences. To address this issue, we introduce Hyperbolic Residual-Quantized Variational AutoEncoder (HypRQ-VAE), the first framework to learn item indexing in hyperbolic space. HypRQ-VAE leverages the unique properties of hyperbolic geometry, whose exponential volume expansion naturally accommodates the power law structure of user-item interactions. This allows the model to encode rich textual semantics while preserving the representational fidelity of sparse, long-tail items. Experiments on three benchmark datasets show that HypRQ-VAE significantly improves the performance of recommendation, particularly in recommending tail items. Our analysis attributes these gains to the superior capacity of hyperbolic space to model item hierarchies and sparsity in generative recommendation. Our code and data are available at: https://github.com/wulongfeng/HypRQ-VAE.

cs.IR↗
arXiv2026

BRACE: Taming Sharp Irregularities via Barycentric Rational Forecasting for Fast Diffusion Transformers Inference

Diffusion Transformers (DiTs) have demonstrated exceptional performance in high-fidelity image and video generation. To alleviate their massive computational overhead, temporal feature caching has been proposed to bypass redundant computations. However, existing cache-then-forecast methods driven by derivative-based polynomials often cause severe quality degradation under high acceleration due to unstable long-step predictions. To address this bottleneck, we propose Barycentric Rational Forecasting with Chebyshev Enhancement (BRACE). Motivated by the observation that DiT feature trajectories are globally smooth yet frequently exhibit sharp irregularities and local non-smoothness, BRACE shifts the paradigm from derivative-driven polynomial extrapolation to feature-driven rational forecasting. Specifically, it maintains a local sliding window to cache sparse historical features and leverages adapted Chebyshev weights to formulate a barycentric rational function, directly aggregating these raw features to ensure numerical stability. Extensive experiments demonstrate that BRACE achieves state-of-the-art quality-efficiency trade-offs across various DiT architectures with negligible computational overhead.

cs.CV↗
arXiv2026

Temporal GRPO: Beyond Trajectory-Level Credit in Vision-Language-Action Reinforcement Learning

Outcome-driven reinforcement learning offers a scalable way to post-train vision-language-action (VLA) policies from sparse task-success feedback. In common GRPO-based VLA post-training, one rollout-level advantage is applied to every action in the trajectory. A rollout that completes several valid stages but fails later can therefore penalize the actions that produced its earlier progress. We call this trajectory-level credit aliasing. Temporal GRPO addresses this problem by constructing detectable task stages, aligning each rollout with stage-specific action intervals, and comparing only rollouts that have entered the same stage. The resulting stage advantages are applied to their corresponding intervals in a single policy update. On RoboTwin 2.0, Temporal GRPO improves task success and sample efficiency, with consistent gains across task horizons. Controlled updates on LIBERO-Long preserve shared prerequisite stages and concentrate improvement at the first stage where rollout outcomes diverge.

cs.RO↗
arXiv2026

LMEB: Long-horizon Memory Embedding Benchmark

Memory embeddings are crucial for memory-augmented systems, such as OpenClaw, but their evaluation is underexplored in current text embedding benchmarks, which narrowly focus on traditional passage retrieval and fail to assess models' ability to handle long-horizon memory retrieval tasks involving fragmented, context-dependent, and temporally distant information. To address this gap, we introduce the Long-horizon Memory Embedding Benchmark (LMEB), a comprehensive framework for evaluating embedding models on complex, long-horizon memory retrieval. LMEB comprises 22 datasets and 193 zero-shot retrieval tasks spanning four memory types: episodic, dialogue, semantic, and procedural. These memory types differ in terms of level of abstraction and temporal dependency, capturing distinct aspects of memory retrieval that reflect the diverse challenges of the real world. We evaluate 15 widely used embedding models, ranging from hundreds of millions to ten billion parameters. The results reveal that (1) LMEB provides a reasonable level of difficulty; (2) Larger models do not always perform better; (3) LMEB and MTEB measure orthogonal capabilities. This suggests that the field has yet to converge on a universal model capable of excelling across all memory retrieval tasks, and that strong performance on traditional passage retrieval does not necessarily transfer to long-horizon memory retrieval. LMEB provides a standardized and reproducible framework that fills a key gap in memory embedding evaluation and supports future advances in long-term, context-dependent retrieval.

cs.CL↗
arXiv2026

LM-mixup: Text Data Augmentation via Language Model based Mixup

Instruction tuning is crucial for aligning Large Language Models (LLMs), yet the quality of instruction-following data varies significantly. While high-quality data is paramount, it is often scarce; conversely, abundant low-quality data is frequently discarded, leading to substantial information loss. Existing data augmentation methods struggle to augment this low-quality data effectively, and the evaluation of such techniques remains poorly defined. To address this, we formally define the task of Instruction Distillation: distilling multiple low-quality and redundant inputs into high-quality and coherent instruction-output pairs. Specifically, we introduce a comprehensive data construction pipeline to create MIXTURE, a 144K-sample dataset pairing low-quality or semantically redundant imperfect instruction clusters with their high-quality distillations. We then introduce LM-Mixup, by first performing supervised fine-tuning on MIXTURE and then optimizing it with reinforcement learning. This process uses three complementary reward signals: quality, semantic alignment, and format compliance, via Group Relative Policy Optimization (GRPO). We demonstrate that LM-Mixup effectively augments imperfect datasets: fine-tuning LLMs on its distilled data, which accounts for only about 3% of the entire dataset, not only surpasses full-dataset training but also competes with state-of-the-art high-quality data selection methods across multiple benchmarks. Our work establishes that low-quality data is a valuable resource when properly distilled and augmented with LM-Mixup, significantly enhancing the efficiency and performance of instruction-tuned LLMs.

cs.CL↗
arXiv2026

Optimized finite-$β$ tokamak-stellarator hybrid configurations achieved by planar dipole-field coils

Tokamak--stellarator hybrids seek to combine tokamak-like compactness and confinement with stellarator-like externally generated rotational transform and steady-state operation. In this work, we build on the recent tokamak--stellarator hybrid study using planar dipole-field coils (PDCs) [Yu et al., arXiv:2605.03599], in which the fixed-position, programmable coils on an axisymmetric winding surface generate flexible three-dimensional shaping fields. Using single-stage free-boundary optimization of coil currents and plasma-equilibrium parameters, we construct vacuum and finite-$β$ configurations. The vacuum cases show controllable external transform and magnetic well. The finite-$β$ cases accommodate various density, temperature, and pressure profiles, producing quasi-axisymmetric (QA) equilibria with self-consistent bootstrap current, favorable Mercier stability, and reduced demand for external current drive. Re-optimization enables $β$ ramp-up and access to different field-period QA branches with moderate coil-current changes. At large rotational transform, a toroidally omnigenous (TO)-like configuration exhibits more favorable infinite-$n$ ideal-ballooning behavior than a QA reference with matched profiles, even though ballooning stability is not directly optimized for. These results demonstrate that PDCs provide a flexible platform for achieving optimized finite-$β$ hybrid configurations.

physics.plasm-ph↗
arXiv2026

Bi-NAS: Towards Effective and Personalized Explanation for Recommender Systems via Bi-Level Neural Architecture Search

Recommender systems are vital in helping users navigate vast amounts of information, offering personalized suggestions and effective explanations for these recommendations. While previous efforts have attempted to provide such explanations, evaluating their effectiveness across various scenarios remains a challenge. Enhancing these explanations is essential for improving user engagement, trust, and decision-making. To facilitate effective explanations within the recommender system, we propose a Bi-level Neural Architecture Search (Bi-NAS) framework to optimize explanations. This approach simultaneously refines cross-attention mechanisms and feature interaction functions by exploring both intra-layer and inter-layer design spaces. Furthermore, we integrate Large Language Models (LLMs) to enhance explanation generation, leveraging zero-shot prompting to produce more effective and personalized justifications. By aligning user feature preferences with item quality scores, our approach ensures that explanations reflect both user intent and item attributes, improving transparency and reasoning depth. Extensive evaluations on four real-world datasets demonstrate that Bi-NAS not only boosts recommendation accuracy but also significantly improves the effectiveness of explanations for recommender systems, providing users with clear and reliable insights into the suggestions they receive.

cs.IR↗
arXiv2026

Approaching physical limits of latent dimensionality in optical computing

The physical implementation of artificial intelligence requires mapping computational processes onto the dynamic physical processes of the underlying computing platform. The photonic processors offer an intrinsically parallel and low energy framework for this mapping, however, a mismatch between the potential computing capability of a bounded optical domain and the human accessible manipulation range sets a hard integration density ceiling on existing architectures. Here, we address this challenge by investigating the integration density limits in photonic processors through exploring the fundamental physical limits on the latent dimensionality for maximum expressivity of a bounded optical domain. These physical limits potentially serve as universal metrics for evaluating optical computing capacity. To validate these, we design and realize ultracompact multimode photonic processors approaching these limits: a 2.2 um by 8 um processor achieves 86.7 % accuracy in experiment for iris flower classification, and a 20.6 um by 44.8 um processor reaches 92.9% accuracy in handwritten digit recognition. Finally, we scale this architecture to highly complex tasks by implementing a generative diffusion model for image synthesis. By grounding photonic processor design in the wave physics origin of latent dimensionality, our results supply the missing theoretical reference point for optical computing architecture.

physics.optics↗
arXiv2026

From Shallow to Deep: Pinning Semantic Intent via Causal GRPO

Large Language Models remain vulnerable to adversarial prefix attacks (e.g., ``Sure, here is'') despite robust standard safety. We diagnose this vulnerability as Shallow Safety Alignment, stemming from a pathology we term semantic representation decay: as the model generates compliant prefixes, its internal malicious intent signal fades. To address this, we propose Two-Stage Causal-GRPO (TSC-GRPO), a framework designed to achieve intent pinning. First, grounded in causal identifiability theory, we train a causal intent probe to disentangle invariant intent from stylistic perturbations. Second, we internalize this causal awareness into the policy via Group Relative Policy Optimization. By employing a cumulative causal penalty within ``fork-in-the-road'' training scenarios, we force the model to learn that accumulating harmful tokens monotonically decreases reward, enabling robust late-stage refusals. Experiments show that TSC-GRPO significantly outperforms baselines in defending against jailbreak attacks while preserving general utility.

cs.LG↗
arXiv2026

Nonlinear magnetohydrodynamic modeling of ideal ballooning modes in high-$β$ Wendelstein 7-X plasmas

We present nonlinear magnetohydrodynamic (MHD) simulations of high-$β$ Wendelstein 7-X plasmas using the stellarator extension of the M3D-$C^1$ code, building on the recent work that shows benign saturation of ideal ballooning modes above the designed $β$ limit in the standard configuration [Y. Zhou et al, Phys. Rev. Lett. 133, 135102 (2024)]. First, we examine the results' sensitivity to the parallel thermal conductivity. It is found that while an increased parallel conductivity reduces the linear growth rate, the saturated pressure profile is barely affected. Second, we consider the dependence on the profile shape. It is shown that an equilibrium with a peaked pressure profile and lower $β$ is subject to more significant change than a broad profile with higher $β$ and a larger growth rate, suggesting that benign saturation, or nonlinear stability, is not guaranteed and not dictated by linear growth. Third, we study the influence of the magnetic configuration, with the equilibrium rotational transform varied by adjusting the planar coil current. With similar growth rates, similar magnitudes of profile change are found regardless of the presence of a low-order resonance, which implies that the saturation mechanism is not specific to a resonant or non-resonant mode. These results indicate that MHD stability should still be treated seriously in stellarator operation and design, for which nonlinear modeling using tools like M3D-$C^1$ can play an instrumental role.

physics.plasm-ph↗
arXiv2026

Available Energy and Ground States of Convective Hydrodynamic and Hydromagnetic Instabilities

We propose a method for predicting the nonlinear saturation level of convective instabilities in neutral and magnetized fluids. The method combines Gardner's restacking algorithm, which computes the available energy and ground states of collisionless plasmas in phase space, and Lagrangian relaxation, where fluid elements find lower-energy equilibria while preserving local invariants. For the incompressible Rayleigh-Taylor instability, the problem is formally equivalent to Gardner's and the restacking algorithm directly applies in configuration space. To treat compressibility, we follow restacking with Lagrangian relaxation to obtain the ground state, and the results show excellent agreement with direct numerical simulations. Successful extension to the $m=0$ interchange instability in a Z-pinch demonstrates the method's potential as a general framework for estimating the nonlinear extent of convective instabilities, which can facilitate the design and operation of fusion reactors.

physics.plasm-ph↗
arXiv2026

Causal Front-Door Adjustment for Robust Jailbreak Attacks on LLMs

Safety alignment mechanisms in Large Language Models (LLMs) often operate as latent internal states, obscuring the model's inherent capabilities. Building on this observation, we model the safety mechanism as an unobserved confounder from a causal perspective. Then, we propose the Causal Front-Door Adjustment Attack (CFA{$^2$}) to jailbreak LLM, which is a framework that leverages Pearl's Front-Door Criterion to sever the confounding associations for robust jailbreaking. Specifically, we employ Sparse Autoencoders (SAEs) to physically strip defense-related features, isolating the core task intent. We further reduce computationally expensive marginalization to a deterministic intervention with low inference complexity. Experiments demonstrate that CFA{$^2$} achieves state-of-the-art attack success rates while offering a mechanistic interpretation of the jailbreaking process.

cs.CL↗
arXiv2025

GeoReasoner: Geo-localization with Reasoning in Street Views using a Large Vision-Language Model

This work tackles the problem of geo-localization with a new paradigm using a large vision-language model (LVLM) augmented with human inference knowledge. A primary challenge here is the scarcity of data for training the LVLM - existing street-view datasets often contain numerous low-quality images lacking visual clues, and lack any reasoning inference. To address the data-quality issue, we devise a CLIP-based network to quantify the degree of street-view images being locatable, leading to the creation of a new dataset comprising highly locatable street views. To enhance reasoning inference, we integrate external knowledge obtained from real geo-localization games, tapping into valuable human inference capabilities. The data are utilized to train GeoReasoner, which undergoes fine-tuning through dedicated reasoning and location-tuning stages. Qualitative and quantitative evaluations illustrate that GeoReasoner outperforms counterpart LVLMs by more than 25% at country-level and 38% at city-level geo-localization tasks, and surpasses StreetCLIP performance while requiring fewer training resources. The data and code are available at https://github.com/lingli1996/GeoReasoner.

cs.CV↗
arXiv2025

A Specialized Large Language Model for Clinical Reasoning and Diagnosis in Rare Diseases

Rare diseases affect hundreds of millions worldwide, yet diagnosis often spans years. Convectional pipelines decouple noisy evidence extraction from downstream inferential diagnosis, and general/medical large language models (LLMs) face scarce real world electronic health records (EHRs), stale domain knowledge, and hallucinations. We assemble a large, domain specialized clinical corpus and a clinician validated reasoning set, and develop RareSeek R1 via staged instruction tuning, chain of thought learning, and graph grounded retrieval. Across multicenter EHR narratives and public benchmarks, RareSeek R1 attains state of the art accuracy, robust generalization, and stability under noisy or overlapping phenotypes. Augmented retrieval yields the largest gains when narratives pair with prioritized variants by resolving ambiguity and aligning candidates to mechanisms. Human studies show performance on par with experienced physicians and consistent gains in assistive use. Notably, transparent reasoning highlights decisive non phenotypic evidence (median 23.1%, such as imaging, interventions, functional tests) underpinning many correct diagnoses. This work advances a narrative first, knowledge integrated reasoning paradigm that shortens the diagnostic odyssey and enables auditable, clinically translatable decision support.

cs.CL↗
arXiv2025

Sawtooth crash in tokamak as a sequence of Multi-region Relaxed MHD equilibria

This study examines the sawtooth crash phenomenon in tokamak plasmas by modelling it as a sequence of Multi-region Relaxed Magnetohydrodynamic (MRxMHD) equilibria. Using the Stepped-Pressure Equilibrium Code (SPEC), we constructed a series of equilibria representing intermediate states during the sawtooth crash, with progressively increasing reconnection regions. Numerical results demonstrated that the system prefers the lower energy non-axisymmetric equilibria with islands and is eventually back to an axisymmetric state, capturing key features of the reconnection process. Comparisons with the nonlinear MHD code M3D-C1 showed remarkable agreement on the field-line topology, the safety factor, and the current profile. However, the simplified MRxMHD model does not resolve the detailed structure of the current sheet. Despite this limitation, MRxMHD offers an insightful approach and a complementary perspective to initial-value MHD simulations.

physics.plasm-ph↗
arXiv2025

Direct Numerical Simulations of Oxygen-Flame-Driven Deflagration-to-Detonation Transition in Type Ia Supernovae

We present direct numerical simulations demonstrating deflagration-to-detonation transition (DDT) driven by oxygen flames in Type Ia supernova progenitors. Using the Castro hydrodynamics code coupled with the ``aprox13'' 13-isotope nuclear network, we simulate combustion in isolated fuel regions where oxygen flames trail carbon flames. In a fiducial one-dimensional run at $ρ_{0}=3.5\times10^{7}\ \mathrm{g\ cm^{-3}}$ we observe spontaneous DDT of the oxygen flame via the Zel'dovich gradient mechanism when the carbon-oxygen separation reaches $\sim 10\ \mathrm{km}$. The oxygen detonation then captures the carbon flame and triggers a stable carbon detonation. Systematic one-dimensional parameter scans show that successful carbon DDT requires upstream densities in the range $(3.1$--$3.6)\times10^{7}\ \mathrm{g\ cm^{-3}}$ and a minimum carbon-flame thickness of $\gtrsim 20\ \mathrm{m}$. Two-dimensional simulations confirm DDT and demonstrate that the multidimensional cellular structure of the oxygen detonation can promote carbon detonation at somewhat lower densities than in one dimension. These results provide direct numerical evidence that oxygen-flame-driven DDT is physically plausible in turbulent white-dwarf environments and underscore the importance of multidimensional effects for Type Ia supernova explosion modeling.

astro-ph.HE↗
arXiv2025

ENTP: Enhancing Low-Quality SFT Data via Neural-Symbolic Text Purge-Mix

Supervised Fine-Tuning (SFT) adapts pre-trained Large Language Models (LLMs) to domain-specific instructions by training on a carefully curated subset of high-quality instruction-response pairs, typically drawn from a larger dataset that often contains many low-quality or noisy samples. However, existing quality-first paradigms often overlook valuable signals in discarded low-quality data and rely on imperfect quality filters. We introduce ENTP (Enhancing low-quality SFT data via Neural-symbolic Text Purge-Mix), a framework that revitalizes low-quality corpora through symbolic purification and neural reconstruction. The symbolic module identifies and prunes noisy samples based on statistical priors, while the neural component synthesizes enriched instruction-response pairs by leveraging latent representations and model knowledge. This neural-symbolic synergy enhances data informativeness and diversity. Experiments show that ENTP-augmented datasets, constructed exclusively from low-quality data, outperform 13 established data-selection baselines across five instruction-following benchmarks, and even surpass fine-tuning on the full original dataset (approximately 300K examples). Our results highlight the untapped potential of low-quality data and underscore the importance of intelligent purification and synthesis for efficient instruction alignment.

cs.CL↗