arXiv subjects

Hua Wu

Publications and source records attributed to Hua Wu.

At least 55 records · Page 3Linked to original sources

arXiv2025

BEE-RAG: Balanced Entropy Engineering for Retrieval-Augmented Generation

With the rapid advancement of large language models (LLMs), retrieval-augmented generation (RAG) has emerged as a critical approach to supplement the inherent knowledge limitations of LLMs. However, due to the typically large volume of retrieved information, RAG tends to operate with long context lengths. From the perspective of entropy engineering, we identify unconstrained entropy growth and attention dilution due to long retrieval context as significant factors affecting RAG performance. In this paper, we propose the balanced entropy-engineered RAG (BEE-RAG) framework, which improves the adaptability of RAG systems to varying context lengths through the principle of entropy invariance. By leveraging balanced context entropy to reformulate attention dynamics, BEE-RAG separates attention sensitivity from context length, ensuring a stable entropy level. Building upon this, we introduce a zero-shot inference strategy for multi-importance estimation and a parameter-efficient adaptive fine-tuning mechanism to obtain the optimal balancing factor for different settings. Extensive experiments across multiple RAG tasks demonstrate the effectiveness of BEE-RAG.

cs.CL↗

arXiv2025

A dynamic shim approach for correcting eddy current effects in diffusion-prepared MRI acquisition using a multi-coil AC/DC shim-array

Purpose: We developed a dynamic B0 shimming approach using a 46-channel AC/DC shim array to correct phase errors caused by eddy currents from diffusion-encoding gradients in diffusion-prepared MRI, enabling high b-value imaging without the SNR loss from the use of magnitude stabilizer. Methods: A 46-channel AC/DC shim array and corresponding amplifier system were built. Spin echo prescans with and without diffusion preparation were then used to rapidly measure eddy current induced phase differences. These phase maps were used as targets in an optimization framework to compute compensatory shim currents for multi-shot 3D diffusion-prepared acquisitions. Results: The proposed method allows flexible use of the AC/DC shim array to correct undesirable eddy current effects in diffusion-prepared MRI. Phantom and in vivo experiments demonstrate whole-brain, cardiac-gated, multi-shot 3D diffusion-prepared imaging without the use of magnitude stabilizers. The approach enables preservation of full SNR while achieving reliable diffusion encoding at b-values up to 2000 s/mm2. Conclusions: This work demonstrates a new strategy for applying an AC/DC shim array to compensate for eddy current induced phase errors in diffusion-prepared MRI. By eliminating the need for magnitude stabilizer, it enables efficient high-quality diffusion imaging with full signal sensitivity retained.

physics.med-ph↗

arXiv2025

LLMs are Single-threaded Reasoners: Demystifying the Working Mechanism of Soft Thinking

Human cognition naturally engages with abstract and fluid concepts, whereas existing reasoning models often rely on generating discrete tokens, potentially constraining their expressive capabilities. Recent advancements aim to address this limitation by enabling large language models (LLMs) to generate soft, abstract tokens, thus facilitating reasoning within a continuous concept space. In this paper, we investigate the Soft Thinking capabilities of various LLMs through a systematic analysis of their internal behavior using a suite of probing techniques. Contrary to the prevailing belief that Soft Thinking supports parallel exploration of diverse reasoning paths, our findings reveal that LLMs behave as single-threaded reasoners--they predominantly rely on the token with the highest probability in the soft input to predict the next step. This behavior induces a greedy feedback loop that suppresses alternative reasoning paths and undermines the benefits of transmitting richer information via Soft Tokens. To address this Greedy Pitfall, we propose Stochastic Soft Thinking, which introduces stochasticity to break free from this Greedy Pitfall. Our experiments demonstrate that incorporating randomness--particularly with the Gumbel-Softmax trick--can alleviate the limitations of vanilla approaches and unleash the potential of Soft Thinking, resulting in superior performance across eight reasoning benchmarks. We further demonstrate that Stochastic Soft Thinking exhibits stronger exploration potential compared to conventional COT. Our findings deepen the understanding of continuous reasoning and establish the foundation for future work on improving Soft Thinking with Reinforcement Learning.

cs.CL↗

arXiv2025

A Survey on Parallel Reasoning

With the increasing capabilities of Large Language Models (LLMs), parallel reasoning has emerged as a new inference paradigm that enhances reasoning robustness by concurrently exploring multiple lines of thought before converging on a final answer. It has become a significant trend to explore parallel reasoning to overcome the fragility of standard sequential methods and improve practical performance. In this paper, we aim to survey and summarize the progress and challenges of parallel reasoning. We first present a formal definition of parallel reasoning and clarify its distinction from related concepts like Chain-of-Thought. Then, we organize and discuss advanced techniques based on a novel taxonomy, including non-interactive reasoning, interactive reasoning, and efficiency-focused decoding strategies. Additionally, we explore various application scenarios, such as solving complex problems and enhancing the reliability of LLM outputs.Finally, we highlight the core challenges of parallel reasoning and suggest potential directions for future research. We hope that our work can provide a useful roadmap for beginners and encourage more research on improving parallel reasoning methods. Related source can be avaliable in https://github.com/PPPP-kaqiu/Awesome-Parallel-Reasoning.

cs.CL↗

arXiv2025

MatryoshkaThinking: Recursive Test-Time Scaling Enables Efficient Reasoning

Test-time scaling has emerged as a promising paradigm in language modeling, wherein additional computational resources are allocated during inference to enhance model performance. Recent approaches, such as DeepConf, have demonstrated the efficacy of this strategy, however, they often incur substantial computational overhead to achieve competitive results. In this work, we propose MatryoshkaThinking, a novel method that significantly reduces computational cost while maintaining state-of-the-art performance. Specifically, MatryoshkaThinking attains a score of 99.79 on AIME2025 using only 4% of the computation required by DeepConf. The core of our approach lies in the recursive exploitation of the model's intrinsic capabilities in reasoning, verification, and summarization, which collectively enhance the retention of correct solutions and reduce the disparity between Pass@k and Pass@1. Comprehensive evaluations across multiple open-source models and challenging multi-modal reasoning benchmarks validate the effectiveness and generality of our method. These findings offer new insights into the design of efficient and scalable test-time inference strategies for advanced language models.

cs.CL↗

arXiv2025

AlignX: Advancing Multilingual Large Language Models with Multilingual Representation Alignment

Multilingual large language models (LLMs) possess impressive multilingual understanding and generation capabilities. However, their performance and cross-lingual alignment often lag for non-dominant languages. A common solution is to fine-tune LLMs on large-scale and more balanced multilingual corpus, but such approaches often lead to imprecise alignment and suboptimal knowledge transfer, struggling with limited improvements across languages. In this paper, we propose AlignX to bridge the multilingual performance gap, which is a two-stage representation-level framework for enhancing multilingual performance of pre-trained LLMs. In the first stage, we align multilingual representations with multilingual semantic alignment and language feature integration. In the second stage, we stimulate the multilingual capability of LLMs via multilingual instruction fine-tuning. Experimental results on several pre-trained LLMs demonstrate that our approach enhances LLMs' multilingual general and cross-lingual generation capability. Further analysis indicates that AlignX brings the multilingual representations closer and improves the cross-lingual alignment.

cs.CL↗

arXiv2025

A2R: An Asymmetric Two-Stage Reasoning Framework for Parallel Reasoning

Recent Large Reasoning Models have achieved significant improvements in complex task-solving capabilities by allocating more computation at the inference stage with a "thinking longer" paradigm. Even as the foundational reasoning capabilities of models advance rapidly, the persistent gap between a model's performance in a single attempt and its latent potential, often revealed only across multiple solution paths, starkly highlights the disparity between its realized and inherent capabilities. To address this, we present A2R, an Asymmetric Two-Stage Reasoning framework designed to explicitly bridge the gap between a model's potential and its actual performance. In this framework, an "explorer" model first generates potential solutions in parallel through repeated sampling. Subsequently,a "synthesizer" model integrates these references for a more refined, second stage of reasoning. This two-stage process allows computation to be scaled orthogonally to existing sequential methods. Our work makes two key innovations: First, we present A2R as a plug-and-play parallel reasoning framework that explicitly enhances a model's capabilities on complex questions. For example, using our framework, the Qwen3-8B-distill model achieves a 75% performance improvement compared to its self-consistency baseline. Second, through a systematic analysis of the explorer and synthesizer roles, we identify an effective asymmetric scaling paradigm. This insight leads to A2R-Efficient, a "small-to-big" variant that combines a Qwen3-4B explorer with a Qwen3-8B synthesizer. This configuration surpasses the average performance of a monolithic Qwen3-32B model at a nearly 30% lower cost. Collectively, these results show that A2R is not only a performance-boosting framework but also an efficient and practical solution for real-world applications.

cs.AI↗

arXiv2025

Orthogonal Finetuning for Direct Preference Optimization

DPO is an effective preference optimization algorithm. However, the DPO-tuned models tend to overfit on the dispreferred samples, manifested as overly long generations lacking diversity. While recent regularization approaches have endeavored to alleviate this issue by modifying the objective function, they achieved that at the cost of alignment performance degradation. In this paper, we innovatively incorporate regularization from the perspective of weight updating to curb alignment overfitting. Through the pilot experiment, we discovered that there exists a positive correlation between overfitting and the hyperspherical energy fluctuation. Hence, we introduce orthogonal finetuning for DPO via a weight-Rotated Preference Optimization (RoPO) method, which merely conducts rotational and magnitude-stretching updates on the weight parameters to maintain the hyperspherical energy invariant, thereby preserving the knowledge encoded in the angle between neurons. Extensive experiments demonstrate that our model aligns perfectly with human preferences while retaining the original expressive capacity using only 0.0086% of the trainable parameters, suggesting an effective regularization against overfitting. Specifically, RoPO outperforms DPO by up to 10 points on MT-Bench and by up to 2.8 points on AlpacaEval 2, while enhancing the generation diversity by an average of 6 points.

cs.CL↗

arXiv2025

Weights-Rotated Preference Optimization for Large Language Models

Despite the efficacy of Direct Preference Optimization (DPO) in aligning Large Language Models (LLMs), reward hacking remains a pivotal challenge. This issue emerges when LLMs excessively reduce the probability of rejected completions to achieve high rewards, without genuinely meeting their intended goals. As a result, this leads to overly lengthy generation lacking diversity, as well as catastrophic forgetting of knowledge. We investigate the underlying reason behind this issue, which is representation redundancy caused by neuron collapse in the parameter space. Hence, we propose a novel Weights-Rotated Preference Optimization (RoPO) algorithm, which implicitly constrains the output layer logits with the KL divergence inherited from DPO and explicitly constrains the intermediate hidden states by fine-tuning on a multi-granularity orthogonal matrix. This design prevents the policy model from deviating too far from the reference model, thereby retaining the knowledge and expressive capabilities acquired during pre-training and SFT stages. Our RoPO achieves up to a 3.27-point improvement on AlpacaEval 2, and surpasses the best baseline by 6.2 to 7.5 points on MT-Bench with merely 0.015% of the trainable parameters, demonstrating its effectiveness in alleviating the reward hacking problem of DPO.

cs.CL↗

arXiv2025

BeamLoRA: Beam-Constraint Low-Rank Adaptation

Due to the demand for efficient fine-tuning of large language models, Low-Rank Adaptation (LoRA) has been widely adopted as one of the most effective parameter-efficient fine-tuning methods. Nevertheless, while LoRA improves efficiency, there remains room for improvement in accuracy. Herein, we adopt a novel perspective to assess the characteristics of LoRA ranks. The results reveal that different ranks within the LoRA modules not only exhibit varying levels of importance but also evolve dynamically throughout the fine-tuning process, which may limit the performance of LoRA. Based on these findings, we propose BeamLoRA, which conceptualizes each LoRA module as a beam where each rank naturally corresponds to a potential sub-solution, and the fine-tuning process becomes a search for the optimal sub-solution combination. BeamLoRA dynamically eliminates underperforming sub-solutions while expanding the parameter space for promising ones, enhancing performance with a fixed rank. Extensive experiments across three base models and 12 datasets spanning math reasoning, code generation, and commonsense reasoning demonstrate that BeamLoRA consistently enhances the performance of LoRA, surpassing the other baseline methods.

cs.CL↗

arXiv2025

Curiosity-Driven Reinforcement Learning from Human Feedback

Reinforcement learning from human feedback (RLHF) has proven effective in aligning large language models (LLMs) with human preferences, but often at the cost of reduced output diversity. This trade-off between diversity and alignment quality remains a significant challenge. Drawing inspiration from curiosity-driven exploration in reinforcement learning, we introduce curiosity-driven RLHF (CD-RLHF), a framework that incorporates intrinsic rewards for novel states, alongside traditional sparse extrinsic rewards, to optimize both output diversity and alignment quality. We demonstrate the effectiveness of CD-RLHF through extensive experiments on a range of tasks, including text summarization and instruction following. Our approach achieves significant gains in diversity on multiple diversity-oriented metrics while maintaining alignment with human preferences comparable to standard RLHF. We make our code publicly available at https://github.com/ernie-research/CD-RLHF.

cs.CL↗

arXiv2025

Unveiling Knowledge Utilization Mechanisms in LLM-based Retrieval-Augmented Generation

Considering the inherent limitations of parametric knowledge in large language models (LLMs), retrieval-augmented generation (RAG) is widely employed to expand their knowledge scope. Since RAG has shown promise in knowledge-intensive tasks like open-domain question answering, its broader application to complex tasks and intelligent assistants has further advanced its utility. Despite this progress, the underlying knowledge utilization mechanisms of LLM-based RAG remain underexplored. In this paper, we present a systematic investigation of the intrinsic mechanisms by which LLMs integrate internal (parametric) and external (retrieved) knowledge in RAG scenarios. Specially, we employ knowledge stream analysis at the macroscopic level, and investigate the function of individual modules at the microscopic level. Drawing on knowledge streaming analyses, we decompose the knowledge utilization process into four distinct stages within LLM layers: knowledge refinement, knowledge elicitation, knowledge expression, and knowledge contestation. We further demonstrate that the relevance of passages guides the streaming of knowledge through these stages. At the module level, we introduce a new method, knowledge activation probability entropy (KAPE) for neuron identification associated with either internal or external knowledge. By selectively deactivating these neurons, we achieve targeted shifts in the LLM's reliance on one knowledge source over the other. Moreover, we discern complementary roles for multi-head attention and multi-layer perceptron layers during knowledge formation. These insights offer a foundation for improving interpretability and reliability in retrieval-augmented LLMs, paving the way for more robust and transparent generative solutions in knowledge-intensive domains.

cs.CL↗

arXiv2025

BEVWorld: A Multimodal World Simulator for Autonomous Driving via Scene-Level BEV Latents

World models have attracted increasing attention in autonomous driving for their ability to forecast potential future scenarios. In this paper, we propose BEVWorld, a novel framework that transforms multimodal sensor inputs into a unified and compact Bird's Eye View (BEV) latent space for holistic environment modeling. The proposed world model consists of two main components: a multi-modal tokenizer and a latent BEV sequence diffusion model. The multi-modal tokenizer first encodes heterogeneous sensory data, and its decoder reconstructs the latent BEV tokens into LiDAR and surround-view image observations via ray-casting rendering in a self-supervised manner. This enables joint modeling and bidirectional encoding-decoding of panoramic imagery and point cloud data within a shared spatial representation. On top of this, the latent BEV sequence diffusion model performs temporally consistent forecasting of future scenes, conditioned on high-level action tokens, enabling scene-level reasoning over time. Extensive experiments demonstrate the effectiveness of BEVWorld on autonomous driving benchmarks, showcasing its capability in realistic future scene generation and its benefits for downstream tasks such as perception and motion prediction.

cs.CV↗

arXiv2025

Contrasting magnetism in VPS3 and CrI3 monolayers with the common honeycomb S = 3/2 spin lattice

Two-dimensional (2D) magnetic materials are promising candidates for spintronics and quantum technologies. One extensively studied example is the ferromagnetic (FM) CrI$_3$ monolayer with the honeycomb Cr$^{3+}$ ($t_{2g}^3$, $S$ = 3/2) spin lattice, while VPS$_3$ has a same honeycomb $S$ = 3/2 spin lattice (V$^{2+}$, $t_{2g}^3$) but displays N$\acute{e}$el antiferromagnetism (AFM). In this work, we study the electronic structure and particularly the contrasting magnetism of VPS$_3$ and CrI$_3$ monolayers. We find that VPS$_3$ is a Mott-Hubbard insulator but CrI$_3$ is a charge-transfer insulator, and therefore their magnetic exchange mechanisms are essentially different. The first nearest-neighbor (1NN) direct $d$-$d$ exchange dominates in VPS$_3$, thus leading to a strong antiferromagnetic (AF) coupling. However, the formation of vanadium vacancies, associated with instability of the low-valence V$^{2+}$ ions, suppresses the AF coupling and thus strongly reduces the N$\acute{e}$el temperature ($T_{\text{N}}$) in line with the experimental observation. In contrast, our results reveal that the major 1NN $d$-$p$-$d$ superexchanges in CrI$_3$ via different channels give rise to competing FM and AF couplings, ultimately resulting in a weak FM coupling as observed experimentally. After revisiting several important superexchange channels reported in the literature, based on our MLWFs and tight-binding analyses, we note that some antiphase contributions must be subtly and simultaneously considered, and thus we provide a deeper insight into the FM coupling of CrI$_3$. Moreover, we identify and compare the major contributions to the magnetic anisotropy, i.e., a weak shape anisotropy in VPS$_3$ and a relatively strong exchange anisotropy in CrI$_3$.

cond-mat.mtrl-sci↗

arXiv2025

StarVid: Enhancing Semantic Alignment in Video Diffusion Models via Spatial and SynTactic Guided Attention Refocusing

Recent advances in text-to-video (T2V) generation with diffusion models have garnered significant attention. However, they typically perform well in scenes with a single object and motion, struggling in compositional scenarios with multiple objects and distinct motions to accurately reflect the semantic content of text prompts. To address these challenges, we propose \textbf{StarVid}, a plug-and-play, training-free method that improves semantic alignment between multiple subjects, their motions, and text prompts in T2V models. StarVid first leverages the spatial reasoning capabilities of large language models (LLMs) for two-stage motion trajectory planning based on text prompts. Such trajectories serve as spatial priors, guiding a spatial-aware loss to refocus cross-attention (CA) maps into distinctive regions. Furthermore, we propose a syntax-guided contrastive constraint to strengthen the correlation between the CA maps of verbs and their corresponding nouns, enhancing motion-subject binding. Both qualitative and quantitative evaluations demonstrate that the proposed framework significantly outperforms baseline methods, delivering videos of higher quality with improved semantic consistency.

cs.CV↗

arXiv2025

Multiferroic Metallic Monolayer Cu(CrSe2)2

The two-dimensional (2D) Cu(CrSe$_2$)$_2$ monolayer stands out for its combined ferromagnetic (FM), ferroelectric (FE), and metallic properties, marking itself as a prominent 2D multiferroic metal. This work studies those properties and the relevant physics, using density functional calculations, Monte Carlo simulations, and $ab$ $initio$ molecular dynamics. Our results show that Cu(CrSe$_2$)$_2$ monolayer is in the Cr$^{3+}$ $t_{2g}^3$ state with $S$ = 3/2 and Cu$^{1+}$ $3d^{10}$ with $S$ = 0. The observed in-plane magnetic anisotropy primarily arises from exchange anisotropy, which is associated with the Cr-Se-Cr itinerant ferromagnetism. In contrast, both single-ion anisotropy and shape magnetic anisotropy contribute negligibly. The Dzyaloshinskii-Moriya interaction is also quite weak, only about 3\% of the intralayer exchange parameters. Our Monte Carlo simulations show a FM Curie temperature ($T_{\rm C}$) of 190 K. Moreover, the monolayer exhibits a vertical FE polarization of 1.79 pC/m and a FE polarization switching barrier of 182 meV/f.u., and the FE state remains stable above room temperature as shown by $ab$ $initio$ molecular dynamics simulations. Furthermore, a magnetoelectric coupling is partially manifested by a magnetization rotation from in-plane to out-of-plane associated with a FE-to-paraelectric transition. The magnetization rotation can also be induced by either hole or electron doping, and the hole doping increases the $T_{\rm C}$ up to 238 K. In addition, tensile strain reduces the FE polarization but enhances $T_{\rm C}$ to 290 K, while a compressive strain gives an opposite effect. Therefore, the multiferroic metallic Cu(CrSe$_2$)$_2$ monolayer may be explored for advanced multifunctional electronic devices.

cond-mat.mtrl-sci↗

arXiv2025

Inner Thinking Transformer: Leveraging Dynamic Depth Scaling to Foster Adaptive Internal Thinking

Large language models (LLMs) face inherent performance bottlenecks under parameter constraints, particularly in processing critical tokens that demand complex reasoning. Empirical analysis reveals challenging tokens induce abrupt gradient spikes across layers, exposing architectural stress points in standard Transformers. Building on this insight, we propose Inner Thinking Transformer (ITT), which reimagines layer computations as implicit thinking steps. ITT dynamically allocates computation through Adaptive Token Routing, iteratively refines representations via Residual Thinking Connections, and distinguishes reasoning phases using Thinking Step Encoding. ITT enables deeper processing of critical tokens without parameter expansion. Evaluations across 162M-466M parameter models show ITT achieves 96.5\% performance of a 466M Transformer using only 162M parameters, reduces training data by 43.2\%, and outperforms Transformer/Loop variants in 11 benchmarks. By enabling elastic computation allocation during inference, ITT balances performance and efficiency through architecture-aware optimization of implicit thinking pathways.

cs.CL↗

arXiv2025

Shall Your Data Strategy Work? Perform a Swift Study

This work presents a swift method to assess the efficacy of particular types of instruction-tuning data, utilizing just a handful of probe examples and eliminating the need for model retraining. This method employs the idea of gradient-based data influence estimation, analyzing the gradient projections of probe examples from the chosen strategy onto evaluation examples to assess its advantages. Building upon this method, we conducted three swift studies to investigate the potential of Chain-of-thought (CoT) data, query clarification data, and response evaluation data in enhancing model generalization. Subsequently, we embarked on a validation study to corroborate the findings of these swift studies. In this validation study, we developed training datasets tailored to each studied strategy and compared model performance with and without the use of these datasets. The results of the validation study aligned with the findings of the swift studies, validating the efficacy of our proposed method.

cs.CL↗