arXiv subjects

Haowei He

Publications and source records attributed to Haowei He.

17 recordsLinked to original sources

arXiv2026

FCPRAG: Fusion-Controller Parametric Retrieval-Augmented Generation for Stable Multi-Passage LoRA Injection

Parametric retrieval-augmented generation (PRAG) injects retrieved evidence into a large language model (LLM) through passage-specific LoRA adapters, reducing reliance on long in-context prompts. When multiple passages are retrieved for the same query, however, evidence-level fusion becomes a bottleneck: equal-weight merging can amplify weak or conflicting evidence, and translating retrieval signals into fusion weights often requires fragile global tuning. We propose FCPRAG, a fusion-controlled parametric RAG framework that adds a lightweight controller for retrieval-conditioned, sample-level adapter fusion. The controller predicts per-passage fusion scores together with sample-level calibration signals, including a mixing gate and an adaptive temperature, enabling fusion that stays selective under informative retrieval signals and conservative under uncertainty. FCPRAG is trained with merge-aware supervision derived from each adapter's marginal contribution within a multi-adapter merge, using training data only. We further show that a single dataset-level temperature is suboptimal under heteroscedastic retrieval uncertainty, motivating sample-level adaptation. Experiments on HotpotQA, 2WikiMultiHopQA, PopQA, and ComplexWebQuestions (CWQ) across three LLM backbones show that FCPRAG consistently improves F1 over standard RAG and parametric RAG baselines, with gains of up to 4.65% on 2WikiMultiHopQA and 7.55% on CWQ, while also reducing tuning cost and improving robustness under retrieval perturbations.

cs.CL↗

arXiv2026

Dual Attention Residuals

Recent work extends Transformer residual pathways along two complementary axes: historical retrieval selects information from earlier depths, whereas multi-stream methods maintain multiple residual trajectories. These capabilities have largely been studied in isolation, and assigning an independent retriever to each stream still prevents one trajectory from influencing depth selection in another. We propose Dual Attention Residuals (DAR), which brings multi-stream interaction into historical retrieval through reciprocal cross-stream addressing. For each target stream, DAR computes depth weights from normalized states in the opposite stream and applies them to values from the target stream's own history. The retrieved states are combined for an unchanged Transformer branch and updated through constrained gated writes; a block-form variant operates on block-level histories to control overhead. Across dense models from 0.1B to 1B parameters and a 7B sparse-MoE model, DAR consistently improves validation loss over standard residual Transformers and Attention Residuals. Routing ablations show that the gain cannot be explained by an additional stream or value projection alone. Representation and intervention analyses further show that reciprocal cross-stream selection preserves depth-wise diversity and avoids the redundancy or functional imbalance observed in alternative two-stream designs.

cs.CL↗

arXiv2026

Where Does Long-Context Supervision Actually Go? Effective-Context Exposure Balancing

Long-context adaptation is often viewed as window scaling, but this misses a token-level supervision mismatch: in packed training with document masking, each target token's effective context remains short. We introduce EXACT, a supervision-allocation objective that assigns extra weight to long effective-context targets by inverse frequency within the long tail. Across seven Qwen/LLaMA CPT configurations, EXACT improves all 28 trained/extrapolated NoLiMa and RULER comparisons. On Qwen2.5-0.5B, NoLiMa improves by +10.09 (trained) and +5.34 (extrapolated); RULER by +10.69 and +5.55. On LLaMA-3.2-3B, RULER improves by +17.91 and +16.11. Standard QA/reasoning are preserved (+0.24 macro change across six benchmarks). A distance-resolved probe shows gains arise when evidence is thousands of tokens away, while short cases remain unchanged. Results support a supervision-centric thesis: long-context adaptation depends on how strongly training supervises long-context predictions.

cs.CL↗

arXiv2025

Training Report of TeleChat3-MoE

TeleChat3-MoE is the latest series of TeleChat large language models, featuring a Mixture-of-Experts (MoE) architecture with parameter counts ranging from 105 billion to over one trillion,trained end-to-end on Ascend NPU cluster. This technical report mainly presents the underlying training infrastructure that enables reliable and efficient scaling to frontier model sizes. We detail systematic methodologies for operator-level and end-to-end numerical accuracy verification, ensuring consistency across hardware platforms and distributed parallelism strategies. Furthermore, we introduce a suite of performance optimizations, including interleaved pipeline scheduling, attention-aware data scheduling for long-sequence training,hierarchical and overlapped communication for expert parallelism, and DVM-based operator fusion. A systematic parallelization framework, leveraging analytical estimation and integer linear programming, is also proposed to optimize multi-dimensional parallelism configurations. Additionally, we present methodological approaches to cluster-level optimizations, addressing host- and device-bound bottlenecks during large-scale training tasks. These infrastructure advancements yield significant throughput improvements and near-linear scaling on clusters comprising thousands of devices, providing a robust foundation for large-scale language model development on hardware ecosystems.

cs.CL↗

arXiv2025

TableReasoner: Advancing Table Reasoning Framework with Large Language Models

The paper presents our system developed for table question answering (TQA). TQA tasks face challenges due to the characteristics of real-world tabular data, such as large size, incomplete column semantics, and entity ambiguity. To address these issues, we propose a large language model (LLM)-powered and programming-based table reasoning framework, named TableReasoner. It models a table using the schema that combines structural and semantic representations, enabling holistic understanding and efficient processing of large tables. We design a multi-step schema linking plan to derive a focused table schema that retains only query-relevant information, eliminating ambiguity and alleviating hallucinations. This focused table schema provides precise and sufficient table details for query refinement and programming. Furthermore, we integrate the reasoning workflow into an iterative thinking architecture, allowing incremental cycles of thinking, reasoning and reflection. Our system achieves first place in both subtasks of SemEval-2025 Task 8.

cs.AI↗

arXiv2022

Trade-off Between Efficiency and Consistency for Removal-based Explanations

In the current landscape of explanation methodologies, most predominant approaches, such as SHAP and LIME, employ removal-based techniques to evaluate the impact of individual features by simulating various scenarios with specific features omitted. Nonetheless, these methods primarily emphasize efficiency in the original context, often resulting in general inconsistencies. In this paper, we demonstrate that such inconsistency is an inherent aspect of these approaches by establishing the Impossible Trinity Theorem, which posits that interpretability, efficiency, and consistency cannot hold simultaneously. Recognizing that the attainment of an ideal explanation remains elusive, we propose the utilization of interpretation error as a metric to gauge inefficiencies and inconsistencies. To this end, we present two novel algorithms founded on the standard polynomial basis, aimed at minimizing interpretation error. Our empirical findings indicate that the proposed methods achieve a substantial reduction in interpretation error, up to 31.8 times lower when compared to alternative techniques. Code is available at https://github.com/trusty-ai/efficient-consistent-explanations.

cs.LG↗

arXiv2022

Anomaly Detection with Test Time Augmentation and Consistency Evaluation

Deep neural networks are known to be vulnerable to unseen data: they may wrongly assign high confidence stcores to out-distribuion samples. Recent works try to solve the problem using representation learning methods and specific metrics. In this paper, we propose a simple, yet effective post-hoc anomaly detection algorithm named Test Time Augmentation Anomaly Detection (TTA-AD), inspired by a novel observation. Specifically, we observe that in-distribution data enjoy more consistent predictions for its original and augmented versions on a trained network than out-distribution data, which separates in-distribution and out-distribution samples. Experiments on various high-resolution image benchmark datasets demonstrate that TTA-AD achieves comparable or better detection performance under dataset-vs-dataset anomaly detection settings with a 60%~90\% running time reduction of existing classifier-based algorithms. We provide empirical verification that the key to TTA-AD lies in the remaining classes between augmented features, which has long been partially ignored by previous works. Additionally, we use RUNS as a surrogate to analyze our algorithm theoretically.

cs.CV↗

arXiv2022

EVBattery: A Large-Scale Electric Vehicle Dataset for Battery Health and Capacity Estimation

Electric vehicles (EVs) play an important role in reducing carbon emissions. As EV adoption accelerates, safety issues caused by EV batteries have become an important research topic. In order to benchmark and develop data-driven methods for this task, we introduce a large and comprehensive dataset of EV batteries. Our dataset includes charging records collected from hundreds of EVs from three manufacturers over several years. Our dataset is the first large-scale public dataset on real-world battery data, as existing data either include only several vehicles or is collected in the lab environment. Meanwhile, our dataset features two types of labels, corresponding to two key tasks - battery health estimation and battery capacity estimation. In addition to demonstrating how existing deep learning algorithms can be applied to this task, we further develop an algorithm that exploits the data structure of battery systems. Our algorithm achieves better results and shows that a customized method can improve model performances. We hope that this public dataset provides valuable resources for researchers, policymakers, and industry professionals to better understand the dynamics of EV battery aging and support the transition toward a sustainable transportation system.

cs.LG↗

arXiv2021

Can Pretext-Based Self-Supervised Learning Be Boosted by Downstream Data? A Theoretical Analysis

Pretext-based self-supervised learning learns the semantic representation via a handcrafted pretext task over unlabeled data and then uses the learned representation for downstream tasks, which effectively reduces the sample complexity of downstream tasks under Conditional Independence (CI) condition. However, the downstream sample complexity gets much worse if the CI condition does not hold. One interesting question is whether we can make the CI condition hold by using downstream data to refine the unlabeled data to boost self-supervised learning. At first glance, one might think that seeing downstream data in advance would always boost the downstream performance. However, we show that it is not intuitively true and point out that in some cases, it hurts the final performance instead. In particular, we prove both model-free and model-dependent lower bounds of the number of downstream samples used for data refinement. Moreover, we conduct various experiments on both synthetic and real-world datasets to verify our theoretical results.

cs.LG↗

arXiv2019

High temperature singlet-based magnetism from Hund's rule correlations

Uranium compounds can manifest a wide range of fascinating many-body phenomena, and are often thought to be poised at a crossover between localized and itinerant regimes for 5f electrons. The antiferromagnetic dipnictide USb2 has been of recent interest due to the discovery of rich proximate phase diagrams and unusual quantum coherence phenomena. Here, linear-dichroic X-ray absorption and elastic neutron scattering are used to characterize electronic symmetries on uranium in USb2 and isostructural UBi2. Of these two materials, only USb2 is found to enable strong Hund's rule alignment of local magnetic degrees of freedom, and to undergo distinctive changes in local atomic multiplet symmetry across the magnetic phase transition. Theoretical analysis reveals that these and other anomalous properties of the material may be understood by attributing it as the first known high temperature realization of a singlet ground state magnet, in which magnetism occurs through a process that resembles exciton condensation.

cond-mat.str-el↗

arXiv2019

Asymmetric Valleys: Beyond Sharp and Flat Local Minima

Despite the non-convex nature of their loss functions, deep neural networks are known to generalize well when optimized with stochastic gradient descent (SGD). Recent work conjectures that SGD with proper configuration is able to find wide and flat local minima, which have been proposed to be associated with good generalization performance. In this paper, we observe that local minima of modern deep networks are more than being flat or sharp. Specifically, at a local minimum there exist many asymmetric directions such that the loss increases abruptly along one side, and slowly along the opposite side--we formally define such minima as asymmetric valleys. Under mild assumptions, we prove that for asymmetric valleys, a solution biased towards the flat side generalizes better than the exact minimizer. Further, we show that simply averaging the weights along the SGD trajectory gives rise to such biased solutions implicitly. This provides a theoretical explanation for the intriguing phenomenon observed by Izmailov et al. (2018). In addition, we empirically find that batch normalization (BN) appears to be a major cause for asymmetric valleys.

cs.LG↗

arXiv2018

Observation of a Topological Insulator Dirac Cone Reshaped by Non-magnetic Impurity Resonance

The massless Dirac electrons found at topological insulator surfaces are thought to be influenced very little by weak, non-magnetic disorder. However, a resonance effect of strongly perturbing non-magnetic impurities has been theoretically predicted to change the dispersion and physical nature of low-energy quasiparticles, resulting in unique particle-like states that lack microscopic translational symmetry. Here we report the direct observation of impurities reshaping the surface Dirac cone of the model 3D topological insulator Bi2Se3. For the first time, a pronounced kink-like dispersion feature is observed in disorder-enriched samples, and found to be closely associated with the anomaly caused by impurity resonance in the surface state density of states, as observed by dichroic angle resolved photoemission spectroscopy (ARPES). The experimental observation of these features, which closely resemble theoretical predictions, has significant implications for the properties of topological Dirac cones in applied scenarios that commonly feature point defect disorder at surfaces or interfaces.

cond-mat.mtrl-sci↗

arXiv2017

Charge transfer excitations in VUV and soft X-ray resonant scattering spectroscopies

The utility of resonant scattering spectroscopies for identifying electronic symmetries and density distributions changes dramatically as a function of photon energy. In the hard X-ray regime, strong core hole monopole potentials tend to produce X-ray absorption features with well defined electron number on the scattering site. By contrast, in the vacuum ultraviolet (VUV), resonant scattering from Mott insulators tends to reveal spectra that are characteristic of only the nominal valence, and are insensitive to deviations from nominal valence brought on by metal-ligand hybridization. Here, atomic multiplet simulations are used to investigate the interplay of monopolar and mulitpolar Coulomb interactions in the VUV and soft X-ray regimes, to identify how charge transfer thresholds and other signatures of mixed valence can manifest in this low photon energy regime. The study focuses on the Mott insulator NiO as a well characterized model system, and extrapolates interactions into non-physical regimes to identify principles that shape the spectral features.

cond-mat.str-el↗

arXiv2017

Irreversible proliferation of magnetic moments at cleaved surfaces of the topological Kondo insulator SmB6

The compound SmB$_6$ is the best established realization of a topological Kondo insulator, in which a topological insulator state is obtained through Kondo coherence. Recent studies have found evidence that the surface of SmB$_6$ hosts ferromagnetic domains, creating an intrinsic platform for unidirectional ballistic transport at the domain boundaries. Here, surface-sensitive X-ray absorption (XAS) and bulk-sensitive resonant inelastic X-ray scattering (RIXS) spectra are measured at the Sm N$_{4,5}$-edge, and used to evaluate electronic symmetries, excitations and temperature dependence near the surface of cleaved samples. The XAS data show that the density of large-moment atomic multiplet states on a cleaved surface grows irreversibly over time, to a degree that likely exceeds a related change that has recently been observed in the surface 4f orbital occupation.

cond-mat.str-el↗

arXiv2017

Disorder enabled band structure engineering of a topological insulator surface

Three dimensional topological insulators are bulk insulators with $\mathbf{Z}_2$ topological electronic order that gives rise to conducting light-like surface states. These surface electrons are exceptionally resistant to localization by non-magnetic disorder, and have been adopted as the basis for a wide range of proposals to achieve new quasiparticle species and device functionality. Recent studies have yielded a surprise by showing that in spite of resisting localization, topological insulator surface electrons can be reshaped by defects into distinctive resonance states. Here we use numerical simulations and scanning tunneling microscopy data to show that these resonance states have significance well beyond the localized regime usually associated with impurity bands. At native densities in the model Bi$_2$X$_3$ (X=Bi, Te) compounds, defect resonance states are predicted to generate a new quantum basis for an emergent electron gas that supports diffusive electrical transport.

cond-mat.mes-hall↗

arXiv2016

Measurement of collective excitations in VO$_2$ by resonant inelastic X-ray scattering

Vanadium dioxide is of broad interest as a spin-1/2 electron system that realizes a metal-insulator transition near room temperature, due to a combination of strongly correlated and itinerant electron physics. Here, resonant inelastic X-ray scattering is used to measure the excitation spectrum of charge, spin, and lattice degrees of freedom at the vanadium L-edge under different polarization and temperature conditions. These spectra reveal the evolution of energetics across the metal-insulator transition, including the low temperature appearance of a strong candidate for the singlet-triplet excitation of a vanadium dimer.

cond-mat.str-el↗

arXiv2015

Spectroscopic determination of the atomic f-electron symmetry underlying hidden order in URu$_2$Si$_2$

The low temperature hidden order state of URu$_2$Si$_2$ has long been a subject of intense speculation, and is thought to represent an as yet undetermined many-body quantum state not realized by other known materials. Here, X-ray absorption spectroscopy (XAS) and high resolution resonant inelastic X-ray scattering (RIXS) are used to observe electronic excitation spectra of URu$_2$Si$_2$, as a means to identify the degrees of freedom available to constitute the hidden order wavefunction. Excitations are shown to have symmetries that derive from a correlated $5f^2$ atomic multiplet basis that is modified by itinerancy. The features, amplitude and temperature dependence of linear dichroism are in agreement with ground states that closely resemble the doublet $\Gamma_5$ crystal field state of uranium.

cond-mat.mtrl-sci↗