arXiv subjects

Boyang Liu

Publications and source records attributed to Boyang Liu.

At least 37 records · Page 2Linked to original sources

arXiv2025

External Large Foundation Model: How to Efficiently Serve Trillions of Parameters for Online Ads Recommendation

Ads recommendation is a prominent service of online advertising systems and has been actively studied. Recent studies indicate that scaling-up and advanced design of the recommendation model can bring significant performance improvement. However, with a larger model scale, such prior studies have a significantly increasing gap from industry as they often neglect two fundamental challenges in industrial-scale applications. First, training and inference budgets are restricted for the model to be served, exceeding which may incur latency and impair user experience. Second, large-volume data arrive in a streaming mode with data distributions dynamically shifting, as new users/ads join and existing users/ads leave the system. We propose the External Large Foundation Model (ExFM) framework to address the overlooked challenges. Specifically, we develop external distillation and a data augmentation system (DAS) to control the computational cost of training/inference while maintaining high performance. We design the teacher in a way like a foundation model (FM) that can serve multiple students as vertical models (VMs) to amortize its building cost. We propose Auxiliary Head and Student Adapter to mitigate the data distribution gap between FM and VMs caused by the streaming data issue. Comprehensive experiments on internal industrial-scale applications and public datasets demonstrate significant performance gain by ExFM.

cs.IR↗

arXiv2025

TACTIC: Translation Agents with Cognitive-Theoretic Interactive Collaboration

Machine translation has long been a central task in natural language processing. With the rapid advancement of large language models (LLMs), there has been remarkable progress in translation quality. However, fully realizing the translation potential of LLMs remains an open challenge. Recent studies have explored multi-agent systems to decompose complex translation tasks into collaborative subtasks, showing initial promise in enhancing translation quality through agent cooperation and specialization. Nevertheless, existing multi-agent translation frameworks largely neglect foundational insights from cognitive translation studies. These insights emphasize how human translators employ different cognitive strategies, such as balancing literal and free translation, refining expressions based on context, and iteratively evaluating outputs. To address this limitation, we propose a cognitively informed multi-agent framework called TACTIC, which stands for T ranslation A gents with Cognitive- T heoretic Interactive Collaboration. The framework comprises six functionally distinct agents that mirror key cognitive processes observed in human translation behavior. These include agents for drafting, refinement, evaluation, scoring, context reasoning, and external knowledge gathering. By simulating an interactive and theory-grounded translation workflow, TACTIC effectively leverages the full capacity of LLMs for high-quality translation. Experimental results on diverse language pairs from the FLORES-200 and WMT24 benchmarks show that our method consistently achieves state-of-the-art performance. Using DeepSeek-V3 as the base model, TACTIC surpasses GPT-4.1 by an average of +0.6 XCOMET and +1.18 COMETKIWI-23. Compared to DeepSeek-R1, it further improves by +0.84 XCOMET and +2.99 COMETKIWI-23. Code is available at https://github.com/weiyali126/TACTIC.

cs.CL↗

arXiv2025

EliteKV: Scalable KV Cache Compression via RoPE Frequency Selection and Joint Low-Rank Projection

Rotary Position Embedding (RoPE) enables each attention head to capture multi-frequency information along the sequence dimension and is widely applied in foundation models. However, the nonlinearity introduced by RoPE complicates optimization of the key state in the Key-Value (KV) cache for RoPE-based attention. Existing KV cache compression methods typically store key state before rotation and apply the transformation during decoding, introducing additional computational overhead. This paper introduces EliteKV, a flexible modification framework for RoPE-based models supporting variable KV cache compression ratios. EliteKV first identifies the intrinsic frequency preference of each head using RoPElite, selectively restoring linearity to certain dimensions of key within attention computation. Building on this, joint low-rank compression of key and value enables partial cache sharing. Experimental results show that with minimal uptraining on only $0.6\%$ of the original training data, RoPE-based models achieve a $75\%$ reduction in KV cache size while preserving performance within a negligible margin. Furthermore, EliteKV consistently performs well across models of different scales within the same family.

cs.LG↗

arXiv2025

MasRouter: Learning to Route LLMs for Multi-Agent Systems

Multi-agent systems (MAS) powered by Large Language Models (LLMs) have been demonstrated to push the boundaries of LLM capabilities, yet they often incur significant costs and face challenges in dynamic LLM selection. Current LLM routing methods effectively reduce overhead in single-agent scenarios by customizing LLM selection for each query, but they overlook the critical decisions regarding collaboration modes and agent roles in MAS. In response to this challenge, we first introduce the problem of Multi-Agent System Routing (MASR), which integrates all components of MAS into a unified routing framework. Toward this goal, we propose MasRouter, the first high-performing, cost-effective, and inductive MASR solution. MasRouter employs collaboration mode determination, role allocation, and LLM routing through a cascaded controller network, progressively constructing a MAS that balances effectiveness and efficiency. Extensive experiments demonstrate that MasRouter is (1) high-performing, achieving a $1.8\%\sim8.2\%$ improvement over the state-of-the-art method on MBPP; (2) economical, reducing overhead by up to $52.07\%$ compared to SOTA methods on HumanEval; and (3) plug-and-play, seamlessly integrating with mainstream MAS frameworks, reducing overhead by $17.21\%\sim28.17\%$ via customized routing. The code is available at https://github.com/yanweiyue/masrouter.

cs.LG↗

arXiv2024

The Atomic Superfluid Quantum Interference Device with tunable Josephson Junctions

The atomic superfluid quantum interference device (ASQUID) with tunable Josephson junctions is theoretically investigated. ASQUID is a device that can be used for the detection of rotation. In this work we establish an analytical theory for the ASQUID using the tunneling Hamiltonian method and find two physical quantities that can be used for the rotation sensing. The first one is the critical population bias, which characterizes the transition between the self-trapping and the Josephson oscillation regimes and demonstrates a periodic modulation behavior due to the rotation of the system. We discuss the variation of critical population bias when the tunneling strengths of the junctions are tuned in different values, and find that the symmetric junctions are better choice than the asymmetric ones in terms of rotation sensing. Furthermore, how the initial phase difference between the two condensates affects the measurement of rotation is also discussed. Finally, we investigate the case of time-dependent junctions and find there is another physical quantity, named critical time, that can be used to detect the rotation.

cond-mat.quant-gas↗

arXiv2024

ConspEmoLLM: Conspiracy Theory Detection Using an Emotion-Based Large Language Model

The internet has brought both benefits and harms to society. A prime example of the latter is misinformation, including conspiracy theories, which flood the web. Recent advances in natural language processing, particularly the emergence of large language models (LLMs), have improved the prospects of accurate misinformation detection. However, most LLM-based approaches to conspiracy theory detection focus only on binary classification and fail to account for the important relationship between misinformation and affective features (i.e., sentiment and emotions). Driven by a comprehensive analysis of conspiracy text that reveals its distinctive affective features, we propose ConspEmoLLM, the first open-source LLM that integrates affective information and is able to perform diverse tasks relating to conspiracy theories. These tasks include not only conspiracy theory detection, but also classification of theory type and detection of related discussion (e.g., opinions towards theories). ConspEmoLLM is fine-tuned based on an emotion-oriented LLM using our novel ConDID dataset, which includes five tasks to support LLM instruction tuning and evaluation. We demonstrate that when applied to these tasks, ConspEmoLLM largely outperforms several open-source general domain LLMs and ChatGPT, as well as an LLM that has been fine-tuned using ConDID, but which does not use affective features. This project will be released on https://github.com/lzw108/ConspEmoLLM/.

cs.CL↗

arXiv2024

Development of Automated Neural Network Prediction for Echocardiographic Left ventricular Ejection Fraction

The echocardiographic measurement of left ventricular ejection fraction (LVEF) is fundamental to the diagnosis and classification of patients with heart failure (HF). In order to quantify LVEF automatically and accurately, this paper proposes a new pipeline method based on deep neural networks and ensemble learning. Within the pipeline, an Atrous Convolutional Neural Network (ACNN) was first trained to segment the left ventricle (LV), before employing the area-length formulation based on the ellipsoid single-plane model to calculate LVEF values. This formulation required inputs of LV area, derived from segmentation using an improved Jeffrey's method, as well as LV length, derived from a novel ensemble learning model. To further improve the pipeline's accuracy, an automated peak detection algorithm was used to identify end-diastolic and end-systolic frames, avoiding issues with human error. Subsequently, single-beat LVEF values were averaged across all cardiac cycles to obtain the final LVEF. This method was developed and internally validated in an open-source dataset containing 10,030 echocardiograms. The Pearson's correlation coefficient was 0.83 for LVEF prediction compared to expert human analysis (p<0.001), with a subsequent area under the receiver operator curve (AUROC) of 0.98 (95% confidence interval 0.97 to 0.99) for categorisation of HF with reduced ejection (HFrEF; LVEF<40%). In an external dataset with 200 echocardiograms, this method achieved an AUC of 0.90 (95% confidence interval 0.88 to 0.91) for HFrEF assessment. This study demonstrates that an automated neural network-based calculation of LVEF is comparable to expert clinicians performing time-consuming, frame-by-frame manual evaluation of cardiac systolic function.

cs.CV↗

arXiv2023

The Dynamical Stripes in Spin-Orbit Coupled Bose-Einstein Condensates with Josephson Junctions

The Josephson dynamics of the Bose-Einstein condensation with Raman-induced spin-orbit coupling is investigated. A quasi-1D trap is divided into two reservoirs by an optical barrier. Before the tunneling between the reservoirs is turned on, the system stays in its equilibrium ground state. For different spin-orbit coupling parameters and interaction strengthes, the ground state displays a rich phase diagram. In this work we focus on the plane wave phase and the stripe phase. Our calculation shows that, when the tunneling is turned on, the plane wave phase evolves into a dynamical stripe phase, that is, the density of the particle changes from uniform to periodically modulated. Basically, this stripe is described by a sine function and the wave length, the amplitude and the initial phase of the function are all varying with time. If the system stays in stripe phase initially, the stripes become ``sliding" when the tunneling is turned on, which reflects the running of one of the phases of the wave function.

cond-mat.quant-gas↗

arXiv2022

Atom-Specific Probing of Electron Dynamics in an Atomic Adsorbate by Time-Resolved X-ray Spectroscopy

The electronic excitation occurring on adsorbates at ultrafast time scales from optical lasers that initiate surface chemical reactions is still an open question. Here, we report the ultrafast temporal evolution of X-ray absorption spectroscopy (XAS) and X-ray emission spectroscopy (XES) of a simple well known adsorbate prototype system, namely carbon (C) atoms adsorbed on a nickel (Ni(100)) surface, following intense laser optical pumping at 400 nm. We observe ultrafast (~100 fs) changes in both XAS and XES showing clear signatures of the formation of a hot electron-hole pair distribution on the adsorbate. This is followed by slower changes on a few ps time scale, shown to be consistent with thermalization of the complete C/Ni system. Density functional theory spectrum simulations support this interpretation.

physics.chem-ph↗

arXiv2022

RUSH: Robust Contrastive Learning via Randomized Smoothing

Recently, adversarial training has been incorporated in self-supervised contrastive pre-training to augment label efficiency with exciting adversarial robustness. However, the robustness came at a cost of expensive adversarial training. In this paper, we show a surprising fact that contrastive pre-training has an interesting yet implicit connection with robustness, and such natural robustness in the pre trained representation enables us to design a powerful robust algorithm against adversarial attacks, RUSH, that combines the standard contrastive pre-training and randomized smoothing. It boosts both standard accuracy and robust accuracy, and significantly reduces training costs as compared with adversarial training. We use extensive empirical studies to show that the proposed RUSH outperforms robust classifiers from adversarial training, by a significant margin on common benchmarks (CIFAR-10, CIFAR-100, and STL-10) under first-order attacks. In particular, under $\ell_{\infty}$-norm perturbations of size 8/255 PGD attack on CIFAR-10, our model using ResNet-18 as backbone reached 77.8% robust accuracy and 87.9% standard accuracy. Our work has an improvement of over 15% in robust accuracy and a slight improvement in standard accuracy, compared to the state-of-the-arts.

cs.LG↗

arXiv2021

Particle and spin transports of spin-orbit coupled Fermi gas through a Quantum Point Contact

The particle and spin transport through a quantum point contact between two Fermi gases with Raman-induced spin-orbit coupling are investigated. We show that the particle and spin conductances both demonstrate the structure of plateau due to the mesoscopic scale of the quantum point contact. Compared with the normal Fermi gases the particle conductance can be significantly enhanced by the spin-orbit coupling effect. Furthermore, the conversion of the particle and spin currents can take place in the spin-orbit coupled system, and we find that it is controlled by the parameter of two-photon detuning. When the parameter of two-photon detuning vanishes the particle and spin currents decouple.

cond-mat.quant-gas↗

arXiv2021

Nesterov Accelerated ADMM for Fast Diffeomorphic Image Registration

Deterministic approaches using iterative optimisation have been historically successful in diffeomorphic image registration (DiffIR). Although these approaches are highly accurate, they typically carry a significant computational burden. Recent developments in stochastic approaches based on deep learning have achieved sub-second runtimes for DiffIR with competitive registration accuracy, offering a fast alternative to conventional iterative methods. In this paper, we attempt to reduce this difference in speed whilst retaining the performance advantage of iterative approaches in DiffIR. We first propose a simple iterative scheme that functionally composes intermediate non-stationary velocity fields to handle large deformations in images whilst guaranteeing diffeomorphisms in the resultant deformation. We then propose a convex optimisation model that uses a regularisation term of arbitrary order to impose smoothness on these velocity fields and solve this model with a fast algorithm that combines Nesterov gradient descent and the alternating direction method of multipliers (ADMM). Finally, we leverage the computational power of GPU to implement this accelerated ADMM solver on a 3D cardiac MRI dataset, further reducing runtime to less than 2 seconds. In addition to producing strictly diffeomorphic deformations, our methods outperform both state-of-the-art deep learning-based and iterative DiffIR approaches in terms of dice and Hausdorff scores, with speed approaching the inference time of deep learning-based methods.

cs.CV↗

arXiv2021

Ultrafast adsorbate excitation probed with sub-ps resolution XAS

We use a pump-probe scheme to measure the time evolution of the C K-edge X-ray absorption spectrum (XAS) from CO/Ru(0001) after excitation by an ultrashort high-intensity optical laser pulse. Due to the short duration of the X-ray probe pulse and precise control of the pulse delay, the excitation-induced dynamics during the first ps after the pump can be resolved with unprecedented time resolution. By comparing with theoretical (DFT) spectrum calculations we find high excitation of the internal stretch and frustrated rotation modes occurring within 200 fs of laser excitation, as well as thermalization of the system in the ps regime. The ~100 fs initial excitation of these CO vibrational modes is not readily rationalized by traditional theories of nonadiabatic coupling of adsorbates to metal surfaces, e. g. electronic frictions based on first order electron-phonon coupling or transient population of adsorbate resonances. We suggest that coupling of the adsorbate to non-thermalized electron-hole pairs is responsible for the ultrafast initial excitation of the modes.

physics.chem-ph↗

arXiv2021

Learning Deep Neural Networks under Agnostic Corrupted Supervision

Training deep neural models in the presence of corrupted supervision is challenging as the corrupted data points may significantly impact the generalization performance. To alleviate this problem, we present an efficient robust algorithm that achieves strong guarantees without any assumption on the type of corruption and provides a unified framework for both classification and regression problems. Unlike many existing approaches that quantify the quality of the data points (e.g., based on their individual loss values), and filter them accordingly, the proposed algorithm focuses on controlling the collective impact of data points on the average gradient. Even when a corrupted data point failed to be excluded by our algorithm, the data point will have a very limited impact on the overall loss, as compared with state-of-the-art filtering methods based on loss values. Extensive experiments on multiple benchmark datasets have demonstrated the robustness of our algorithm under different types of corruption.

cs.LG↗

arXiv2021

Polarons in a ferromagnetic spinor Bose-Einstein condensates

We investigate the polarons formed by immersing a spinor impurity in a ferromagnetic state of $F=1$ spinor Bose-Einstein condensate. The ground state energies and effective masses of the polarons are calculated in both weak-coupling regime and strong-coupling regime. In the weakly interacting regime the second order perturbation theory is performed. In the strong coupling regime we use a simple variational treatment. The analytical approximations to the energy and effective mass of the polarons are constructed. Especially, a transition from the mobile state to the self-trapping state of the polaron in the strong coupling regime is discussed. We also estimate the signatures of polaron effects in spinor BEC for the future experiments.

cond-mat.quant-gas↗

arXiv2020

Revealing the site selective local electronic structure of Co$_{3}$O$_{4}$ using $2p3d$ resonant inelastic X-ray scattering

We investigate mixed-valence oxide Co$_3$O$_4$ using Co $2p3d$ resonant inelastic X-ray scattering (RIXS). By setting resonant edges at Co$^{2+}$ and Co$^{3+}$ ions, the $dd$ excitations on the two Co sites are probed selectively, providing detailed information on the local electronic structure of Co$_3$O$_4$. The $2p3d$ RIXS result reveals the $^4$T$_{2}$ excited state of tetrahedral Co$^{2+}$ site at 0.5 eV beyond the discriminative power of optical absorption spectroscopies. Additionally, the $^3$T$_{2g}$ excited stated at 1.3 eV is uniquely identified for the octahedral Co$^{3+}$ site. Guided by cluster multiplet simulations, the ground-state character of the Co$^{2+}$ and Co$^{3+}$ site is determined to be high-spin $^4$A$_{2}$(T$_d$) and low-spin $^1$A$_{1g}$(O$_h$), respectively. This indicates that only the Co$^{2+}$ site is magnetically active site at low-temperatures in Co$_3$O$_4$. The ligand-to-metal charge transfer analysis suggests a formation of a strong covalent bonding between Co and O ions at the Co$^{3+}$ site, while Co$^{2+}$ is rather ionic.

cond-mat.mtrl-sci↗

arXiv2020

Quantum Chaos of Unitary Fermi Gases in Strong Pairing Fluctuation Region

The growth rate of the out-of-time-ordered correlator in a N-flavor Fermi gas is investigated and the Lyapunove exponent $λ_L$ is calculated to the order of $1/N$. We find that the Lyapunove exponent monotonically increases as the the interaction strength increases from the BCS limit to the unitary region. At the unitarity the Lyapunove exponent increases while the temperature drops and it can reach to the order of $λ_L\sim T$ around the critical temperature for the $N=1$ case. The system scrambles faster for stronger pairing fluctuations. At the BCS limit, the Lyapunov exponent behaviors as $λ_L\propto e^{μ/T} a^2_s T^2/N$.

cond-mat.quant-gas↗

arXiv2019

Tracing the Formation of Molecular Clouds in a Low-Metallicity Galaxy: A HI Narrow Self-Absorption Survey of the Large Magellanic Cloud

Cold atomic hydrogen clouds are the precursors of molecular clouds. Due to self-absorption, the opacity of cold atomic hydrogen may be high, and this gas may constitute an important mass component of the interstellar medium (ISM). Atomic hydrogen gas can be cooled to temperatures much lower than found in the cold neutral medium (CNM) through collisions with molecular hydrogen. In this paper, we search for HI Narrow Self-Absorption (HINSA) features in the Large Magellanic Cloud (LMC) as an indicator of such cold HI clouds, and use the results to quantify atomic masses and atomic-to-molecular gas ratio. Our search for HINSA features was conducted towards molecular clouds in the LMC using the ATCA+Parkes HI survey and the MAGMA CO survey. HINSA features are prevalent in the surveyed sightlines. This is the first detection of HINSA in an external galaxy. The HINSA-HI/$\rm{H}_{2}$ ratio in the LMC varies from 0.5\e{-3} to 3.4\e{-3} (68\% interval), with a mean value of $(1.31 \pm 0.03)$\e{-3}, after correcting for the effect of foreground HI gas. This is similar to the Milky Way value and indicates that similar fractions of cold gas exist in the LMC and the Milky Way, despite their differing metallicities, dust content and radiation fields. The low ratio also confirms that, as with the Milky Way, the formation timescale of molecular clouds is short. The ratio shows no radial gradient, unlike the case for stellar metallicity. No correlation is found between our results and those from previous HI absorption studies of the LMC.

astro-ph.GA↗