arXiv subjects

Xiaoyu Wang

Publications and source records attributed to Xiaoyu Wang.

At least 109 records · Page 6Linked to original sources

arXiv2025

Multi-Party Supervised Fine-tuning of Language Models for Multi-Party Dialogue Generation

Large Language Models (LLM) are usually fine-tuned to participate in dyadic or two-party dialogues, which can not adapt well to multi-party dialogues (MPD), which hinders their applications in such scenarios including multi-personal meetings, discussions and daily communication. Previous LLM-based researches mainly focus on the multi-agent framework, while their base LLMs are still pairwisely fine-tuned. In this work, we design a multi-party fine-tuning framework (MuPaS) for LLMs on the multi-party dialogue datasets, and prove such a straightforward framework can let the LLM align with the multi-party conversation style efficiently and effectively. We also design two training strategies which can convert MuPaS into the MPD simulator. Substantial experiments show that MuPaS can achieve state-of-the-art multi-party response, higher accuracy of the-next-speaker prediction, higher human and automatic evaluated utterance qualities, and can even generate reasonably with out-of-distribution scene, topic and role descriptions. The MuPaS framework bridges the LLM training with more complicated multi-party applications, such as conversation generation, virtual rehearsal or meta-universe.

cs.CL↗

arXiv2025

Reflect-then-Plan: Offline Model-Based Planning through a Doubly Bayesian Lens

Offline reinforcement learning (RL) is crucial when online exploration is costly or unsafe but often struggles with high epistemic uncertainty due to limited data. Existing methods rely on fixed conservative policies, restricting adaptivity and generalization. To address this, we propose Reflect-then-Plan (RefPlan), a novel doubly Bayesian offline model-based (MB) planning approach. RefPlan unifies uncertainty modeling and MB planning by recasting planning as Bayesian posterior estimation. At deployment, it updates a belief over environment dynamics using real-time observations, incorporating uncertainty into MB planning via marginalization. Empirical results on standard benchmarks show that RefPlan significantly improves the performance of conservative offline RL policies. In particular, RefPlan maintains robust performance under high epistemic uncertainty and limited data, while demonstrating resilience to changing environment dynamics, improving the flexibility, generalizability, and robustness of offline-learned policies.

cs.AI↗

arXiv2025

Observation of giant nonlinear Hall conductivity in Bernal bilayer graphene

In a system of two-dimensional electrons, a combination of broken symmetry, interactions, and nontrivial topology can conspire to give rise to a nonlinear transport regime, where electric current density scales as the square of electric field. This regime has become a venue for exciting discoveries such as the nonlinear Hall effect and diode-like nonreciprocal transport. However, interpretation of experimental data is challenging in the nonlinear regime as DC transport is described by a rank-3 conductivity tensor with 6 free parameters. Here, we resolve this challenge by analytically solving for the nonlinear potential distribution across the disk sample for an arbitrary linear and nonlinear conductivity tensors. This allows us to unambiguously extract all components of the nonlinear tensor from experimental measurement. Using this novel tool, we identify giant nonlinear Hall effect in Bernal bilayer graphene. Our methodology provides the first systematic framework for interpreting nonlinear transport and uncovers a new route towards understanding quasi-2D materials.

cond-mat.mes-hall↗

arXiv2025

DenseLoRA: Dense Low-Rank Adaptation of Large Language Models

Low-rank adaptation (LoRA) has been developed as an efficient approach for adapting large language models (LLMs) by fine-tuning two low-rank matrices, thereby reducing the number of trainable parameters. However, prior research indicates that many of the weights in these matrices are redundant, leading to inefficiencies in parameter utilization. To address this limitation, we introduce Dense Low-Rank Adaptation (DenseLoRA), a novel approach that enhances parameter efficiency while achieving superior performance compared to LoRA. DenseLoRA builds upon the concept of representation fine-tuning, incorporating a single Encoder-Decoder to refine and compress hidden representations across all adaptation layers before applying adaptation. Instead of relying on two redundant low-rank matrices as in LoRA, DenseLoRA adapts LLMs through a dense low-rank matrix, improving parameter utilization and adaptation efficiency. We evaluate DenseLoRA on various benchmarks, showing that it achieves 83.8% accuracy with only 0.01% of trainable parameters, compared to LoRA's 80.8% accuracy with 0.70% of trainable parameters on LLaMA3-8B. Additionally, we conduct extensive experiments to systematically assess the impact of DenseLoRA's components on overall model performance. Code is available at https://github.com/mulin-ahu/DenseLoRA.

cs.CL↗

arXiv2025

ScaleBiO: Scalable Bilevel Optimization for LLM Data Reweighting

Bilevel optimization has shown its utility across various machine learning settings, yet most algorithms in practice require second-order information, making it challenging to scale them up. Only recently, a paradigm of first-order algorithms has emerged in the theoretical literature, capable of effectively addressing bilevel optimization problems. Nevertheless, the practical efficiency of this paradigm remains unverified, particularly in the context of large language models (LLMs). This paper introduces the first scalable instantiation of this paradigm called ScaleBiO, focusing on bilevel optimization for large-scale LLM data reweighting. By combining with a recently proposed memory-efficient training technique called LISA, our novel algorithm allows the paradigm to scale to $\sim$30B-sized LLMs on $8\times$H100 GPUs, marking the first successful application of bilevel optimization under practical scenarios for large-sized LLMs. Empirically, extensive experiments on data reweighting verify the effectiveness of ScaleBiO for different-scaled models, including Llama-3-8B, Gemma-2-9B, Qwen-2-7B, and Qwen-2.5-32B, where bilevel optimization succeeds in instruction-following and math reasoning tasks, outperforming several popular baselines, including uniform sampling, influence-aware data filtering, and reference-model-based sampling methods. Theoretically, ScaleBiO ensures the optimality of the learned data weights, along with a convergence guarantee matching the conventional first-order bilevel optimization paradigm on smooth and strongly convex objectives.

cs.LG↗

arXiv2025

Discontinuous hybrid neural networks for the one-dimensional partial differential equations

A feedforward neural network, including hidden layers, motivated by nonlinear functions (such as Tanh, ReLU, and Sigmoid functions), exhibits uniform approximation properties in Sobolev space, and discontinuous neural networks can reduce computational complexity. In this work, we present a discontinuous hybrid neural network method for solving the partial differential equations, construct a new hybrid loss functional that incorporates the variational of the approximation equation, interface jump stencil and boundary constraints. The RMSprop algorithm and discontinuous Galerkin method are employed to update the nonlinear parameters and linear parameters in neural networks, respectively. This approach guarantees the convergence of the loss functional and provides an approximate solution with high accuracy.

math.NA↗

arXiv2025

Beam single spin asymmetry $A_{LU}^{\sin ϕ_R}$ of the dihadron production in SIDIS process

We study the longitudinal beam single-spin asymmetry $A_{LU}^{\sin ϕ_R}$ of dihadron production in semi-inclusive deep inelastic scattering process with a polarized electron beam scattering off an unpolarized proton target. The asymmetry arises from the convolutions of twist-3 PDF $e(x)$ and twist-2 DiFF $H_1^\sphericalangle$ as well as twist-2 PDF $f_1(x)$ and twist-3 DiFF $\widetilde{G}^{\sphericalangle}$. Using the spectator model calculations for the twist-3 PDF $e(x)$, the DiFFs $D_1 , H_1^\sphericalangle$, and $\widetilde{G}^{\sphericalangle}$, we estimate the BSA $A^{\sin ϕ_R}_{LU}$ at the kinematical configuration of CLAS and CLAS12. We find good agreement with $z$- and $M_h$-dependencies data, though $x$- and $Q^2$- dependencies show discrepancies. We include DGLAP evolution of $e(x)$ and provide predictions for COMPASS, EIC, and EicC. The negligible contribution from $f_1(x) \otimes \widetilde{G}^{\sphericalangle}$ supports the Wandzura-Wilczek approximation, while our results highlight the importance of QCD evolution in twist-3 PDF analyses.

hep-ph↗

arXiv2025

Probing orbital magnetism of a kagome metal CsV3Sb5 by a tuning fork resonator

The recently discovered kagome metal CsV$_3$Sb$_5$ exhibits a complex phase diagram that encompasses frustrated magnetism, topological charge density wave (CDW), and superconductivity. One CDW state that breaks time-reversal symmetry was proposed in this compound, while the exact nature of the putative magnetic state remains elusive. To examine the thermodynamic state of CsV$_3$Sb$_5$ and assess the character of the associated magnetism, we conducted tuning fork resonator measurements of magnetotropic susceptibility over a broad range of angles, magnetic fields, and temperature. We found a cascade of phase transition in the CDW phase. Of particular interest is a highly anisotropic magnetic structure that arises below about 30~K, with a magnetic moment along the $c$-axis that has an extremely small magnitude. This magnetic state demonstrates extremely slow dynamics and small saturate field, all suggest that electronic phase below 30~K breaks time reversal symmetry and has an unconventional origin.

cond-mat.mtrl-sci↗

arXiv2025

Lab-AI: Using Retrieval Augmentation to Enhance Language Models for Personalized Lab Test Interpretation in Clinical Medicine

Accurate interpretation of lab results is crucial in clinical medicine, yet most patient portals use universal normal ranges, ignoring conditional factors like age and gender. This study introduces Lab-AI, an interactive system that offers personalized normal ranges using retrieval-augmented generation (RAG) from credible health sources. Lab-AI has two modules: factor retrieval and normal range retrieval. We tested these on 122 lab tests: 40 with conditional factors and 82 without. For tests with factors, normal ranges depend on patient-specific information. Our results show GPT-4-turbo with RAG achieved a 0.948 F1 score for factor retrieval and 0.995 accuracy for normal range retrieval. GPT-4-turbo with RAG outperformed the best non-RAG system by 33.5% in factor retrieval and showed 132% and 100% improvements in question-level and lab-level performance, respectively, for normal range retrieval. These findings highlight Lab-AI's potential to enhance patient understanding of lab results.

cs.CL↗

arXiv2025

Infinitely many solutions for a biharmonic-Kirchhoff system on locally finite graphs

The study on the partial differential equations (systems) in the graph setting is a hot topic in recent years because of their applications to image processing and data clustering. Our motivation is to develop some existence results for biharmonic-Kirchhoff systems and biharmonic systems in the Euclidean setting, which are the continuous models, to the corresponding systems in the locally finite graph setting, which are the discrete models. We mainly focus on the existence of infinitely many solutions for a biharmonic-Kirchhoff system on a locally finite graph. The method is variational and the main tool is the symmetric mountain pass theorem. We obtain that the system has infinitely many solutions when the nonlinear term admits the super-$4$ linear growth, and we also present the corresponding results to the biharmonic system. We also find that the results in the locally finite graph setting are better than that in the Euclidean setting, which caused by the better embedding theorem in the locally finite graph.

math.AP↗

arXiv2025

Non-Reversible Langevin Algorithms for Constrained Sampling

We consider the constrained sampling problem where the goal is to sample from a target distribution on a constrained domain. We propose skew-reflected non-reversible Langevin dynamics (SRNLD), a continuous-time stochastic differential equation with skew-reflected boundary. We obtain non-asymptotic convergence rate of SRNLD to the target distribution in both total variation and 1-Wasserstein distances. By breaking reversibility, we show that the convergence is faster than the special case of the reversible dynamics. Based on the discretization of SRNLD, we propose skew-reflected non-reversible Langevin Monte Carlo (SRNLMC), and obtain non-asymptotic discretization error from SRNLD, and convergence guarantees to the target distribution in 1-Wasserstein distance. We show better performance guarantees than the projected Langevin Monte Carlo in the literature that is based on the reversible dynamics. Numerical experiments are provided for both synthetic and real datasets to show efficiency of the proposed algorithms.

cs.LG↗

arXiv2025

Large Language Model Can Be a Foundation for Hidden Rationale-Based Retrieval

Despite the recent advancement in Retrieval-Augmented Generation (RAG) systems, most retrieval methodologies are often developed for factual retrieval, which assumes query and positive documents are semantically similar. In this paper, we instead propose and study a more challenging type of retrieval task, called hidden rationale retrieval, in which query and document are not similar but can be inferred by reasoning chains, logic relationships, or empirical experiences. To address such problems, an instruction-tuned Large language model (LLM) with a cross-encoder architecture could be a reasonable choice. To further strengthen pioneering LLM-based retrievers, we design a special instruction that transforms the retrieval task into a generative task by prompting LLM to answer a binary-choice question. The model can be fine-tuned with direct preference optimization (DPO). The framework is also optimized for computational efficiency with no performance degradation. We name this retrieval framework by RaHoRe and verify its zero-shot and fine-tuned performance superiority on Emotional Support Conversation (ESC), compared with previous retrieval works. Our study suggests the potential to employ LLM as a foundation for a wider scope of retrieval tasks. Our codes, models, and datasets are available on https://github.com/flyfree5/LaHoRe.

cs.IR↗

arXiv2025

Signaling Human Intentions to Service Robots: Understanding the Use of Social Cues during In-Person Conversations

As social service robots become commonplace, it is essential for them to effectively interpret human signals, such as verbal, gesture, and eye gaze, when people need to focus on their primary tasks to minimize interruptions and distractions. Toward such a socially acceptable Human-Robot Interaction, we conducted a study ($N=24$) in an AR-simulated context of a coffee chat. Participants elicited social cues to signal intentions to an anthropomorphic, zoomorphic, grounded technical, or aerial technical robot waiter when they were speakers or listeners. Our findings reveal common patterns of social cues over intentions, the effects of robot morphology on social cue position and conversational role on social cue complexity, and users' rationale in choosing social cues. We offer insights into understanding social cues concerning perceptions of robots, cognitive load, and social context. Additionally, we discuss design considerations on approaching, social cue recognition, and response strategies for future service robots.

cs.HC↗

arXiv2025

Realizing a topological diode effect on the surface of a topological Kondo insulator

Introducing the concept of topology into material science has sparked a revolution from classic electronic and optoelectronic devices to topological quantum devices. The latter has potential for transferring energy and information with unprecedented efficiency. Here, we demonstrate a topological diode effect on the surface of a three-dimensional material, SmB6, a candidate topological Kondo insulator. The diode effect is evidenced by pronounced rectification and photogalvanic effects under electromagnetic modulation and radiation at radio frequency. Our experimental results and modeling suggest that these prominent effects are intimately tied to the spatially inhomogeneous formation of topological surface states (TSS) at the intermediate temperature. This work provides a manner of breaking the mirror symmetry (in addition to the inversion symmetry), resulting in the formation of pn-junctions between puddles of metallic TSS. This effect paves the way for efficient current rectifiers or energy-harvesting devices working down to radio frequency range at low temperature, which could be extended to high temperatures using other topological insulators with large bulk gap.

cond-mat.str-el↗

arXiv2025

Multi-constraint Graph Partitioning Problems Via Recursive Bipartition Algorithm Based on Subspace Minimization Conjugate Gradient Method

The graph partitioning problem is a well-known NP-hard problem. In this paper, we formulate a 0-1 quadratic integer programming model for the graph partitioning problem with vertex weight constraints and fixed vertex constraints, and propose a recursive bipartition algorithm based on the subspace minimization conjugate gradient method. To alleviate the difficulty of solving the model, the constrained problem is transformed into an unconstrained optimization problem using equilibrium terms, elimination methods, and trigonometric properties, and solved via an accelerated subspace minimization conjugate gradient algorithm. Initial feasible partitions are generated using a hyperplane rounding algorithm, followed by heuristic refinement strategies, including one-neighborhood and two-interchange adjustments, to iteratively improve the results. Numerical experiments on knapsack-constrained graph partitioning and industrial examples demonstrate the effectiveness and feasibility of the proposed algorithm.

math.OC↗

arXiv2025

Fully First-Order Methods for Decentralized Bilevel Optimization

This paper focuses on decentralized stochastic bilevel optimization (DSBO) where agents only communicate with their neighbors. We propose Decentralized Stochastic Gradient Descent and Ascent with Gradient Tracking (DSGDA-GT), a novel algorithm that only requires first-order oracles that are much cheaper than second-order oracles widely adopted in existing works. We further provide a finite-time convergence analysis showing that for $n$ agents collaboratively solving the DSBO problem, the sample complexity of finding an $ε$-stationary point in our algorithm is $\mathcal{O}(n^{-1}ε^{-7})$, which matches the currently best-known results of the single-agent counterpart with linear speedup. The numerical experiments demonstrate both the communication and training efficiency of our algorithm.

math.OC↗

arXiv2025

A Novel Interactive-Guided Differential Testing Approach for FPGA Simulation Debugger Tools

Field-Programmable Gate Array (FPGA) development tool chains are widely used in FPGA design, simulation, and verification in critical areas like communications, automotive electronics, and aerospace. Commercial FPGA tool chains such as Xilinx' Vivado aids developers in swiftly identifying and rectifying bugs and issues in FPGA designs through a robust built-in debugger, ensuring the correctness and development efficiency of the FPGA design. Hardening such FPGA chip debugger tools by testing is crucial since engineers might misinterpret code and introduce incorrect fixes, leading to security risks. However, FPGA chip debugger tools are challenging to test as they require assessing both RTL designs and a series of debugging actions, including setting breakpoints and stepping through the code. To address this issue, we propose a interactive differential testing approach called DB-Hunter to detect bugs in Vivado's FPGA chip debugger tools. Specifically, DB-Hunter consists of three components: RTL design transformation component, debug action transformation component, and interactive differential testing component. By performing RTL design and debug action transformations, DB-Hunter generates diverse and complex RTL designs and debug actions, to thoroughly test the Vivado debugger using interactive differential testing to detect bugs. In three months, DB-Hunter reported 18 issues, including 10 confirmed as bugs by Xilinx Support, 6 bugs had been fixed in last version.

cs.SE↗

arXiv2025

Theoretical study of the open-flavor tetraquark $T_{c\bar{s}}(2900)$ in the process $\bar{B}_s^0 \to K^0 D^0 π^0$

Recently, the LHCb Collaboration has measured two decay processes $B^0\to\bar{D}^0D_s^+π^-$ and $B^+\to D^-D_s^+π^+$ related to isospin symmetry, where two new open-flavor tetraquark states $T_{c\bar{s}}(2900)^0$ and $T_{c\bar{s}}(2900)^{++}$ that belong to an isospin triplet were observed in the $D_s^+π^-$ and $D_s^+π^+$ invariant mass distributions. In this work, we have investigated the validity of the process $\bar{B}_s^0\to K^0D^0π^0$ as the promising process to confirm the existence of $T_{c\bar{s}}(2900)^0$ resonance. Taking into account the tetraquark state $T_{c\bar{s}}(2900)$, as well as intermediate resonances $K^*(892)$, $K_0^*(1430)$, and $K_2^*(1430)$, it has been shown that a clear peak of the open-flavor tetraquark $T_{c\bar{s}}(2900)$ appears in the $K^0D^0$ invariant mass distribution of the process $\bar{B}_s^0\to K^0D^0π^0$, which could be tested by future experiments.

hep-ph↗