arXiv subjects

Lei Li

Publications and source records attributed to Lei Li.

At least 253 records · Page 14Linked to original sources

arXiv2025

Propagation of chaos and approximation error of random batch particle system in the mean field regime

The random batch method [J. Comput. Phys. 400 (2020) 108877] is not only an efficient algorithm for simulation of classical $N$-particle systems and their mean-field limit, but also a new model for interacting particle system that could be more physical in some applications. In this work, we establish the propagation of chaos for the random batch particle system and at the same time obtain its sharp approximation error to the classical mean field limit of $N$-particle systems. The proof leverages the BBGKY hierarchy and achieves a sharp bound both in the particle number $N$ and the time step $τ$. In particular, by introducing a coupling of the division of the random batches to resolve the $N$-dependence, we derive an $\mathcal{O}(k^2/N^2 + kτ^2)$ bound on the $k$-particle relative entropy between the law of the system and the tensorized law of the mean-field limit. This result provides a useful understanding of the convergence properties of the random batch system in the mean field regime.

math.NA↗

arXiv2025

A generalized discontinuous Hamilton Monte Carlo for transdimensional sampling

In this paper, we propose a discontinuous Hamilton Monte Carlo (DHMC) to sample from dimensional varying distributions, and particularly the grand canonical ensemble. The DHMC was proposed in [Biometrika, 107(2)] for discontinuous potential where the variable has a fixed dimension. When the dimension changes, there is no clear explanation of the volume-preserving property, and the conservation of energy is also not necessary. We use a random sampling for the extra dimensions, which corresponds to a measure transform. We show that when the energy is corrected suitably for the trans-dimensional Hamiltonian dynamics, the detailed balance condition is then satisfied. For the grand canonical ensemble, such a procedure can be explained very naturally to be the extra free energy change brought by the newly added particles, which justifies the rationality of our approach. To sample the grand canonical ensemble for interacting particle systems, the DHMC is then combined with the random batch method to yield an efficient sampling method. In experiments, we show that the proposed DHMC combined with the random batch method generates samples with much less correlation when compared with the traditional Metropolis-Hastings method.

math.NA↗

arXiv2025

Generation and Stabilization of Bound States in the Continuum in Dissipative Floquet Optical Lattices

This paper investigates the generation and stabilization of bound states in the continuum (BICs) in a one-dimensional dissipative Floquet lattice. We find a different mechanism for the generation of stable BICs in the open one-dimensional lattice system, which stems from a peculiar dark Floquet state, a state with zero quasi-energy and negligible population on the lossy sites. Our results reveal that the evolutionary stability of BICs resulting from the dark Floquet state can be significantly enhanced, as evidenced by their very low decay rate, by increasing the driving frequency or, counterintuitively, increasing the dissipation strength. We further demonstrate that stable dark Floquet BICs can robustly persist even in nonlinear regimes. The existence of these stable dark Floquet BICs can be attributed to the role of higher-order correction terms in the effective Floquet Hamiltonian derived via the high-frequency expansion (HFE) method. Furthermore, we demonstrate that incorporating non-Hermitian dissipation can extend the parameter regime for the existence of BICs, and the dissipation-induced BICs can lead to complete reflection of wave packets. Our findings provide theoretical support for the experimental realization of stable BICs in dissipative quantum systems.

quant-ph↗

arXiv2025

Mind2Matter: Creating 3D Models from EEG Signals

The reconstruction of 3D objects from brain signals has gained significant attention in brain-computer interface (BCI) research. Current research predominantly utilizes functional magnetic resonance imaging (fMRI) for 3D reconstruction tasks due to its excellent spatial resolution. Nevertheless, the clinical utility of fMRI is limited by its prohibitive costs and inability to support real-time operations. In comparison, electroencephalography (EEG) presents distinct advantages as an affordable, non-invasive, and mobile solution for real-time brain-computer interaction systems. While recent advances in deep learning have enabled remarkable progress in image generation from neural data, decoding EEG signals into structured 3D representations remains largely unexplored. In this paper, we propose a novel framework that translates EEG recordings into 3D object reconstructions by leveraging neural decoding techniques and generative models. Our approach involves training an EEG encoder to extract spatiotemporal visual features, fine-tuning a large language model to interpret these features into descriptive multimodal outputs, and leveraging generative 3D Gaussians with layout-guided control to synthesize the final 3D structures. Experiments demonstrate that our model captures salient geometric and semantic features, paving the way for applications in brain-computer interfaces (BCIs), virtual reality, and neuroprosthetics. Our code is available in https://github.com/sddwwww/Mind2Matter.

cs.GR↗

arXiv2025

LT3SD: Latent Trees for 3D Scene Diffusion

We present LT3SD, a novel latent diffusion model for large-scale 3D scene generation. Recent advances in diffusion models have shown impressive results in 3D object generation, but are limited in spatial extent and quality when extended to 3D scenes. To generate complex and diverse 3D scene structures, we introduce a latent tree representation to effectively encode both lower-frequency geometry and higher-frequency detail in a coarse-to-fine hierarchy. We can then learn a generative diffusion process in this latent 3D scene space, modeling the latent components of a scene at each resolution level. To synthesize large-scale scenes with varying sizes, we train our diffusion model on scene patches and synthesize arbitrary-sized output 3D scenes through shared diffusion generation across multiple scene patches. Through extensive experiments, we demonstrate the efficacy and benefits of LT3SD for large-scale, high-quality unconditional 3D scene generation and for probabilistic completion for partial scene observations.

cs.CV↗

arXiv2025

Speculative Knowledge Distillation: Bridging the Teacher-Student Gap Through Interleaved Sampling

Recent advances in knowledge distillation (KD) have enabled smaller student models to approach the performance of larger teacher models. However, popular methods such as supervised KD and on-policy KD, are adversely impacted by the knowledge gaps between teacher-student in practical scenarios. Supervised KD suffers from a distribution mismatch between training with a static dataset and inference over final student-generated outputs. Conversely, on-policy KD, which uses student-generated samples for training, can suffer from low-quality training examples with which teacher models are not familiar, resulting in inaccurate teacher feedback. To address these limitations, we introduce Speculative Knowledge Distillation (SKD), a novel approach that leverages cooperation between student and teacher models to generate high-quality training data on-the-fly while aligning with the student's inference-time distribution. In SKD, the student proposes tokens, and the teacher replaces poorly ranked ones based on its own distribution, transferring high-quality knowledge adaptively. We evaluate SKD on various text generation tasks, including translation, summarization, math, and instruction following, and show that SKD consistently outperforms existing KD methods across different domains, data sizes, and model initialization strategies.

cs.CL↗

arXiv2025

Quantum-Enhanced LLM Efficient Fine Tuning

Low-Rank Adaptation (LoRA) enables efficient fine-tuning of pre-trained language models through low-rank matrix approximation, achieving effectiveness in many scenarios. However, its representation capacity is constrained in complex tasks or high-rank dependency settings, potentially limiting model adaptability. To overcome the expressive bottleneck in classical low-rank approximation for fine-tuning large language models (LLMs), we propose Quantum Tensor Hybrid Adaptation (QTHA), a parameter-efficient fine-tuning method that integrates a quantum neural network (QNN) with a tensor network. QTHA explores quantum tensor hybrid fine-tuning within low-rank spaces by decomposing pre-trained weights into quantum neural network and tensor network representations, leveraging quantum state superposition to overcome classical rank limitations. Experiments demonstrate that QTHA achieves performance comparable to or surpassing LoRA in parameter-efficient fine-tuning. Compared to LoRA, QTHA reduces trainable parameters by 76% while reducing training loss by up to 17% and improving test set performance by up to 17% within the same training steps. This research not only enables lightweight adaptation of quantum resources to the billion-parameter models but also validates the feasibility of quantum hardware optimization driven by LLM tasks. It establishes the first engineering-ready foundation for future quantum-enhanced Artificial General Intelligence (AGI) systems.

quant-ph↗

arXiv2025

Equilibrium-diffusion limit of the radiation model

We justify rigorously the equilibrium-diffusion limit of the model consists of a radiative transfer satisfied by the specific intensity of radiation coupled to a diffusion equation satisfied by the material temperature. For general initial data, we construct the existence of the solution to the coupled model in $\mathbb{T}^{3}$ by the Hilbert expansion and prove the convergence of the solutions to the limiting system in the equilibrium-diffusion regime. Moreover, the initial layer for the radiative density and the temperature are constructed to get the strong convergence in $L^\infty$ norm. We also get the convergence rates about the intensity of radiation and temperature in this paper.

math.AP↗

arXiv2025

Measurement of the inner horizon in the analog of rotating BTZ black holes by an improved photon fluid

We study how to include the inner horizon in the analog of rotating black holes using photon fluids. We find that a vortex beam carrying an improved phase can simulate the rotating BTZ black holes experimentally. In the experiment, we develop a new photon fluid model in a graphene/methanol thermal optical solution, and measure the variation of photon fluid velocity with the radial position using a Fourier plane light spot localization method, while also determining the variation of phonon velocity with the same radial position from the optical vortex intensity distribution. The result provides an extension for the application of optical vortex and a potential possibility for the future experimental exploration about the properties of BTZ black holes and even the anti-de Sitter space.

gr-qc↗

arXiv2025

TimeChat-Online: 80% Visual Tokens are Naturally Redundant in Streaming Videos

The rapid growth of online video platforms, particularly live streaming services, has created an urgent need for real-time video understanding systems. These systems must process continuous video streams and respond to user queries instantaneously, presenting unique challenges for current Video Large Language Models (VideoLLMs). While existing VideoLLMs excel at processing complete videos, they face significant limitations in streaming scenarios due to their inability to handle dense, redundant frames efficiently. We introduce TimeChat-Online, a novel online VideoLLM that revolutionizes real-time video interaction. At its core lies our innovative Differential Token Drop (DTD) module, which addresses the fundamental challenge of visual redundancy in streaming videos. Drawing inspiration from human visual perception's Change Blindness phenomenon, DTD preserves meaningful temporal changes while filtering out static, redundant content between frames. Remarkably, our experiments demonstrate that DTD achieves an 82.8% reduction in video tokens while maintaining 98% performance on StreamingBench, revealing that over 80% of visual content in streaming videos is naturally redundant without requiring language guidance. To enable seamless real-time interaction, we present TimeChat-Online-139K, a comprehensive streaming video dataset featuring diverse interaction patterns including backward-tracing, current-perception, and future-responding scenarios. TimeChat-Online's unique Proactive Response capability, naturally achieved through continuous monitoring of video scene transitions via DTD, sets it apart from conventional approaches. Our extensive evaluation demonstrates TimeChat-Online's superior performance on streaming benchmarks (StreamingBench and OvOBench) and maintaining competitive results on long-form video tasks such as Video-MME and MLVU.

cs.CV↗

arXiv2025

TypedThinker: Diversify Large Language Model Reasoning with Typed Thinking

Large Language Models (LLMs) have demonstrated strong reasoning capabilities in solving complex problems. However, current approaches primarily enhance reasoning through the elaboration of thoughts while neglecting the diversity of reasoning types. LLMs typically employ deductive reasoning, proceeding step-by-step from given conditions, which limits their exploration during problem-solving. Our analysis reveals that certain problems are exclusively solvable through specific reasoning strategies like inductive, abductive, or analogical reasoning. However, incorporating diverse reasoning approaches presents two key challenges: identifying the appropriate reasoning type for each problem and exploiting this approach during problem-solving. Therefore, we propose the TypedThinker that predicts suitable reasoning types based on the problem and their previous effectiveness and provides relevant demonstrations to guide LLMs in applying these strategies. Experimental results show significant improvements across multiple benchmarks, with performance gains of 3.4% for Mistral 7B, 6.5% for LLaMA3 8B, and 7% for Qwen 2 7B on logical and mathematical reasoning tasks. TypedThinker enhances LLM reasoning without requiring knowledge distillation from larger models. It can be integrated into more advanced systems like GPT-4o or specialized models like MetaMath to diversify their reasoning approaches and improve their problem-solving capabilities.

cs.CL↗

arXiv2025

SeaLLM: Service-Aware and Latency-Optimized Resource Sharing for Large Language Model Inference

Large language models (LLMs) with different architectures and sizes have been developed. Serving each LLM with dedicated GPUs leads to resource waste and service inefficiency due to the varying demand of LLM requests. A common practice is to share multiple LLMs. However, existing sharing systems either do not consider the autoregressive pattern of LLM services, or only focus on improving the throughput, which impairs the sharing performance, especially the serving latency. We present SeaLLM, which enables service-aware and latency-optimized LLM sharing. SeaLLM improves the overall sharing performance by (1) a latency-optimized scheduling algorithm utilizing the characteristics of LLM services, (2) a placement algorithm to determine the placement plan and an adaptive replacement algorithm to decide the replacement interval, and (3) a unified key-value cache to share GPU memory among LLM services efficiently. Our evaluation under real-world traces and LLM services demonstrates that SeaLLM improves the normalized latency by up to $13.60\times$, the tail latency by up to $18.69\times$, and the SLO attainment by up to $3.64\times$ compared to existing solutions.

cs.DC↗

arXiv2025

BenchMAX: A Comprehensive Multilingual Evaluation Suite for Large Language Models

Previous multilingual benchmarks focus primarily on simple understanding tasks, but for large language models(LLMs), we emphasize proficiency in instruction following, reasoning, long context understanding, code generation, and so on. However, measuring these advanced capabilities across languages is underexplored. To address the disparity, we introduce BenchMAX, a multi-way multilingual evaluation benchmark that allows for fair comparisons of these important abilities across languages. To maintain high quality, three distinct native-speaking annotators independently annotate each sample within all tasks after the data was machine-translated from English into 16 other languages. Additionally, we present a novel translation challenge stemming from dataset construction. Extensive experiments on BenchMAX reveal varying effectiveness of core capabilities across languages, highlighting performance gaps that cannot be bridged by simply scaling up model size. BenchMAX serves as a comprehensive multilingual evaluation platform, providing a promising test bed to promote the development of multilingual language models. The dataset and code are publicly accessible.

cs.CL↗

arXiv2025

Jailbreaking as a Reward Misspecification Problem

The widespread adoption of large language models (LLMs) has raised concerns about their safety and reliability, particularly regarding their vulnerability to adversarial attacks. In this paper, we propose a novel perspective that attributes this vulnerability to reward misspecification during the alignment process. This misspecification occurs when the reward function fails to accurately capture the intended behavior, leading to misaligned model outputs. We introduce a metric ReGap to quantify the extent of reward misspecification and demonstrate its effectiveness and robustness in detecting harmful backdoor prompts. Building upon these insights, we present ReMiss, a system for automated red teaming that generates adversarial prompts in a reward-misspecified space. ReMiss achieves state-of-the-art attack success rates on the AdvBench benchmark against various target aligned LLMs while preserving the human readability of the generated prompts. Furthermore, these attacks on open-source models demonstrate high transferability to closed-source models like GPT-4o and out-of-distribution tasks from HarmBench. Detailed analysis highlights the unique advantages of the proposed reward misspecification objective compared to previous methods, offering new insights for improving LLM safety and robustness.

cs.LG↗

arXiv2025

A note on the lower bounds of genuine multipartite entanglement concurrence

Quantum entanglement plays a pivotal role in quantum information processing. Quantifying quantum entanglement is a challenging and essential research area within the field. This manuscript explores the relationships between bipartite entanglement concurrence, multipartite entanglement concurrence, and genuine multipartite entanglement (GME) concurrence. We derive lower bounds on GME concurrence from these relationships, demonstrating their superiority over existing results through rigorous proofs and numerical examples. Additionally, we investigate the connections between GME concurrence and other entanglement measures, such as tangle and global negativity, in multipartite quantum systems.

quant-ph↗

arXiv2025

Tight upper bound and monogamy relation for the maximum quantum value of the parity-CHSH inequality and applied to device-independent randomness

Based on the violation of Bell inequalities, we can verify quantum random numbers by examining the correlation between device inputs and outputs. In this paper, we derive the maximum quantum value of the parity-CHSH inequality for a three-qubit system, establishing a tight upper bound applicable to any quantum state. Simultaneously, the necessary constraints for achieving saturation are analyzed. Utilizing this method, we present necessary and sufficient conditions for certain states to violate the parity-CHSH inequality. Building upon our proposal, the relationship between the noise parameter and the certifiable randomness in a bipartite entangled state is probed. Furthermore, we derive a monogamy relationship between the average values of the parity-CHSH inequality associated with the reduced three-qubit density matrices of GHZ-class states comprising four qubits.

quant-ph↗

arXiv2025

Could Thinking Multilingually Empower LLM Reasoning?

Previous work indicates that large language models exhibit a significant "English bias", i.e. they often perform better when tasks are presented in English. Interestingly, we have observed that using certain other languages in reasoning tasks can yield better performance than English. However, this phenomenon remains under-explored. In this paper, we explore the upper bound of harnessing multilingualism in reasoning tasks, suggesting that multilingual reasoning promises significantly (by nearly 10 Acc@$k$ points) and robustly (tolerance for variations in translation quality and language choice) higher upper bounds than English-only reasoning. Besides analyzing the reason behind the upper bound and challenges in reaching it, we also find that common answer selection methods cannot achieve this upper bound, due to their limitations and biases. These insights could pave the way for future research aimed at fully harnessing the potential of multilingual reasoning in LLMs.

cs.CL↗

arXiv2025

Floquet control of topological phases and Hall effects in Z2 nodal line semimetals

Dynamic control of topological properties in materials is central to modern condensed matter physics, and Floquet engineering, utilizing periodic light fields, provides a promising avenue. Here, we use Floquet theory to theoretically study the topological response of a Z2 nodal line semimetal (NLSM) when driven by circularly polarized light (CPL). We demonstrate that the direction of CPL irradiation critically dictates the resulting topological phase transitions. Specifically, when light is incident perpendicular to the nodal line plane, increasing the light amplitude induces two successive topological phase transitions: first, from the Z2 NLSM to a vortex NLSM, a rare and intriguing topological state; and second, a transition from the vortex NLSM to a semimetal with a pair of Weyl points (WPs). In stark contrast, irradiation along other directions directly transforms the Z2 nodal line into a pair of WPs. We further investigate the transport properties of the Floquet Z2 NLSM, focusing on the anomalous and planar Hall effects. The anomalous Hall effect exhibits a direction-dependent amplitude variation, deviating from conventional two-band NLSM behavior. Importantly, we reveal a significant and tunable planar Hall effect, a phenomenon largely unexplored in Floquet topological materials, which is highly sensitive to both light amplitude and direction. Our findings not only present a novel route to realize the vortex NLSM, but also establish an efficient way to control Hall transport phenomena in Z2 NLSMs.

cond-mat.str-el↗