arXiv subjects

Lei Chen

Publications and source records attributed to Lei Chen.

At least 235 records · Page 13Linked to original sources

arXiv2025

Emergent heavy-fermion physics in a new family of topological insulators RAsS (R = Y, La, and Sm)

Realizing topological phases in strongly correlated materials has become a major impetus in condensed matter physics. Although many compounds are now classified as topological insulators, $f$-electron systems (with their strong electron correlations) provide an especially fertile platform for emergent heavy-fermion phenomena driven by the interplay of topology and many-body effects. In this study, we examine the crystalline topology of a new RAsS series (R = Y, La, Sm), revealing a structural variant from previous reports. We demonstrate that YAsS and SmAsS host hourglass fermions protected by glide symmetry. SmAsS notably exhibits a strong effective-mass enhancement, placing it alongside SmB${}_6$ and YbB${}_{12}$ as a material that couples topological surface states with emergent Kondo physics, yet distinguished by its crystalline symmetry constraints and $f$-$p$ orbital hybridization. To capture these features, we construct a minimal model incorporating $f$-electron degrees of freedom, which reproduces the observed topological properties and predicts that the surface states survive in the correlated regime, albeit shifted in energy. Our work thus introduces a new family of correlated topological materials and forecasts the robustness of their surface states under Kondo correlations.

cond-mat.mtrl-sci↗

arXiv2025

InstaRevive: One-Step Image Enhancement via Dynamic Score Matching

Image enhancement finds wide-ranging applications in real-world scenarios due to complex environments and the inherent limitations of imaging devices. Recent diffusion-based methods yield promising outcomes but necessitate prolonged and computationally intensive iterative sampling. In response, we propose InstaRevive, a straightforward yet powerful image enhancement framework that employs score-based diffusion distillation to harness potent generative capability and minimize the sampling steps. To fully exploit the potential of the pre-trained diffusion model, we devise a practical and effective diffusion distillation pipeline using dynamic control to address inaccuracies in updating direction during score matching. Our control strategy enables a dynamic diffusing scope, facilitating precise learning of denoising trajectories within the diffusion model and ensuring accurate distribution matching gradients during training. Additionally, to enrich guidance for the generative power, we incorporate textual prompts via image captioning as auxiliary conditions, fostering further exploration of the diffusion model. Extensive experiments substantiate the efficacy of our framework across a diverse array of challenging tasks and datasets, unveiling the compelling efficacy and efficiency of InstaRevive in delivering high-quality and visually appealing results. Code is available at https://github.com/EternalEvan/InstaRevive.

cs.CV↗

arXiv2025

Apt-Serve: Adaptive Request Scheduling on Hybrid Cache for Scalable LLM Inference Serving

Large language model (LLM) inference serving systems are essential to various LLM-based applications. As demand for LLM services continues to grow, scaling these systems to handle high request rates while meeting latency Service-Level Objectives (SLOs), referred to as effective throughput, becomes critical. However, existing systems often struggle to improve effective throughput, primarily due to a significant decline in Time To First Token (TTFT) SLO attainment. We identify two major causes of this bottleneck: (1) memory-intensive KV cache that limits batch size expansion under GPU memory constraints, and (2) rigid batch composition enforced by the default First-Come-First-Serve scheduling policy. In this paper, we introduce Apt-Serve, a scalable framework designed to enhance effective throughput in LLM inference serving. Apt-Serve features a new hybrid cache scheme that combines KV cache with a memory-efficient hidden cache for reusable input hidden state vectors, allowing large batch sizes and improving request concurrency. Based on the hybrid cache, Apt-Serve employs an adaptive runtime scheduling mechanism that dynamically optimizes batch composition. We formally define the adaptive scheduling optimization problem and propose an efficient algorithm with theoretical guarantees. Extensive evaluations on three real-world datasets and LLMs ranging from 13B to 66B parameters demonstrate that Apt-Serve achieves up to 8.8x improvement in effective throughput compared to the state-of-the-art inference serving systems.

cs.LG↗

arXiv2025

Privacy in Fine-tuning Large Language Models: Attacks, Defenses, and Future Directions

Fine-tuning has emerged as a critical process in leveraging Large Language Models (LLMs) for specific downstream tasks, enabling these models to achieve state-of-the-art performance across various domains. However, the fine-tuning process often involves sensitive datasets, introducing privacy risks that exploit the unique characteristics of this stage. In this paper, we provide a comprehensive survey of privacy challenges associated with fine-tuning LLMs, highlighting vulnerabilities to various privacy attacks, including membership inference, data extraction, and backdoor attacks. We further review defense mechanisms designed to mitigate privacy risks in the fine-tuning phase, such as differential privacy, federated learning, and knowledge unlearning, discussing their effectiveness and limitations in addressing privacy risks and maintaining model utility. By identifying key gaps in existing research, we highlight challenges and propose directions to advance the development of privacy-preserving methods for fine-tuning LLMs, promoting their responsible use in diverse applications.

cs.AI↗

arXiv2025

Learning Towards Emergence: Paving the Way to Induce Emergence by Inhibiting Monosemantic Neurons on Pre-trained Models

Emergence, the phenomenon of a rapid performance increase once the model scale reaches a threshold, has achieved widespread attention recently. The literature has observed that monosemantic neurons in neural networks gradually diminish as the model scale increases. Subsequently, Learning From Emergence is proposed to actively inhibit monosemantic neurons in relatively small neural networks (e.g., BERT and Swin-Transformer) for promoting model performance with fine-tuning. However, to ultimately achieve emergence, it is demanding to support the monosemantic neuron inhibition in the pretraining phase of large-scale models. Thus, this work further pushes the boundary of this research direction to be Learning Towards Emergence (L2E) and enables the training and validating of the impact of inhibiting monosemantic neurons on larger pre-trained neural networks (e.g., Pythia-70M, 410M, and 2.8B). More specifically, to bridge the gap in current research, we first conduct experiments on models of various scales (up to 6.9B) to validate the monosemantic ideas. Then, we present a novel method L2E to address the inefficient monosemantic neuron retrieval and ineffective monosemantic neuron inhibition when existing methods are applied in the pretraining phase of large-scale models. It employs an adjustable thresholding technique for efficient neuron retrieval, incorporates a False Killing Rate metric to assess inhibition effects, and proposes a regularization-style inhibition approach, which addresses the limitations of previous approaches in both efficiency and effectiveness. Experimental results demonstrate the effectiveness of L2E's monosemantic neuron inhibition and its efficiency in implementation with large-scale models.

cs.ET↗

arXiv2025

FuXi-RTM: A Physics-Guided Prediction Framework with Radiative Transfer Modeling

Similar to conventional video generation, current deep learning-based weather prediction frameworks often lack explicit physical constraints, leading to unphysical outputs that limit their reliability for operational forecasting. Among various physical processes requiring proper representation, radiation plays a fundamental role as it drives Earth's weather and climate systems. However, accurate simulation of radiative transfer processes remains challenging for traditional numerical weather prediction (NWP) models due to their inherent complexity and high computational costs. Here, we propose FuXi-RTM, a hybrid physics-guided deep learning framework designed to enhance weather forecast accuracy while enforcing physical consistency. FuXi-RTM integrates a primary forecasting model (FuXi) with a fixed deep learning-based radiative transfer model (DLRTM) surrogate that efficiently replaces conventional radiation parameterization schemes. This represents the first deep learning-based weather forecasting framework to explicitly incorporate physical process modeling. Evaluated over a comprehensive 5-year dataset, FuXi-RTM outperforms its unconstrained counterpart in 88.51% of 3320 variable and lead time combinations, with improvements in radiative flux predictions. By incorporating additional physical processes, FuXi-RTM paves the way for next-generation weather forecasting systems that are both accurate and physically consistent.

physics.ao-ph↗

arXiv2025

Improving the End-to-End Efficiency of Offline Inference for Multi-LLM Applications Based on Sampling and Simulation

As large language models (LLMs) have shown great success in many tasks, they are used in various applications. While a lot of works have focused on the efficiency of single-LLM application (e.g., offloading, request scheduling, parallelism strategy selection), multi-LLM applications receive less attention, particularly in offline inference scenarios. In this work, we aim to improve the offline end-to-end inference efficiency of multi-LLM applications in the single-node multi-GPU environment. The problem involves two key decisions: (1) determining which LLMs to run concurrently each time (we may not run all the models at the same time), and (2) selecting a parallelism strategy to use for each LLM. This problem is NP-hard. Naive solutions may not work well because the running time for a model to complete a set of requests depends on the request workload and the selected parallelism strategy, and they lack an accurate model of the running time. As the LLM output lengths are unknown before running, to estimate the model running time, we propose a sampling-then-simulation method which first estimates the output lengths by sampling from an empirical cumulative function we obtained from a large dataset in advance, and then simulates the LLM inference process accordingly. Based on the simulation, we estimate the per-iteration latencys to get the total latency. A greedy method is proposed to optimize the scheduling of the LLMs in the application across the GPUs. We then propose a framework SamuLLM which contains two phases: planning, which calls the greedy method for an application and running, which runs the application and dynamically adjust the model scheduling based on the runtime information. Experiments on 3 applications and a mixed application show that SamuLLM can achieve 1.0-2.4$\times$ end-to-end speedups compared to the competitors.

cs.DC↗

arXiv2025

PLM: Efficient Peripheral Language Models Hardware-Co-Designed for Ubiquitous Computing

While scaling laws have been continuously validated in large language models (LLMs) with increasing model parameters, the inherent tension between the inference demands of LLMs and the limited resources of edge devices poses a critical challenge to the development of edge intelligence. Recently, numerous small language models have emerged, aiming to distill the capabilities of LLMs into smaller footprints. However, these models often retain the fundamental architectural principles of their larger counterparts, still imposing considerable strain on the storage and bandwidth capacities of edge devices. In this paper, we introduce the PLM, a Peripheral Language Model, developed through a co-design process that jointly optimizes model architecture and edge system constraints. The PLM utilizes a Multi-head Latent Attention mechanism and employs the squared ReLU activation function to encourage sparsity, thereby reducing peak memory footprint during inference. During training, we collect and reorganize open-source datasets, implement a multi-phase training strategy, and empirically investigate the Warmup-Stable-Decay-Constant (WSDC) learning rate scheduler. Additionally, we incorporate Reinforcement Learning from Human Feedback (RLHF) by adopting the ARIES preference learning approach. Following a two-phase SFT process, this method yields performance gains of 2% in general tasks, 9% in the GSM8K task, and 11% in coding tasks. In addition to its novel architecture, evaluation results demonstrate that PLM outperforms existing small language models trained on publicly available data while maintaining the lowest number of activated parameters. Furthermore, deployment across various edge devices, including consumer-grade GPUs, mobile phones, and Raspberry Pis, validates PLM's suitability for peripheral applications. The PLM series models are publicly available at https://github.com/plm-team/PLM.

cs.CL↗

arXiv2025

A Tale of Single-channel Electroencephalogram: Devices, Datasets, Signal Processing, Applications, and Future Directions

Single-channel electroencephalogram (EEG) is a cost-effective, comfortable, and non-invasive method for monitoring brain activity, widely adopted by researchers, consumers, and clinicians. The increasing number and proportion of articles on single-channel EEG underscore its growing potential. This paper provides a comprehensive review of single-channel EEG, focusing on development trends, devices, datasets, signal processing methods, recent applications, and future directions. Definitions of bipolar and unipolar configurations in single-channel EEG are clarified to guide future advancements. Applications mainly span sleep staging, emotion recognition, educational research, and clinical diagnosis. Ongoing advancements of single-channel EEG in AI-based EEG generation techniques suggest potential parity or superiority over multichannel EEG performance.

cs.CV↗

arXiv2025

E-SAM: Training-Free Segment Every Entity Model

Entity Segmentation (ES) aims at identifying and segmenting distinct entities within an image without the need for predefined class labels. This characteristic makes ES well-suited to open-world applications with adaptation to diverse and dynamically changing environments, where new and previously unseen entities may appear frequently. Existing ES methods either require large annotated datasets or high training costs, limiting their scalability and adaptability. Recently, the Segment Anything Model (SAM), especially in its Automatic Mask Generation (AMG) mode, has shown potential for holistic image segmentation. However, it struggles with over-segmentation and under-segmentation, making it less effective for ES. In this paper, we introduce E-SAM, a novel training-free framework that exhibits exceptional ES capability. Specifically, we first propose Multi-level Mask Generation (MMG) that hierarchically processes SAM's AMG outputs to generate reliable object-level masks while preserving fine details at other levels. Entity-level Mask Refinement (EMR) then refines these object-level masks into accurate entity-level masks. That is, it separates overlapping masks to address the redundancy issues inherent in SAM's outputs and merges similar masks by evaluating entity-level consistency. Lastly, Under-Segmentation Refinement (USR) addresses under-segmentation by generating additional high-confidence masks fused with EMR outputs to produce the final ES map. These three modules are seamlessly optimized to achieve the best ES without additional training overhead. Extensive experiments demonstrate that E-SAM achieves state-of-the-art performance compared to prior ES methods, demonstrating a significant improvement by +30.1 on benchmark metrics.

cs.CV↗

arXiv2025

Resolving the Kagome Origin of the Strange Metallicity in Ni$_3$In

Strong correlations promote singular properties such as strange metallicity, which shows considerable commonality across quantum materials platforms. Understanding the mechanism for such emerging universality is an outstanding challenge, given that the underlying degrees of freedom can be complex and varied. Progress may be made in flat band systems, especially kagome and other frustrated-lattice metals with active flat bands. These systems show strange metal behavior that bears a striking resemblance to what happens in heavy-fermion metals. Here, in scanning tunneling spectroscopy of kagome metal Ni$_3$In, we find a zero-bias peak-dip structure whose variation with magnetic field and temperature tracks the evolution of the strange metal properties. We identify the origin of the peak as compact molecular orbitals formed by destructive interference over the kagome sites, resulting in emergent $f$-shell-like localized moments. Using quasi-particle interference, we visualize their interaction with the Dirac light bands. We thus unveil the essential microscopic ingredients of the $d$-electron-based kagome metals that, while distinct from the atomic orbitals of the $f$-electron-based heavy fermion materials, are responsible for a shared phenomenology between the two types of systems. Our findings provide a new window to uncover and interconnect the essential and yet diverse microscopic building blocks in disparate families of quantum materials that drive a convergence towards a universal understanding in the regime of amplified quantum fluctuations.

cond-mat.str-el↗

arXiv2025

Correlated flat-band physics in a bilayer kagome metal based on compact molecular orbitals

Flat bands, when located close to the Fermi energy, can considerably enhance the influence of electron correlations on the low energy physics in kagome and other frustrated-lattice metals. A major challenge in describing the interaction effects in such bulk materials is that the flat band is often intermixed with a large number of other bands. Here we show that the recently introduced notion of compact molecular orbitals (CMOs) enable a path forward in describing the dominant effect of the Coulomb interactions in spite of the complexity of the bandstructure. Our materials-based analysis allows for the understanding of the scanning-tunneling-microscopy experiment [J. C. Souza et al., preprint (2024)] of the bilayer kagome metal Ni$_3$In in terms of the CMO notion. From the resulting CMO, an effective Anderson lattice model can be set up. This CMO-based approach enables the calculation of correlation effects that is difficult to do based on the atomic orbitals. Furthermore, it suggests an enriched phase diagram for the strange metal physics of the kagome metal, which can be tested by future experiments. We discuss the implications of our results for the general correlation physics of flat band systems and beyond.

cond-mat.str-el↗

arXiv2025

Quantum critical metals and loss of quasiparticles

Strange metals develop near quantum critical points in a variety of strongly correlated systems. Some of the issues that are central to the field include how the quantum-critical state loses quasiparticles, how it drives superconductivity, and to what extent the strange-metal physics in different classes of correlated systems are interconnected. In this Review, we survey some of these issues from the vantage point of heavy fermion metals. We will describe the notion of Kondo destruction and how it leads to several crucial effects. These include a transformation of the Fermi surface from large to small when the system is tuned across the quantum-critical point, a loss of quasiparticles everywhere on the Fermi surface when it is perched at the quantum-critical point, and a dynamical Planckian scaling in various physical properties including charge responses. We close with a discussion about the connections between the strange-metal physics in heavy fermion metals and its counterparts in the cuprates and other correlated materials.

cond-mat.str-el↗

arXiv2025

Metallic quantum criticality enabled by flat bands in a kagome lattice

Strange metals arise in a variety of platforms for strongly correlated electrons, ranging from the cuprates, heavy fermions to flat band systems. Motivated by recent experiments in kagome metals, we study a Hubbard model on a kagome lattice whose noninteracting limit contains flat bands. A Kondo lattice description is constructed, in which the correlation effects are captured by symmetry preserving and exponentially localized molecular orbitals. These compact molecular orbitals represent the local degrees of freedom that emerge from topological flat bands. We identify a quantum critical point at which quasiparticles are lost and strange metallicity emerges. Our theoretical work opens up a new route for realizing beyond-Landau quantum criticality, as well as the associated strange metallicity and emergent quantum phases.

cond-mat.str-el↗

arXiv2025

KAA: Kolmogorov-Arnold Attention for Enhancing Attentive Graph Neural Networks

Graph neural networks (GNNs) with attention mechanisms, often referred to as attentive GNNs, have emerged as a prominent paradigm in advanced GNN models in recent years. However, our understanding of the critical process of scoring neighbor nodes remains limited, leading to the underperformance of many existing attentive GNNs. In this paper, we unify the scoring functions of current attentive GNNs and propose Kolmogorov-Arnold Attention (KAA), which integrates the Kolmogorov-Arnold Network (KAN) architecture into the scoring process. KAA enhances the performance of scoring functions across the board and can be applied to nearly all existing attentive GNNs. To compare the expressive power of KAA with other scoring functions, we introduce Maximum Ranking Distance (MRD) to quantitatively estimate their upper bounds in ranking errors for node importance. Our analysis reveals that, under limited parameters and constraints on width and depth, both linear transformation-based and MLP-based scoring functions exhibit finite expressive power. In contrast, our proposed KAA, even with a single-layer KAN parameterized by zero-order B-spline functions, demonstrates nearly infinite expressive power. Extensive experiments on both node-level and graph-level tasks using various backbone models show that KAA-enhanced scoring functions consistently outperform their original counterparts, achieving performance improvements of over 20% in some cases.

cs.LG↗

arXiv2025

OpenRAG: Optimizing RAG End-to-End via In-Context Retrieval Learning

In this paper, we analyze and empirically show that the learned relevance for conventional information retrieval (IR) scenarios may be inconsistent in retrieval-augmented generation (RAG) scenarios. To bridge this gap, we introduce OpenRAG, a RAG framework that is optimized end-to-end by tuning the retriever to capture in-context relevance, enabling adaptation to the diverse and evolving needs. Extensive experiments across a wide range of tasks demonstrate that OpenRAG, by tuning a retriever end-to-end, leads to a consistent improvement of 4.0% over the original retriever, consistently outperforming existing state-of-the-art retrievers by 2.1%. Additionally, our results indicate that for some tasks, an end-to-end tuned 0.2B retriever can achieve improvements that surpass those of RAG-oriented or instruction-tuned 8B large language models (LLMs), highlighting the cost-effectiveness of our approach in enhancing RAG systems.

cs.CL↗

arXiv2025

Amplified multipartite entanglement witnessed in a quantum critical metal

Strong correlations in matter promote a landscape of quantum phases and associated quantum critical points. For metallic systems, there is increasing recognition that the quantum criticality goes beyond the Landau framework and, thus, novel means are needed to characterize the quantum critical fluid. Here we do so by studying an entanglement quantity, the quantum Fisher information, in a strange metal system, focusing on the exemplary case of an Anderson/Kondo lattice model near its Kondo destruction quantum critical point. The spin quantum Fisher information peaks at the quantum critical point and indicates a strongly entangled ground state. Our results are supported by the quantum Fisher information extracted from inelastic neutron scattering measurements in heavy fermion metals. Our work elucidates the loss of quasiparticles in strange metals, opens a quantum information avenue to advance the understanding of metallic quantum criticality in a broad range of strongly correlated systems, and points to a novel regime of quantum matter to realize amplified entanglement.

cond-mat.str-el↗

arXiv2025

E2ED^2:Direct Mapping from Noise to Data for Enhanced Diffusion Models

Diffusion models have established themselves as the de facto primary paradigm in visual generative modeling, revolutionizing the field through remarkable success across various diverse applications ranging from high-quality image synthesis to temporal aware video generation. Despite these advancements, three fundamental limitations persist, including 1) discrepancy between training and inference processes, 2) progressive information leakage throughout the noise corruption procedures, and 3) inherent constraints preventing effective integration of modern optimization criteria like perceptual and adversarial loss. To mitigate these critical challenges, we in this paper present a novel end-to-end learning paradigm that establishes direct optimization from the final generated samples to initial noises. Our proposed End-to-End Differentiable Diffusion, dubbed E2ED^2, introduces several key improvements: it eliminates the sequential training-sampling mismatch and intermediate information leakage via conceptualizing training as a direct transformation from isotropic Gaussian noise to the target data distribution. Additionally, such training framework enables seamless incorporation of adversarial and perceptual losses into the core optimization objective. Comprehensive evaluation across standard benchmarks including COCO30K and HW30K reveals that our method achieves substantial performance gains in terms of Fréchet Inception Distance (FID) and CLIP score, even with fewer sampling steps (less than 4). Our findings highlight that the end-to-end mechanism might pave the way for more robust and efficient solutions, \emph{i.e.,} combining diffusion stability with GAN-like discriminative optimization in an end-to-end manner.

cs.CV↗