arXiv subjects

Lu Lu

Publications and source records attributed to Lu Lu.

At least 91 records · Page 5Linked to original sources

arXiv2025

SCM: Enhancing Large Language Model with Self-Controlled Memory Framework

Large Language Models (LLMs) are constrained by their inability to process lengthy inputs, resulting in the loss of critical historical information. To address this limitation, in this paper, we propose the Self-Controlled Memory (SCM) framework to enhance the ability of LLMs to maintain long-term memory and recall relevant information. Our SCM framework comprises three key components: an LLM-based agent serving as the backbone of the framework, a memory stream storing agent memories, and a memory controller updating memories and determining when and how to utilize memories from memory stream. Additionally, the proposed SCM is able to process ultra-long texts without any modification or fine-tuning, which can integrate with any instruction following LLMs in a plug-and-play paradigm. Furthermore, we annotate a dataset to evaluate the effectiveness of SCM for handling lengthy inputs. The annotated dataset covers three tasks: long-term dialogues, book summarization, and meeting summarization. Experimental results demonstrate that our method achieves better retrieval recall and generates more informative responses compared to competitive baselines in long-term dialogues. (https://github.com/wbbeyourself/SCM4LLMs)

cs.CL↗

arXiv2025

TCM-3CEval: A Triaxial Benchmark for Assessing Responses from Large Language Models in Traditional Chinese Medicine

Large language models (LLMs) excel in various NLP tasks and modern medicine, but their evaluation in traditional Chinese medicine (TCM) is underexplored. To address this, we introduce TCM3CEval, a benchmark assessing LLMs in TCM across three dimensions: core knowledge mastery, classical text understanding, and clinical decision-making. We evaluate diverse models, including international (e.g., GPT-4o), Chinese (e.g., InternLM), and medical-specific (e.g., PLUSE). Results show a performance hierarchy: all models have limitations in specialized subdomains like Meridian & Acupoint theory and Various TCM Schools, revealing gaps between current capabilities and clinical needs. Models with Chinese linguistic and cultural priors perform better in classical text interpretation and clinical reasoning. TCM-3CEval sets a standard for AI evaluation in TCM, offering insights for optimizing LLMs in culturally grounded medical domains. The benchmark is available on Medbench's TCM track, aiming to assess LLMs' TCM capabilities in basic knowledge, classic texts, and clinical decision-making through multidimensional questions and real cases.

cs.CL↗

arXiv2025

Benchmarking Chinese Medical LLMs: A Medbench-based Analysis of Performance Gaps and Hierarchical Optimization Strategies

The evaluation and improvement of medical large language models (LLMs) are critical for their real-world deployment, particularly in ensuring accuracy, safety, and ethical alignment. Existing frameworks inadequately dissect domain-specific error patterns or address cross-modal challenges. This study introduces a granular error taxonomy through systematic analysis of top 10 models on MedBench, categorizing incorrect responses into eight types: Omissions, Hallucination, Format Mismatch, Causal Reasoning Deficiency, Contextual Inconsistency, Unanswered, Output Error, and Deficiency in Medical Language Generation. Evaluation of 10 leading models reveals vulnerabilities: despite achieving 0.86 accuracy in medical knowledge recall, critical reasoning tasks show 96.3% omission, while safety ethics evaluations expose alarming inconsistency (robustness score: 0.79) under option shuffled. Our analysis uncovers systemic weaknesses in knowledge boundary enforcement and multi-step reasoning. To address these, we propose a tiered optimization strategy spanning four levels, from prompt engineering and knowledge-augmented retrieval to hybrid neuro-symbolic architectures and causal reasoning frameworks. This work establishes an actionable roadmap for developing clinically robust LLMs while redefining evaluation paradigms through error-driven insights, ultimately advancing the safety and trustworthiness of AI in high-stakes medical environments.

cs.CL↗

arXiv2025

Efficient neural topology optimization via active learning for enhancing turbulent mass transfer in fluid channels

The design of fluid channel structures of reactors or separators of chemical processes is key to enhancing the mass transfer processes inside the devices. However, the systematic design of channel topological structures is difficult for complex turbulent flows. Here, we address this challenge by developing a machine learning framework to efficiently perform topology optimization of channel structures for turbulent mass transfer. We represent a topological structure using a neural network (referred to as `neural topology', which is optimized by employing pre-trained neural operators combined with a fine-tuning strategy with active data augmentation. The optimization is performed with two objectives: maximization of mass transfer efficiency and minimization of energy consumption, for the possible considerations of compromise between the two in real-world designs. The developed neural operator with active learning is data efficient in network training and demonstrates superior computational efficiency compared with traditional methods in obtaining optimal structures across a large design space. The optimization results are validated through experiments, proving that the optimized channel improves concentration uniformity by 37% compared with the original channel. We also demonstrate the variation of the optimal structures with changes in inlet velocity conditions, providing a reference for designing turbulent mass-transfer devices under different operating conditions.

physics.flu-dyn↗

arXiv2025

Fourier analysis on distance-regular Cayley graphs over abelian groups

The problem of constructing or characterizing strongly regular Cayley graphs (or equivalently, regular partial difference sets) has garnered significant attention over the past half-century. In 2003, Miklavič and Potočnik [European J. Combin. 24 (2003) 777--784] expanded upon this field by achieving a complete characterization of distance-regular Cayley graphs over cyclic groups through the method of Schur rings. Building on this work, Miklavič and Potočnik [J. Combin. Theory Ser. B 97 (2007) 14--33] formally proposed the problem of characterizing distance-regular Cayley graphs for arbitrary classes of groups. Within this framework, abelian groups hold particular significance, as numerous distance-regular graphs with classical parameters are precisely Cayley graphs over abelian groups. In this paper, we employ Fourier analysis on abelian groups to establish connections between distance-regular Cayley graphs over abelian groups and combinatorial objects in finite geometry. By combining these insights with classical results from finite geometry, we classify all distance-regular Cayley graphs over the group $\mathbb{Z}_n \oplus \mathbb{Z}_p$, where $p$ is an odd prime.

math.CO↗

arXiv2025

Emergence of a neutrino flux above 5 PeV and implications for ultrahigh energy cosmic rays

The rare detections of astrophysical neutrinos with energies above 5~PeV by two neutrino telescopes underscore the existence of a flux at these energies. In addition to over a decade of data taken by the IceCube Neutrino Observatory, the KM3NeT neutrino telescope has recently highlighted their discovery of a possible $\mathcal{O}(100~PeV)$ neutrino candidate. A connection between the highest-energy astrophysical neutrinos and the highest-energy cosmic rays is expected, and well-established theoretically. Here, for the first time, we simultaneously fit the neutrino data from IceCube and KM3NeT, as well as the ultrahigh-energy cosmic ray spectrum and composition data from the Pierre Auger Observatory (Auger), to test a common-origin hypothesis. We show that a phenomenological model is able to describe the combined data across these three observatories, and, depending on the true energy of the event detected by KM3NeT, suggests an additional cosmic ray source population not yet robustly detected by Auger. Although a measurement of the neutrino flux in this energy regime is at the sensitivity limit of cubic-kilometer-scale neutrino telescopes, next-generation observatories, such IceCube-Gen2, will have the sensitivity to make a significant detection of this flux.

astro-ph.HE↗

arXiv2025

ADEPT: A Noninvasive Method for Determining Elastic Parameters of Valve Tissue

Computer simulation of "virtual interventions" may inform optimal valve repair for a given patient prior to intervention. However, the paucity of noninvasive methods to determine in vivo mechanical parameters of valves limits the accuracy of computer prediction and their clinical application. To address this, we propose ADEPT: A noninvasive method for Determining Elastic Parameters of valve Tissue. In this work, we demonstrated its application to the tricuspid valve of a child. We first tracked valve displacements from open to closed frames within a 3D echocardiogram time sequence using image registration. Physics-informed neural networks were subsequently applied to estimate the nonlinear mechanical properties from first principles and reference displacements. The simulated model using these patient-specific parameters closely aligned with the reference image segmentation, achieving a mean symmetric distance of less than 1 mm. Our approach doubled the accuracy of the simulated model compared to the generic parameters reported in the literature.

q-bio.QM↗

arXiv2025

Multiple equilibrium states of a curved-sided hexagram: Part I-Stability of states

The stability of the multiple equilibrium states of a hexagram ring with six curved sides is investigated. Each of the six segments is a rod having the same length and uniform natural curvature. These rods are bent uniformly in the plane of the hexagram into equal arcs of 120deg or 240deg and joined at a cusp where their ends meet to form a 1-loop planar ring. The 1-loop rings formed from 120deg or 240deg arcs are inversions of one another and they, in turn, can be folded into a 3-loop straight line configuration or a 3-loop ring with each loop in an "8" shape. Each of these four equilibrium states has a uniform bending moment. Two additional intriguing planar shapes, 6-circle hexagrams, with equilibrium states that are also uniform bending, are identified and analyzed for stability. Stability is lost when the natural curvature falls outside the upper and lower limits in the form of a bifurcation mode involving coupled out-of-plane deflection and torsion of the rod segments. Conditions for stability, or lack thereof, depend on the geometry of the rod cross-section as well as its natural curvature. Rods with circular and rectangular cross-sections will be analyzed using a specialized form of Kirchhoff rod theory, and properties will be detailed such that all four of the states of interest are mutually stable. Experimental demonstrations of the various states and their stability are presented. Part II presents numerical simulations of transitions between states using both rod theory and a three-dimensional finite element formulation, includes confirmation of the stability limits established in Part I, and presents additional experimental demonstrations and verifications.

physics.app-ph↗

arXiv2025

Multiple equilibrium states of a curved-sided hexagram: Part II-Transitions between states

Curved-sided hexagrams with multiple equilibrium states have great potential in engineering applications such as foldable architectures, deployable aerospace structures, and shape-morphing soft robots. In Part I, the classical stability criterion based on energy variation was used to study the elastic stability of the curved-sided hexagram and identify the natural curvature range for stability of each state for circular and rectangular rod cross-sections. Here, we combine a multi-segment Kirchhoff rod model, finite element simulations, and experiments to investigate the transitions between four basic equilibrium states of the curved-sided hexagram. The four equilibrium states, namely the star hexagram, the daisy hexagram, the 3-loop line, and the 3-loop "8", carry uniform bending moments in their initial states, and the magnitudes of these moments depend on the natural curvatures and their initial curvatures. Transitions between these equilibrium states are triggered by applying bending loads at their corners or edges. It is found that transitions between the stable equilibrium states of the curved-sided hexagram are influenced by both the natural curvature and the loading position. Within a specific natural curvature range, the star hexagram, the daisy hexagram, and the 3-loop "8" can transform among one another by bending at different positions. Based on these findings, we identify the natural curvature range and loading conditions to achieve transition among these three equilibrium states plus a folded 3-loop line state for one specific ring having a rectangular cross-section. The results obtained in this part also validate the elastic stability range of the four equilibrium states of the curved-sided hexagram in Part I. We envision that the present work could provide a new perspective for the design of multi-functional deployable and foldable structures.

physics.app-ph↗

arXiv2025

Selective Actuation Enabled Multifunctional Magneto-mechanical Metamaterial for Programming Elastic Wave Propagation

Active metamaterials are a type of metamaterial with tunable properties enabled by structural reconfigurations. Existing active metamaterials often achieve only a limited number of structural reconfigurations upon the application of an external load across the entire structure. Here, we propose a selective actuation strategy for inhomogeneous deformations of magneto-mechanical metamaterials, which allows for the integration of multiple elastic wave tuning functionalities into a single metamaterial design. Central to this actuation strategy is that a magnetic field is applied to specific unit cells instead of the entire metamaterial, and the unit cell can transform between two geometrically distinct shapes, which exhibit very different mechanical responses to elastic wave excitations. Our numerical simulations and experiments demonstrate that the tunable response of the unit cell, coupled with inhomogeneous deformation achieved through selective actuation, unlocks multifunctional capabilities of magneto-mechanical metamaterials such as tunable elastic wave transmittance, elastic waveguide, and vibration isolation. The proposed selective actuation strategy offers a simple but effective way to control the tunable properties and thus enhances the programmability of magneto-mechanical metamaterials, which also expands the application space of magneto-mechanical metamaterials in elastic wave manipulation.

physics.app-ph↗

arXiv2025

SD-Eval: A Benchmark Dataset for Spoken Dialogue Understanding Beyond Words

Speech encompasses a wealth of information, including but not limited to content, paralinguistic, and environmental information. This comprehensive nature of speech significantly impacts communication and is crucial for human-computer interaction. Chat-Oriented Large Language Models (LLMs), known for their general-purpose assistance capabilities, have evolved to handle multi-modal inputs, including speech. Although these models can be adept at recognizing and analyzing speech, they often fall short of generating appropriate responses. We argue that this is due to the lack of principles on task definition and model development, which requires open-source datasets and metrics suitable for model evaluation. To bridge the gap, we present SD-Eval, a benchmark dataset aimed at multidimensional evaluation of spoken dialogue understanding and generation. SD-Eval focuses on paralinguistic and environmental information and includes 7,303 utterances, amounting to 8.76 hours of speech data. The data is aggregated from eight public datasets, representing four perspectives: emotion, accent, age, and background sound. To assess the SD-Eval benchmark dataset, we implement three different models and construct a training set following a process similar to that of SD-Eval. The training set contains 1,052.72 hours of speech data and 724.4k utterances. We also conduct a comprehensive evaluation using objective evaluation methods (e.g. BLEU and ROUGE), subjective evaluations and LLM-based metrics for the generated responses. Models conditioned with paralinguistic and environmental information outperform their counterparts in both objective and subjective measures. Moreover, experiments demonstrate that LLM-based metrics show a higher correlation with human evaluation compared to traditional metrics. We open-source SD-Eval at https://github.com/amphionspace/SD-Eval.

cs.CL↗

arXiv2024

ST$^3$: Accelerating Multimodal Large Language Model by Spatial-Temporal Visual Token Trimming

Multimodal large language models (MLLMs) enhance their perceptual capabilities by integrating visual and textual information. However, processing the massive number of visual tokens incurs a significant computational cost. Existing analysis of the MLLM attention mechanisms remains shallow, leading to coarse-grain token pruning strategies that fail to effectively balance speed and accuracy. In this paper, we conduct a comprehensive investigation of MLLM attention mechanisms with LLaVA. We find that numerous visual tokens and partial attention computations are redundant during the decoding process. Based on this insight, we propose Spatial-Temporal Visual Token Trimming ($\textbf{ST}^{3}$), a framework designed to accelerate MLLM inference without retraining. $\textbf{ST}^{3}$ consists of two primary components: 1) Progressive Visual Token Pruning (\textbf{PVTP}), which eliminates inattentive visual tokens across layers, and 2) Visual Token Annealing (\textbf{VTA}), which dynamically reduces the number of visual tokens in each layer as the generated tokens grow. Together, these techniques deliver around $\mathbf{2\times}$ faster inference with only about $\mathbf{30\%}$ KV cache memory compared to the original LLaVA, while maintaining consistent performance across various datasets. Crucially, $\textbf{ST}^{3}$ can be seamlessly integrated into existing pre-trained MLLMs, providing a plug-and-play solution for efficient inference.

cs.CV↗

arXiv2024

SALMONN-omni: A Codec-free LLM for Full-duplex Speech Understanding and Generation

Full-duplex multimodal large language models (LLMs) provide a unified framework for addressing diverse speech understanding and generation tasks, enabling more natural and seamless human-machine conversations. Unlike traditional modularised conversational AI systems, which separate speech recognition, understanding, and text-to-speech generation into distinct components, multimodal LLMs operate as single end-to-end models. This streamlined design eliminates error propagation across components and fully leverages the rich non-verbal information embedded in input speech signals. We introduce SALMONN-omni, a codec-free, full-duplex speech understanding and generation model capable of simultaneously listening to its own generated speech and background sounds while speaking. To support this capability, we propose a novel duplex spoken dialogue framework incorporating a ``thinking'' mechanism that facilitates asynchronous text and speech generation relying on embeddings instead of codecs (quantized speech and audio tokens). Experimental results demonstrate SALMONN-omni's versatility across a broad range of streaming speech tasks, including speech recognition, speech enhancement, and spoken question answering. Additionally, SALMONN-omni excels at managing turn-taking, barge-in, and echo cancellation scenarios, establishing its potential as a robust prototype for full-duplex conversational AI systems. To the best of our knowledge, SALMONN-omni is the first codec-free model of its kind. A full technical report along with model checkpoints will be released soon.

eess.AS↗

arXiv2024

DeepONet as a Multi-Operator Extrapolation Model: Distributed Pretraining with Physics-Informed Fine-Tuning

We propose a novel fine-tuning method to achieve multi-operator learning through training a distributed neural operator with diverse function data and then zero-shot fine-tuning the neural network using physics-informed losses for downstream tasks. Operator learning effectively approximates solution operators for PDEs and various PDE-related problems, yet it often struggles to generalize to new tasks. To address this, we investigate fine-tuning a pretrained model, while carefully selecting an initialization that enables rapid adaptation to new tasks with minimal data. Our approach combines distributed learning to integrate data from various operators in pre-training, while physics-informed methods enable zero-shot fine-tuning, minimizing the reliance on downstream data. We investigate standard fine-tuning and Low-Rank Adaptation fine-tuning, applying both to train complex nonlinear target operators that are difficult to learn only using random initialization. Through comprehensive numerical examples, we demonstrate the advantages of our approach, showcasing significant improvements in accuracy. Our findings provide a robust framework for advancing multi-operator learning and highlight the potential of transfer learning techniques in this domain.

cs.LG↗

arXiv2024

Magnetic Milli-spinner for Robotic Endovascular Surgery

Vascular diseases such as thrombosis, atherosclerosis, and aneurysm, which can lead to blockage of blood flow or blood vessel rupture, are common and life-threatening. Conventional minimally invasive treatments utilize catheters, or long tubes, to guide small devices or therapeutic agents to targeted regions for intervention. Unfortunately, catheters suffer from difficult and unreliable navigation in narrow, winding vessels such as those found in the brain. Magnetically actuated untethered robots, which have been extensively explored as an alternative, are promising for navigation in complex vasculatures and vascular disease treatments. Most current robots, however, cannot swim against high flows or are inadequate in treating certain conditions. Here, we introduce a multifunctional and magnetically actuated milli-spinner robot for rapid navigation and performance of various treatments in complicated vasculatures. The milli-spinner, with a unique hollow structure including helical fins and slits for propulsion, generates a distinct flow field upon spinning. The milli-spinner is the fastest-ever untethered magnetic robot for movement in tubular environments, easily achieving speeds of 23 cm/s, demonstrating promise as an untethered medical device for effective navigation in blood vessels and robotic treatment of numerous vascular diseases.

cs.RO↗

arXiv2024

Federated scientific machine learning for approximating functions and solving differential equations with data heterogeneity

By leveraging neural networks, the emerging field of scientific machine learning (SciML) offers novel approaches to address complex problems governed by partial differential equations (PDEs). In practical applications, challenges arise due to the distributed essence of data, concerns about data privacy, or the impracticality of transferring large volumes of data. Federated learning (FL), a decentralized framework that enables the collaborative training of a global model while preserving data privacy, offers a solution to the challenges posed by isolated data pools and sensitive data issues. Here, this paper explores the integration of FL and SciML to approximate complex functions and solve differential equations. We propose two novel models: federated physics-informed neural networks (FedPINN) and federated deep operator networks (FedDeepONet). We further introduce various data generation methods to control the degree of non-independent and identically distributed (non-iid) data and utilize the 1-Wasserstein distance to quantify data heterogeneity in function approximation and PDE learning. We systematically investigate the relationship between data heterogeneity and federated model performance. Additionally, we propose a measure of weight divergence and develop a theoretical framework to establish growth bounds for weight divergence in federated learning compared to traditional centralized learning. To demonstrate the effectiveness of our methods, we conducted 10 experiments, including 2 on function approximation, 5 PDE problems on FedPINN, and 3 PDE problems on FedDeepONet. These experiments demonstrate that proposed federated methods surpass the models trained only using local data and achieve competitive accuracy of centralized models trained using all data.

cs.LG↗

arXiv2024

Rapid aerodynamic prediction of swept wings via physics-embedded transfer learning

Machine learning-based models provide a promising way to rapidly acquire transonic swept wing flow fields but suffer from large computational costs in establishing training datasets. Here, we propose a physics-embedded transfer learning framework to efficiently train the model by leveraging the idea that a three-dimensional flow field around wings can be analyzed with two-dimensional flow fields around cross-sectional airfoils. An airfoil aerodynamics prediction model is pretrained with airfoil samples. Then, an airfoil-to-wing transfer model is fine-tuned with a few wing samples to predict three-dimensional flow fields based on two-dimensional results on each spanwise cross section. Sweep theory is embedded when determining the corresponding airfoil geometry and operating conditions, and to obtain the sectional airfoil lift coefficient, which is one of the operating conditions, the low-fidelity vortex lattice method and data-driven methods are proposed and evaluated. Compared to a nontransfer model, introducing the pretrained model reduces the error by 30%, while introducing sweep theory further reduces the error by 9%. When reducing the dataset size, less than half of the wing training samples are need to reach the same error level as the nontransfer framework, which makes establishing the model much easier.

physics.flu-dyn↗

arXiv2024

Efficient and generalizable nested Fourier-DeepONet for three-dimensional geological carbon sequestration

Geological carbon sequestration (GCS) involves injecting CO$_2$ into subsurface geological formations for permanent storage. Numerical simulations could guide decisions in GCS projects by predicting CO$_2$ migration pathways and the pressure distribution in storage formation. However, these simulations are often computationally expensive due to highly coupled physics and large spatial-temporal simulation domains. Surrogate modeling with data-driven machine learning has become a promising alternative to accelerate physics-based simulations. Among these, the Fourier neural operator (FNO) has been applied to three-dimensional synthetic subsurface models. Here, to further improve performance, we have developed a nested Fourier-DeepONet by combining the expressiveness of the FNO with the modularity of a deep operator network (DeepONet). This new framework is twice as efficient as a nested FNO for training and has at least 80% lower GPU memory requirement due to its flexibility to treat temporal coordinates separately. These performance improvements are achieved without compromising prediction accuracy. In addition, the generalization and extrapolation ability of nested Fourier-DeepONet beyond the training range has been thoroughly evaluated. Nested Fourier-DeepONet outperformed the nested FNO for extrapolation in time with more than 50% reduced error. It also exhibited good extrapolation accuracy beyond the training range in terms of reservoir properties, number of wells, and injection rate.

cs.LG↗