arXiv subjects

Lei Li

Publications and source records attributed to Lei Li.

At least 271 records · Page 15Linked to original sources

arXiv2025

Human Motion Instruction Tuning

This paper presents LLaMo (Large Language and Human Motion Assistant), a multimodal framework for human motion instruction tuning. In contrast to conventional instruction-tuning approaches that convert non-linguistic inputs, such as video or motion sequences, into language tokens, LLaMo retains motion in its native form for instruction tuning. This method preserves motion-specific details that are often diminished in tokenization, thereby improving the model's ability to interpret complex human behaviors. By processing both video and motion data alongside textual inputs, LLaMo enables a flexible, human-centric analysis. Experimental evaluations across high-complexity domains, including human behaviors and professional activities, indicate that LLaMo effectively captures domain-specific knowledge, enhancing comprehension and prediction in motion-intensive scenarios. We hope LLaMo offers a foundation for future multimodal AI systems with broad applications, from sports analytics to behavioral prediction. Our code and models are available on the project website: https://github.com/ILGLJ/LLaMo.

cs.AI↗

arXiv2025

Solving Schrödinger bridge problem via continuous normalizing flow

The Schrödinger Bridge Problem (SBP), which can be understood as an entropy-regularized optimal transport, seeks to compute stochastic dynamic mappings connecting two given distributions. SBP has shown significant theoretical importance and broad practical potential, with applications spanning a wide range of interdisciplinary fields. While theoretical aspects of the SBP are well-understood, practical computational solutions for general cases have remained challenging. This work introduces a computational framework that leverages continuous normalizing flows and score matching methods to approximate the drift in the dynamic formulation of the SBP. The learned drift term can be used for building generative models, opening new possibilities for applications in probability flow-based methods. We also provide a rigorous $Γ-$convergence analysis for our algorithm, demonstrating that the neuron network solutions converge to the theoretical ones as the regularization parameter tends to infinity. Lastly, we validate our algorithm through numerical experiments on fundamental cases.

math.OC↗

arXiv2025

An algebraic characterization of linearity for additive maps preserving orthogonality

We study when an additive mapping preserving orthogonality between two complex inner product spaces is automatically complex-linear or conjugate-linear. Concretely, let $H$ and $K$ be complex inner product spaces with dim$(H)\geq 2$, and let $A: H\to K$ be an additive map preserving orthogonality. We obtain that $A$ is zero or a positive scalar multiple of a real-linear isometry from $H$ into $K$. We further prove that the following statements are equivalent: $(a)$ $A$ is complex-linear or conjugate-linear. $(b)$ For every $z\in H$ we have $A(i z) \in \{\pm i A(z)\}$. $(c)$ There exists a non-zero point $z\in H$ such that $A(i z) \in \{\pm i A(z)\}$. $(d)$ There exists a non-zero point $z\in H$ such that $i A(z) \in A(H)$. The mapping $A$ neither is complex-linear nor conjugate-linear if, and only if, there exists a non-zero $x\in H$ such that $i A(x)\notin A(H)$ (equivalently, for every non-zero $x\in H$, $i A(x)\notin A(H)$). Among the consequences we show that, under the hypothesis above, the mapping $A$ is automatically complex-linear or conjugate-linear if $A$ has dense range, or if $H$ and $K$ are finite dimensional with dim$(K)< 2\hbox{dim}(H)$.

math.FA↗

arXiv2025

A sharp uniform-in-time error estimate for Stochastic Gradient Langevin Dynamics

We establish a sharp uniform-in-time error estimate for the Stochastic Gradient Langevin Dynamics (SGLD), which is a widely-used sampling algorithm. Under mild assumptions, we obtain a uniform-in-time $O(η^2)$ bound for the KL-divergence between the SGLD iteration and the Langevin diffusion, where $η$ is the step size (or learning rate). Our analysis is also valid for varying step sizes. Consequently, we are able to derive an $O(η)$ bound for the distance between the invariant measures of the SGLD iteration and the Langevin diffusion, in terms of Wasserstein or total variation distances. Our result can be viewed as a significant improvement compared with existing analysis for SGLD in related literature.

math.PR↗

arXiv2025

When GNNs meet symmetry in ILPs: an orbit-based feature augmentation approach

A common characteristic in integer linear programs (ILPs) is symmetry, allowing variables to be permuted without altering the underlying problem structure. Recently, GNNs have emerged as a promising approach for solving ILPs. However, a significant challenge arises when applying GNNs to ILPs with symmetry: classic GNN architectures struggle to differentiate between symmetric variables, which limits their predictive accuracy. In this work, we investigate the properties of permutation equivariance and invariance in GNNs, particularly in relation to the inherent symmetry of ILP formulations. We reveal that the interaction between these two factors contributes to the difficulty of distinguishing between symmetric variables. To address this challenge, we explore the potential of feature augmentation and propose several guiding principles for constructing augmented features. Building on these principles, we develop an orbit-based augmentation scheme that first groups symmetric variables and then samples augmented features for each group from a discrete uniform distribution. Empirical results demonstrate that our proposed approach significantly enhances both training efficiency and predictive performance.

cs.LG↗

arXiv2025

Permute-and-Flip: An optimally stable and watermarkable decoder for LLMs

In this paper, we propose a new decoding method called Permute-and-Flip (PF) decoder. It enjoys stability properties similar to the standard sampling decoder, but is provably up to 2x better in its quality-stability tradeoff than sampling and never worse than any other decoder. We also design a cryptographic watermarking scheme analogous to Aaronson (2023)'s Gumbel watermark, but naturally tailored for PF decoder. The watermarking scheme does not change the distribution to sample, while allowing arbitrarily low false positive rate and high recall whenever the generated text has high entropy. Our experiments show that the PF decoder (and its watermarked counterpart) significantly outperform(s) naive sampling (and its Gumbel watermarked counterpart) in terms of perplexity, while retaining the same stability (and detectability), hence making it a promising new approach for LLM decoding. The code is available at https://github.com/XuandongZhao/pf-decoding

cs.CL↗

arXiv2025

Early signs of stuck pipe detection based on Crossformer

Stuck pipe incidents are one of the major challenges in drilling engineering,leading to massive time loss and additional costs.To address the limitations of insufficient long sequence modeling capability,the difficulty in accurately establishing warning threshold,and the lack of model interpretability in existing methods,we utilize Crossformer for early signs of detection indicating potential stuck events in order to provide guidance for on-site drilling engineers and prevent stuck pipe incidents.The sliding window technique is integrated into Crossformer to allow it to output and display longer outputs,the improved Crossformer model is trained using normal time series drilling data to generate predictions for various parameters at each time step.The relative reconstruction error of model is regard as the risk of stuck pipe,thereby considering data that the model can't predict as anomalies,which represent the early signs of stuck pipe incidents.The multi-step prediction capability of Crossformer and relative reconstruction error are combined to assess stuck pipe risk at each time step in advance.We partition the reconstruction error into modeling error and error due to anomalous data fluctuations,furthermore,the dynamic warning threshold and warning time for stuck pipe incidents are determined using the probability density function of reconstruction errors from normal drilling data.The results indicate that our method can effectively detect early signs of stuck pipe incidents during the drilling process.Crossformer exhibits superior modeling and predictive capabilities compared with other deep learning models.Transformer-based models with multi-step prediction capability are more suitable for stuck pipe prediction compared to the current single-step prediction models.

cs.CE↗

arXiv2025

LONGCODEU: Benchmarking Long-Context Language Models on Long Code Understanding

Current advanced long-context language models offer great potential for real-world software engineering applications. However, progress in this critical domain remains hampered by a fundamental limitation: the absence of a rigorous evaluation framework for long code understanding. To gap this obstacle, we propose a long code understanding benchmark LONGCODEU from four aspects (8 tasks) to evaluate LCLMs' long code understanding ability required for practical applications, including code unit perception, intra-code unit understanding, inter-code unit relation understanding, and long code documentation understanding. We evaluate 9 popular LCLMs on LONGCODEU (i.e., 6 general models and 3 code models). Our experimental results reveal key limitations in current LCLMs' capabilities for long code understanding. Particularly, the performance of LCLMs drops dramatically when the long code length is greater than 32K, falling far short of their claimed 128K-1M context windows. In the four aspects, inter-code unit relation understanding is the most challenging for LCLMs. Our study provides valuable insights for optimizing LCLMs and driving advancements in software engineering.

cs.SE↗

arXiv2025

An Automated Computational Pipeline for Generating Large-Scale Cohorts of Patient-Specific Ventricular Models in Electromechanical In Silico Trials

In recent years, human in silico trials have gained significant traction as a powerful approach to evaluate the effects of drugs, clinical interventions, and medical devices. In silico trials not only minimise patient risks but also reduce reliance on animal testing. However, the implementation of in silico trials presents several time-consuming challenges. It requires the creation of large cohorts of virtual patients. Each virtual patient is described by their anatomy with a volumetric mesh and electrophysiological and mechanical dynamics through mathematical equations and parameters. Furthermore, simulated conditions need definition including stimulation protocols and therapy evaluation. For large virtual cohorts, this requires automatic and efficient pipelines for generation of corresponding files. In this work, we present a computational pipeline to automatically create large virtual patient cohort files to conduct large-scale in silico trials through cardiac electromechanical simulations. The pipeline generates the files describing meshes, labels, and data required for the simulations directly from unprocessed surface meshes. We applied the pipeline to generate over 100 virtual patients from various datasets and performed simulations to demonstrate capacity to conduct in silico trials for virtual patients using verified and validated electrophysiology and electromechanics models for the context of use. The proposed pipeline is adaptable to accommodate different types of ventricular geometries and mesh processing tools, ensuring its versatility in handling diverse clinical datasets. By establishing an automated framework for large scale simulation studies as required for in silico trials and providing open-source code, our work aims to support scalable, personalised cardiac simulations in research and clinical applications.

cs.CE↗

arXiv2025

Gradient Imbalance in Direct Preference Optimization

Direct Preference Optimization (DPO) has been proposed as a promising alternative to Proximal Policy Optimization (PPO) based Reinforcement Learning with Human Feedback (RLHF). However, empirical evaluations consistently reveal suboptimal performance in DPO compared to common RLHF pipelines. In this work, we conduct a systematic analysis of DPO's training dynamics and identify gradient imbalance as a critical limitation. We demonstrate theoretically and empirically that this imbalance perturbs optimization trajectories, destabilizes learning, and induces suboptimal convergence. To address this issue, we propose Balanced-DPO, a simple yet effective modification to the DPO objective that introduces a computationally efficient gradient reweighting mechanism. Our experiments demonstrate the effectiveness of Balanced-DPO, validating the theoretical findings and confirming that addressing gradient imbalance is key to improving DPO's performance, highlighting a promising direction for future research.

cs.LG↗

arXiv2025

A Universal Scheme for Dynamic Partitioned Shortest Path Index: Survey, Improvement, and Experiments

Shortest Path (SP) computation is a fundamental operation in many real-life applications such as navigation on road networks, link analysis on social networks, etc. These networks tend to be massive, and graph partitioning is commonly leveraged to scale up the SP algorithms. However, the Partitioned Shortest Path (PSP) index has never been systematically investigated. Moreover, few studies have explored its index maintenance in dynamic networks. In this paper, we survey the dynamic PSP index and propose a universal scheme for its design and analysis. Specifically, we first review the SP algorithms and put forward a novel structure-based partition method classification to facilitate the selection of partition methods. Furthermore, we summarize the existing Pre-boundary PSP strategy and propose two novel strategies (No-boundary and Post-boundary) to improve its index performance. Lastly, we propose a universal scheme with three dimensions (SP algorithm, partition method, and PSP strategy) to facilitate the analysis and design of the PSP index. Based on this scheme, we put forward five new PSP indexes with a prominent query or update efficiency performance. Extensive experiments are conducted to evaluate the performance of the PSP index and the effectiveness of the proposed techniques, with valuable guidance on the PSP index design.

cs.DB↗

arXiv2025

ChatMotion: A Multimodal Multi-Agent for Human Motion Analysis

Advancements in Multimodal Large Language Models (MLLMs) have improved human motion understanding. However, these models remain constrained by their "instruct-only" nature, lacking interactivity and adaptability for diverse analytical perspectives. To address these challenges, we introduce ChatMotion, a multimodal multi-agent framework for human motion analysis. ChatMotion dynamically interprets user intent, decomposes complex tasks into meta-tasks, and activates specialized function modules for motion comprehension. It integrates multiple specialized modules, such as the MotionCore, to analyze human motion from various perspectives. Extensive experiments demonstrate ChatMotion's precision, adaptability, and user engagement for human motion understanding.

cs.AI↗

arXiv2025

Convergence of random splitting method for the Allen-Cahn equation in a background flow

We study in this paper the convergence of the random splitting method for Allen-Cahn equation in a background flow that plays as a simplified model for phase separation in multiphase flows. The model does not own the gradient flow structure as the usual Allen-Cahn equation does, and the random splitting method is advantageous due to its simplicity and better convergence rate. Though the random splitting is a classical method, the analysis of the convergence is not straightforward for this model due to the nonlinearity and unboundedness of the operators. We obtain uniform estimates of various Sobolev norms of the numerical solutions and the stability of the model. Based on the Sobolev estimates, the local trunction errors are then rigorously obtained. We then prove that the random operator splitting has an expected single run error with order $1.5$ and a bias with order $2$. Numerical experiments are then performed to confirm our theoretic findings.

math.NA↗

arXiv2025

Personalized Topology-Informed Localization of Standard 12-Lead ECG Electrode Placement from Incomplete Cardiac MRIs for Efficient Cardiac Digital Twins

Cardiac digital twins (CDTs) offer personalized in-silico cardiac representations for the inference of multi-scale properties tied to cardiac mechanisms. The creation of CDTs requires precise information about the electrode position on the torso, especially for the personalized electrocardiogram (ECG) calibration. However, current studies commonly rely on additional acquisition of torso imaging and manual/semi-automatic methods for ECG electrode localization. In this study, we propose a novel and efficient topology-informed model to fully automatically extract personalized ECG standard electrode locations from 2D clinically standard cardiac MRIs. Specifically, we obtain the sparse torso contours from the cardiac MRIs and then localize the standard electrodes of 12-lead ECG from the contours. Cardiac MRIs aim at imaging of the heart instead of the torso, leading to incomplete torso geometry within the imaging. To tackle the missing topology, we incorporate the electrodes as a subset of the keypoints, which can be explicitly aligned with the 3D torso topology. The experimental results demonstrate that the proposed model outperforms the time-consuming conventional model projection-based method in terms of accuracy (Euclidean distance: $1.24 \pm 0.293$ cm vs. $1.48 \pm 0.362$ cm) and efficiency ($2$~s vs. $30$-$35$~min). We further demonstrate the effectiveness of using the detected electrodes for in-silico ECG simulation, highlighting their potential for creating accurate and efficient CDT models. The code is available at https://github.com/lileitech/12lead_ECG_electrode_localizer.

eess.IV↗

arXiv2025

Leave No One Behind: Enhancing Diversity While Maintaining Accuracy in Social Recommendation

Social recommendation, which incorporates social connections into recommender systems, has proven effective in improving recommendation accuracy. However, beyond accuracy, diversity is also crucial for enhancing user engagement. Despite its importance, the impact of social recommendation models on diversity remains largely unexplored. In this study, we systematically examine the dual performance of existing social recommendation algorithms in terms of both accuracy and diversity. Our empirical analysis reveals a concerning trend: while social recommendation models enhance accuracy, they often reduce diversity. To address this issue, we propose Diversified Social Recommendation (DivSR), a novel approach that employs relational knowledge distillation to transfer high-diversity structured knowledge from non-social recommendation models to social recommendation models. DivSR is a lightweight, model-agnostic framework that seamlessly integrates with existing social recommendation architectures. Experiments on three benchmark datasets demonstrate that DivSR significantly enhances diversity while maintaining competitive accuracy, achieving a superior accuracy-diversity trade-off. Our code and data are publicly available at: https://github.com/ll0ruc/DivSR.

cs.IR↗

arXiv2025

Random Source Iteration Method: Mitigating the Ray Effect in the Discrete Ordinates Method

The commonly used velocity discretization for simulating the radiative transport equation (RTE) is the discrete ordinates method (DOM). One of the long-standing drawbacks of DOM is the phenomenon known as the ray effect. Due to the high dimensionality of the RTE, DOM results in a large algebraic system to solve. The Source Iteration (SI) method is the most standard iterative method for solving this system. In this paper, by introducing randomness into the SI method, we propose a novel random source iteration (RSI) method that offers a new way to mitigate the ray effect without increasing the computational cost. We have rigorously proved that RSI is unbiased with respect to the SI method and that its variance is uniformly bounded across iteration steps; thus, the convergence order with respect to the number of samples is $1/2$. Furthermore, we prove that the RSI iteration process, as a Markov chain, is ergodic under mild assumptions. Numerical examples are presented to demonstrate the convergence of RSI and its effectiveness in mitigating the ray effect.

math.NA↗

arXiv2025

Ariadne: A Hotness-Aware and Size-Adaptive Compressed Swap Technique for Fast Application Relaunch and Reduced CPU Usage on Mobile Devices

Growing application memory demands and concurrent usage are making mobile device memory scarce. When memory pressure is high, current mobile systems use a RAM-based compressed swap scheme (called ZRAM) to compress unused execution-related data (called anonymous data in Linux) in main memory. We observe that the state-of-the-art ZRAM scheme prolongs relaunch latency and wastes CPU time because it does not differentiate between hot and cold data or leverage different compression chunk sizes and data locality. We make three new observations. 1) anonymous data has different levels of hotness. Hot data, used during application relaunch, is usually similar between consecutive relaunches. 2) when compressing the same amount of anonymous data, small-size compression is very fast, while large-size compression achieves a better compression ratio. 3) there is locality in data access during application relaunch. We propose Ariadne, a compressed swap scheme for mobile devices that reduces relaunch latency and CPU usage with three key techniques. 1) a low-overhead hotness-aware data organization scheme aims to quickly identify the hotness of anonymous data without significant overhead. 2) a size-adaptive compression scheme uses different compression chunk sizes based on the data's hotness level to ensure fast decompression of hot and warm data. 3) a proactive decompression scheme predicts the next set of data to be used and decompresses it in advance, reducing the impact of data swapping back into main memory during application relaunch. Our experimental evaluation results on Google Pixel 7 show that, on average, Ariadne reduces application relaunch latency by 50% and decreases the CPU usage of compression and decompression procedures by 15% compared to the state-of-the-art ZRAM scheme.

cs.OS↗

arXiv2025

High Throughput Shortest Distance Query Processing on Large Dynamic Road Networks

Shortest path (SP) computation is the building block for many location-based services, and achieving high throughput SP query processing with real-time response is crucial for those services. However, existing solutions can hardly handle high throughput queries on large dynamic road networks due to either slow query efficiency or poor dynamic adaption. In this paper, we leverage graph partitioning and propose novel Partitioned Shortest Path (PSP) indexes to address this problem. Specifically, we first put forward a cross-boundary strategy to accelerate the query processing of PSP index and analyze its efficiency upper bound theoretically. After that, we propose a non-trivial Partitioned Multi-stage Hub Labeling (PMHL) that subtly aggregates multiple PSP strategies to achieve fast index maintenance and consecutive query efficiency improvement during index update. Lastly, to further optimize throughput, we design tree decomposition-based graph partitioning and propose Post-partitioned MHL (PostMHL) with faster query processing and index update. Experiments on real-world road networks show that our methods outperform state-of-the-art baselines in query throughput, yielding up to 2 orders of magnitude improvement.

cs.DB↗