arXiv subjects

Zheng Li

Publications and source records attributed to Zheng Li.

At least 217 records · Page 12Linked to original sources

arXiv2025

Hephaestus: Improving Fundamental Agent Capabilities of Large Language Models through Continual Pre-Training

Due to the scarcity of agent-oriented pre-training data, LLM-based autonomous agents typically rely on complex prompting or extensive fine-tuning, which often fails to introduce new capabilities while preserving strong generalizability. We introduce Hephaestus-Forge, the first large-scale pre-training corpus designed to enhance the fundamental capabilities of LLM agents in API function calling, intrinsic reasoning and planning, and adapting to environmental feedback. Hephaestus-Forge comprises 103B agent-specific data encompassing 76,537 APIs, including both tool documentation to introduce knowledge of API functions and function calling trajectories to strengthen intrinsic reasoning. To explore effective training protocols, we investigate scaling laws to identify the optimal recipe in data mixing ratios. By continual pre-training on Hephaestus-Forge, Hephaestus outperforms small- to medium-scale open-source LLMs and rivals commercial LLMs on three agent benchmarks, demonstrating the effectiveness of our pre-training corpus in enhancing fundamental agentic capabilities and generalization of LLMs to new tasks or environments.

cs.CL↗

arXiv2025

Membership Inference Attacks Against Vision-Language Models

Vision-Language Models (VLMs), built on pre-trained vision encoders and large language models (LLMs), have shown exceptional multi-modal understanding and dialog capabilities, positioning them as catalysts for the next technological revolution. However, while most VLM research focuses on enhancing multi-modal interaction, the risks of data misuse and leakage have been largely unexplored. This prompts the need for a comprehensive investigation of such risks in VLMs. In this paper, we conduct the first analysis of misuse and leakage detection in VLMs through the lens of membership inference attack (MIA). In specific, we focus on the instruction tuning data of VLMs, which is more likely to contain sensitive or unauthorized information. To address the limitation of existing MIA methods, we introduce a novel approach that infers membership based on a set of samples and their sensitivity to temperature, a unique parameter in VLMs. Based on this, we propose four membership inference methods, each tailored to different levels of background knowledge, ultimately arriving at the most challenging scenario. Our comprehensive evaluations show that these methods can accurately determine membership status, e.g., achieving an AUC greater than 0.8 targeting a small set consisting of only 5 samples on LLaVA.

cs.CR↗

arXiv2025

Static Batching of Irregular Workloads on GPUs: Framework and Application to Efficient MoE Model Inference

It has long been a problem to arrange and execute irregular workloads on massively parallel devices. We propose a general framework for statically batching irregular workloads into a single kernel with a runtime task mapping mechanism on GPUs. We further apply this framework to Mixture-of-Experts (MoE) model inference and implement an optimized and efficient CUDA kernel. Our MoE kernel achieves up to 91% of the peak Tensor Core throughput on NVIDIA H800 GPU and 95% on NVIDIA H20 GPU.

cs.DC↗

arXiv2025

Giant Anomalous Hall Effect in Kagome Nodal Surface Semimetal Fe$_3$Ge

It is well known that the intrinsic anomalous Hall effect (AHE) arises from the integration of the non-zero Berry curvature (BC), conventionally observed in the Dirac/Weyl and nodal-line semimetals. Moreover, nodal surface semimetals are expected to exhibit more significant BC under the prevalence of degenerate points near the Fermi level. In this work, we report the detection of a giant AHE in the Kagome magnet Fe$_3$Ge with a two-dimensional (2D) nodal surface (NS) at $k_{z}=π$ plane, exhibiting an anomalous Hall conductivity (AHC) of 1500 $Ω^{-1}$cm$^{-1}$ at 160 K, the highest among all reported Kagome topological materials. This finding suggests a new platform for searching large AHC materials and facilitates potential room-temperature applications in spintronic devices and quantum computing.

cond-mat.mes-hall↗

arXiv2025

Assessing Markov Property in Driving Behaviors: Insights from Statistical Tests

The Markov property serves as a foundational assumption in most existing work on vehicle driving behavior, positing that future states depend solely on the current state, not the series of preceding states. This study validates the Markov properties of vehicle trajectories for both Autonomous Vehicles (AVs) and Human-driven Vehicles (HVs). A statistical method used to test whether time series data exhibits Markov properties is applied to examine whether the trajectory data possesses Markov characteristics. t test and F test are additionally introduced to characterize the differences in Markov properties between AVs and HVs. Based on two public trajectory datasets, we investigate the presence and order of the Markov property of different types of vehicles through rigorous statistical tests. Our findings reveal that AV trajectories generally exhibit stronger Markov properties compared to HV trajectories, with a higher percentage conforming to the Markov property and lower Markov orders. In contrast, HV trajectories display greater variability and heterogeneity in decision-making processes, reflecting the complex perception and information processing involved in human driving. These results have significant implications for the development of driving behavior models, AV controllers, and traffic simulation systems. Our study also demonstrates the feasibility of using statistical methods to test the presence of Markov properties in driving trajectory data.

cs.LG↗

arXiv2025

The coherence peak of unconventional superconductors in the charge channel

In this work, we carry out a systematic investigation of the coherence peak in unconventional superconductors as they transition into the superconducting phase at $T_c$. Using $d$-wave cuprates as an example, we reveal the presence of a coherence peak below $T_c$ in the charge channel. The nuclear quadrupole relaxation rate is shown to be an effective method for detecting this unconventional coherence peak, with the superconducting coherence factor playing a pivotal role in its emergence. Additionally, we explore the influence of correlation effects, which further enhance this phenomenon. Extending our analysis, we demonstrate the existence of a similar coherence peak in ultrasonic attenuation and iron-based superconductors. Our findings offer a fresh perspective on probing superconducting gap symmetry in unconventional superconductors.

cond-mat.supr-con↗

arXiv2025

Unconventional Coherence Peak in Cuprate Superconductors

The Hebel-Slichter coherence peak, observed in the spin-lattice relaxation rate $1/T_1$ just below the critical temperature $T_{\rm c}$, serves as a crucial experimental validation of the Bardeen-Cooper-Schrieffer pairing symmetry in conventional superconductors. However, no coherence peak in $1/T_1$ has been observed in unconventional superconductors like cuprates. In this study, an unconventional coherence peak is identified for the first time using nuclear quadrupole resonance on YBa$_2$Cu$_4$O$_8$, pointing to a distinctive pairing symmetry. The spin-lattice relaxation rate in nuclear quadrupole resonance and nuclear magnetic resonance with nuclear spin $I>1/2$ comprises the magnetic relaxation rate $1/T_{1}^{\rm mag}$, which probes magnetic fluctuations, and the quadrupole relaxation rate $1/T_{1}^{\rm quad}$, which probes charge fluctuations. By utilizing $^{63}$Cu and $^{65}$Cu isotopes, we successfully distinguish $1/T_{1}^{\rm mag}$ and $1/T_{1 }^{\rm quad}$ of YBa$_2$Cu$_4$O$_8$ and reveal the presence of the coherence peak in $1/T_{1 }^{\rm quad}$ but not in $1/T_{1}^{\rm mag}$, in contrast to conventional superconductors. Our finding demonstrates that unconventional superconductors do not exhibit a coherence peak in $1/T_{1}$ when the relaxation is due to fluctuations of the hyperfine field. Conversely, a coherence peak is expected when the relaxation is caused by electric field gradient fluctuations, due to the different coherence factors between charge and magnetic fluctuations. Our successful measurements of $1/T_{1}$ for the chains of YBa$_2$Cu$_4$O$_8$ suggest that, should the conditions for predominant quadrupole relaxation be satisfied, this phenomenon could provide a novel approach to exploring the unconventional nature of the pairing mechanism in other superconductors.

cond-mat.supr-con↗

arXiv2024

Plug-and-Play Training Framework for Preference Optimization

Recently, preference optimization methods such as DPO have significantly enhanced large language models (LLMs) in wide tasks including dialogue and question-answering. However, current methods fail to account for the varying difficulty levels of training samples during preference optimization, leading to mediocre performance in tasks with high accuracy requirements, particularly in mathematical reasoning. To address this limitation, we propose a novel training framework, which employs multiple sampling to analyze output distributions, assign different weights to samples, and incorporate these weights into the preference optimization process. This plug-and-play approach enables LLMs to prioritize challenging examples during training, improving learning efficiency. Experimental results demonstrate that our framework integrates seamlessly with various preference optimization methods and achieves consistent improvements in mathematical reasoning tasks.

cs.CL↗

arXiv2024

Evaluating the Influence of Satellite Systems on Terrestrial Networks: Analyzing S-Band Interference

The co-existence of terrestrial and non-terrestrial networks (NTNs) is essential for achieving comprehensive global coverage in sixth-generation cellular networks. Given the escalating demand for spectrum, there is an ongoing global discourse on the feasibility of sharing certain frequencies currently utilized by terrestrial networks (TNs) with NTNs. However, this sharing leads to co-channel interference and subsequent performance degradation. This paper specifically investigates the interference caused by NTNs on TNs in the S-band and its relationship with the relative position between satellite and TN user equipment. We analyzed the transmission mechanisms of satellite signals and employed the ITU two-state model for our interference analysis. Through simulations, we evaluated the interference intensity at different separation distances and slant ranges. Our findings reveal that the angle between the user equipment direction and the sub-satellite point direction from the beam center significantly influences the interference level. Furthermore, we determine the minimum separation distance needed to keep the interference-to-noise ratio of NTN interference below 0 dB.

eess.SP↗

arXiv2024

MiMoTable: A Multi-scale Spreadsheet Benchmark with Meta Operations for Table Reasoning

Extensive research has been conducted to explore the capability of Large Language Models (LLMs) for table reasoning and has significantly improved the performance on existing benchmarks. However, tables and user questions in real-world applications are more complex and diverse, presenting an unignorable gap compared to the existing benchmarks. To fill the gap, we propose a \textbf{M}ult\textbf{i}-scale spreadsheet benchmark with \textbf{M}eta \textbf{o}perations for \textbf{Table} reasoning, named as MiMoTable. Specifically, MiMoTable incorporates two key features. First, the tables in MiMoTable are all spreadsheets used in real-world scenarios, which cover seven domains and contain different types. Second, we define a new criterion with six categories of meta operations for measuring the difficulty of each question in MiMoTable, simultaneously as a new perspective for measuring the difficulty of the existing benchmarks. Experimental results show that Claude-3.5-Sonnet achieves the best performance with 77.4\% accuracy, indicating that there is still significant room to improve for LLMs on MiMoTable. Furthermore, we grade the difficulty of existing benchmarks according to our new criteria. Experiments have shown that the performance of LLMs decreases as the difficulty of benchmarks increases, thereby proving the effectiveness of our proposed new criterion.

cs.CL↗

arXiv2024

Diffraction patterns in attosecond photoionization time delay

Upon absorbing a photon, the ionized electron sails through the target force field in attoseconds to reach free space. This navigation probes details of the potential landscape that get imprinted into the phase of the ionization amplitude. The Eisenbud-Wigner-Smith (EWS) time delay, the energy derivative of this phase, provides the navigation time relative to the time of the electron's ``free'' exit. This time is influenced by the diffraction of the electron from the potential landscape, offering structural and dynamical information about interactions. If the potential has an intrinsic symmetry, a regular pattern in the time delay, including subpatterns of delays and advances, may occur from the diffraction process. The recent synthesis of a polyhedral fluorocarbon instigates the current study of photoionization from a cubic molecule. Our simulation of the EWS delay unravels rich diffraction motifs within $\pm$100 attoseconds in both energy and angular distributions. Averaging over the Euler angles from the laboratory to the molecular frame and over the photoelectron azimuthal direction indicates that the pattern should be discernible in ultrafast chronoscopy. The study benchmarks diffraction in molecular photoionization as a fundamental process which can be experimentally accessed through ultrafast time delay.

physics.atom-ph↗

arXiv2024

Contrasting Adversarial Perturbations: The Space of Harmless Perturbations

Existing works have extensively studied adversarial examples, which are minimal perturbations that can mislead the output of deep neural networks (DNNs) while remaining imperceptible to humans. However, in this work, we reveal the existence of a harmless perturbation space, in which perturbations drawn from this space, regardless of their magnitudes, leave the network output unchanged when applied to inputs. Essentially, the harmless perturbation space emerges from the usage of non-injective functions (linear or non-linear layers) within DNNs, enabling multiple distinct inputs to be mapped to the same output. For linear layers with input dimensions exceeding output dimensions, any linear combination of the orthogonal bases of the nullspace of the parameter consistently yields no change in their output. For non-linear layers, the harmless perturbation space may expand, depending on the properties of the layers and input samples. Inspired by this property of DNNs, we solve for a family of general perturbation spaces that are redundant for the DNN's decision, and can be used to hide sensitive data and serve as a means of model identification. Our work highlights the distinctive robustness of DNNs (i.e., consistency under large magnitude perturbations) in contrast to adversarial examples (vulnerability for small imperceptible noises).

cs.LG↗

arXiv2024

Steering Non-Equilibrium Molecular Dynamics in Optical Cavities

Optical resonators have shown outstanding abilities to tailor chemical landscapes through enhanced light-matter interaction between confined optical modes and molecule vibrations. We propose a theoretical model to study cooperative vibrational strong coupling in an open quantum system. The non-equilibrium stochastic molecular dynamics in an optical cavity with an auxiliary ensemble is investigated. It shows that coupling with a cavity mode introduces an additional colored noise and a negative feedback, both of which enable control over thermalization rates (i.e. the lifetime of excitation) of reactive molecules. Our work offers a pathway to steer stability of chemical bonds for chemical reactivity under cooperative vibrational strong coupling.

physics.optics↗

arXiv2024

Rotation Invariant Quantization for Model Compression

Post-training Neural Network (NN) model compression is an attractive approach for deploying large, memory-consuming models on devices with limited memory resources. In this study, we investigate the rate-distortion tradeoff for NN model compression. First, we suggest a Rotation-Invariant Quantization (RIQ) technique that utilizes a single parameter to quantize the entire NN model, yielding a different rate at each layer, i.e., mixed-precision quantization. Then, we prove that our rotation-invariant approach is optimal in terms of compression. We rigorously evaluate RIQ and demonstrate its capabilities on various models and tasks. For example, RIQ facilitates $\times 19.4$ and $\times 52.9$ compression ratios on pre-trained VGG dense and pruned models, respectively, with $<0.4\%$ accuracy degradation. Code is available in \href{https://github.com/ehaleva/RIQ}{github.com/ehaleva/RIQ}.

cs.LG↗

arXiv2024

Finite Volume Physical Informed Neural Network (FV-PINN) with Reduced Derivative Order for Incompressible Flows

Physics-Informed Neural Networks (PINN) has evolved into a powerful tool for solving partial differential equations, which has been applied to various fields such as energy, environment, en-gineering, etc. When utilizing PINN to solve partial differential equations, it is common to rely on Automatic Differentiation (AD) to compute the residuals of the governing equations. This can lead to certain precision losses, thus affecting the accuracy of the network prediction. This paper pro-poses a Finite Volume Physics-Informed Neural Network (FV-PINN), designed to address steady-state problems of incompressible flow. This method divides the solution domain into mul-tiple grids. Instead of calculating the residuals of the Navier-Stokes equations at collocation points within the grid, as is common in traditional PINNs, this approach evaluates them at Gaussian in-tegral points on the grid boundaries using Gauss's theorem. The loss function is constructed using the Gaussian integral method, and the differentiation order for velocity is reduced. To validate the effectiveness of this approach, we predict the velocity and pressure fields for two typical examples in fluid topology optimization. The results are compared with commercial software COMSOL, which indicates that FVI-PINN significantly improves the prediction accuracy of both the velocity and pressure fields while accelerating the training speed of the network.

physics.flu-dyn↗

arXiv2024

Personalized Playback Technology: How Short Video Services Create Excellent User Experience

Short-form video content has become increasingly popular and influential in recent years. Its concise yet engaging format aligns well with todays' fast-paced and on-the-go lifestyles, making it a dominating trend in the digital world. As one of the front runners in the short video platform space, ByteDance has been highly successful in delivering a one-of-a-kind short video experience and attracting billions of users worldwide. One key contributing factor is its advanced end-to-end personalized short video playback technology, where we pioneered and developed the new technical field over the past five years to optimize user experience. This paper introduces the major concepts and methodologies of this personalized video playback technology that distinguish it from traditional multimedia technologies. More details, including goal setting, iterative process, modeling, experimental methods and required supporting systems, are also provided to encourage deeper research in this area.

cs.MM↗

arXiv2024

Personalized News Recommendation System via LLM Embedding and Co-Occurrence Patterns

In the past two years, large language models (LLMs) have achieved rapid development and demonstrated remarkable emerging capabilities. Concurrently, with powerful semantic understanding and reasoning capabilities, LLMs have significantly empowered the rapid advancement of the recommendation system field. Specifically, in news recommendation (NR), systems must comprehend and process a vast amount of clicked news text to infer the probability of candidate news clicks. This requirement exceeds the capabilities of traditional NR models but aligns well with the strengths of LLMs. In this paper, we propose a novel NR algorithm to reshape the news model via LLM Embedding and Co-Occurrence Pattern (LECOP). On one hand, we fintuned LLM by contrastive learning using large-scale datasets to encode news, which can fully explore the semantic information of news to thoroughly identify user preferences. On the other hand, we explored multiple co-occurrence patterns to mine collaborative information. Those patterns include news ID co-occurrence, Item-Item keywords co-occurrence and Intra-Item keywords co-occurrence. The keywords mentioned above are all generated by LLM. As far as we know, this is the first time that constructing such detailed Co-Occurrence Patterns via LLM to capture collaboration. Extensive experiments demonstrate the superior performance of our proposed novel method

cs.AI↗

arXiv2024

Shopping MMLU: A Massive Multi-Task Online Shopping Benchmark for Large Language Models

Online shopping is a complex multi-task, few-shot learning problem with a wide and evolving range of entities, relations, and tasks. However, existing models and benchmarks are commonly tailored to specific tasks, falling short of capturing the full complexity of online shopping. Large Language Models (LLMs), with their multi-task and few-shot learning abilities, have the potential to profoundly transform online shopping by alleviating task-specific engineering efforts and by providing users with interactive conversations. Despite the potential, LLMs face unique challenges in online shopping, such as domain-specific concepts, implicit knowledge, and heterogeneous user behaviors. Motivated by the potential and challenges, we propose Shopping MMLU, a diverse multi-task online shopping benchmark derived from real-world Amazon data. Shopping MMLU consists of 57 tasks covering 4 major shopping skills: concept understanding, knowledge reasoning, user behavior alignment, and multi-linguality, and can thus comprehensively evaluate the abilities of LLMs as general shop assistants. With Shopping MMLU, we benchmark over 20 existing LLMs and uncover valuable insights about practices and prospects of building versatile LLM-based shop assistants. Shopping MMLU can be publicly accessed at https://github.com/KL4805/ShoppingMMLU. In addition, with Shopping MMLU, we host a competition in KDD Cup 2024 with over 500 participating teams. The winning solutions and the associated workshop can be accessed at our website https://amazon-kddcup24.github.io/.

cs.LG↗