arXiv subjects

Peng Cheng

Publications and source records attributed to Peng Cheng.

At least 109 records · Page 6Linked to original sources

arXiv2025

ContextCache: Context-Aware Semantic Cache for Multi-Turn Queries in Large Language Models

Semantic caching significantly reduces computational costs and improves efficiency by storing and reusing large language model (LLM) responses. However, existing systems rely primarily on matching individual queries, lacking awareness of multi-turn dialogue contexts, which leads to incorrect cache hits when similar queries appear in different conversational settings. This demonstration introduces ContextCache, a context-aware semantic caching system for multi-turn dialogues. ContextCache employs a two-stage retrieval architecture that first executes vector-based retrieval on the current query to identify potential matches and then integrates current and historical dialogue representations through self-attention mechanisms for precise contextual matching. Evaluation of real-world conversations shows that ContextCache improves precision and recall compared to existing methods. Additionally, cached responses exhibit approximately 10 times lower latency than direct LLM invocation, enabling significant computational cost reductions for LLM conversational applications.

cs.CL↗

arXiv2025

Design and construction of the multiplexing cold neutron spectrometer BOYA with double-column Rowland focusing analyzers

Developing neutron spectrometers with higher counting efficiency has been an essential pursuit in neutron instrumentation. In this work, we present BOYA, a multiplexing cold neutron spectrometers designed and implemented at the China Advanced Research Reactor. Equipped with 34 angular analyzing channels spanning 119°, each containing 5 inelastic channels and 1 diffraction channel, BOYA enhances the measurement efficiency by two orders of magnitude over a traditional triple-axis spectrometer. To optimize both intensity and energy resolution, innovative double-column Rowland focusing analyzers have been developed. By filling the crystal gaps in the traditional Rowland focusing geometry, our design enhances the neutron beam coverage without introducing appreciable double-scattering. Our commissioning results on vanadium and MnWO4 have confirmed the success of the design, establishing BOYA as a successful multiplexing instrument for neutron spectroscopy.

physics.ins-det↗

arXiv2025

Reasoning on a Budget: A Survey of Adaptive and Controllable Test-Time Compute in LLMs

Large language models (LLMs) have rapidly progressed into general-purpose agents capable of solving a broad spectrum of tasks. However, current models remain inefficient at reasoning: they apply fixed inference-time compute regardless of task complexity, often overthinking simple problems while underthinking hard ones. This survey presents a comprehensive review of efficient test-time compute (TTC) strategies, which aim to improve the computational efficiency of LLM reasoning. We introduce a two-tiered taxonomy that distinguishes between L1-controllability, methods that operate under fixed compute budgets, and L2-adaptiveness, methods that dynamically scale inference based on input difficulty or model confidence. We benchmark leading proprietary LLMs across diverse datasets, highlighting critical trade-offs between reasoning performance and token usage. Compared to prior surveys on efficient reasoning, our review emphasizes the practical control, adaptability, and scalability of TTC methods. Finally, we discuss emerging trends such as hybrid thinking models and identify key challenges for future work towards making LLMs more computationally efficient, robust, and responsive to user constraints.

cs.AI↗

arXiv2025

Privacy-Preserving Resilient Vector Consensus

This paper studies privacy-preserving resilient vector consensus in multi-agent systems against faulty agents, where normal agents can achieve consensus within the convex hull of their initial states while protecting state vectors from being disclosed. Specifically, we consider a modification of an existing algorithm known as Approximate Distributed Robust Convergence Using Centerpoints (ADRC), i.e., Privacy-Preserving ADRC (PP-ADRC). Under PP-ADRC, each normal agent introduces multivariate Gaussian noise to its state during each iteration. We first provide sufficient conditions to ensure that all normal agents' states can achieve mean square convergence under PP-ADRC. Then, we analyze convergence accuracy from two perspectives, i.e., the Mahalanobis distance of the final value from its expectation and the Hausdorff distance-based alteration of the convex hull caused by noise when only partial dimensions are added with noise. Then, we employ concentrated geo-privacy to characterize privacy preservation and conduct a thorough comparison with differential privacy. Finally, numerical simulations demonstrate the theoretical results.

eess.SY↗

arXiv2025

ImprovDML: Improved Trade-off in Private Byzantine-Resilient Distributed Machine Learning

Jointly addressing Byzantine attacks and privacy leakage in distributed machine learning (DML) has become an important issue. A common strategy involves integrating Byzantine-resilient aggregation rules with differential privacy mechanisms. However, the incorporation of these techniques often results in a significant degradation in model accuracy. To address this issue, we propose a decentralized DML framework, named ImprovDML, that achieves high model accuracy while simultaneously ensuring privacy preservation and resilience to Byzantine attacks. The framework leverages a kind of resilient vector consensus algorithms that can compute a point within the normal (non-Byzantine) agents' convex hull for resilient aggregation at each iteration. Then, multivariate Gaussian noises are introduced to the gradients for privacy preservation. We provide convergence guarantees and derive asymptotic learning error bounds under non-convex settings, which are tighter than those reported in existing works. For the privacy analysis, we adopt the notion of concentrated geo-privacy, which quantifies privacy preservation based on the Euclidean distance between inputs. We demonstrate that it enables an improved trade-off between privacy preservation and model accuracy compared to differential privacy. Finally, numerical simulations validate our theoretical results.

cs.LG↗

arXiv2025

Atomic to mesoscale hierarchical structures and magnetic states in an anisotropic layered ferromagnet FePd2Te2

Two-dimensional (2D) magnetic materials have predominantly exhibited easy-axis or easy-plane anisotropy and display a high sensitivity to the underlying crystal structure and lattice symmetry. Recently, an in-plane anisotropic 2D ferromagnet of FePd2Te2 has been discovered with intriguing structure and quasi-one-dimensional spin system. Here, we report a real-space investigation of its twinning structure and magnetic states using atomic/magnetic force microscopy (AFM/MFM) combined with scanning tunneling microscopy (STM). The atomic to mesoscale hierarchical structures with the orthogonal and corrugated compressive /tensile(C/T) regions are directly observed due to the intrinsic twinning-domain characteristic. The structure-related intact ferromagnetic (FM), field-induced polarized-FM states and their transitions are comparatively discussed at the mesoscale with the corresponding macroscopic magnetic measurements. Temperature- and field-dependent evolution of magnetic phase are further investigated at the FM and PM states, and summarized to obtain a unique H-T phase diagram of FePd2Te2. Our work provides key results for understanding the complicated magnetic properties of FePd2Te2, and suggests new directions for manipulating magnetic states through the atomic and mesoscale structure engineering.

cond-mat.mtrl-sci↗

arXiv2025

Field-induced magnetic order in DyTa$_7$O$_{19}$ with two-dimensional pseudospin-$\frac{1}{2}$ triangular lattice

The magnetic ground state of geometrically frustrated antiferromagnet attracts great research interests due to the possibility to realize novel quantum magnetic state such as a quantum spin liquid. Here we present a comprehensive magnetic characterization of DyTa$_7$O$_{19}$ with ideal two-dimensional triangular lattice. DyTa$_7$O$_{19}$ exhibits $c$-axis single-ion magnetic anisotropy. Although long-range magnetic order is not observed down to 100 mK under zero field, by applying a small magnetic field ($\sim$0.1 T), a magnetically ordered state with net magnetization of $M_s$/3 below $T_m$=0.14 K is identified ($M_s$ denotes the saturated magnetization). We argue that this state is an up-up-down magnetic structure phase driven by the dipole-dipole interactions between Ising-like spins of Dy$^{3+}$ in a two-dimensional triangular lattice, since its ordering temperature and temperature-field phase diagram can be well explained by the theoretical calculations based on dipolar interactions. DyTa$_7$O$_{19}$ could be viewed as a rare material platform that realizing pure Ising-like dipolar interaction in a geometrically frustrated lattice.

cond-mat.str-el↗

arXiv2025

Skywork Open Reasoner 1 Technical Report

The success of DeepSeek-R1 underscores the significant role of reinforcement learning (RL) in enhancing the reasoning capabilities of large language models (LLMs). In this work, we present Skywork-OR1, an effective and scalable RL implementation for long Chain-of-Thought (CoT) models. Building on the DeepSeek-R1-Distill model series, our RL approach achieves notable performance gains, increasing average accuracy across AIME24, AIME25, and LiveCodeBench from 57.8% to 72.8% (+15.0%) for the 32B model and from 43.6% to 57.5% (+13.9%) for the 7B model. Our Skywork-OR1-32B model surpasses both DeepSeek-R1 and Qwen3-32B on the AIME24 and AIME25 benchmarks, while achieving comparable results on LiveCodeBench. The Skywork-OR1-7B and Skywork-OR1-Math-7B models demonstrate competitive reasoning capabilities among models of similar size. We perform comprehensive ablation studies on the core components of our training pipeline to validate their effectiveness. Additionally, we thoroughly investigate the phenomenon of entropy collapse, identify key factors affecting entropy dynamics, and demonstrate that mitigating premature entropy collapse is critical for improved test performance. To support community research, we fully open-source our model weights, training code, and training datasets.

cs.LG↗

arXiv2025

OSS-UAgent: An Agent-based Usability Evaluation Framework for Open Source Software

Usability evaluation is critical to the impact and adoption of open source software (OSS), yet traditional methods relying on human evaluators suffer from high costs and limited scalability. To address these limitations, we introduce OSS-UAgent, an automated, configurable, and interactive agent-based usability evaluation framework specifically designed for open source software. Our framework employs intelligent agents powered by large language models (LLMs) to simulate developers performing programming tasks across various experience levels (from Junior to Expert). By dynamically constructing platform-specific knowledge bases, OSS-UAgent ensures accurate and context-aware code generation. The generated code is automatically evaluated across multiple dimensions, including compliance, correctness, and readability, providing a comprehensive measure of the software's usability. Additionally, our demonstration showcases OSS-UAgent's practical application in evaluating graph analytics platforms, highlighting its effectiveness in automating usability evaluation.

cs.SE↗

arXiv2025

SoK: Dataset Copyright Auditing in Machine Learning Systems

As the implementation of machine learning (ML) systems becomes more widespread, especially with the introduction of larger ML models, we perceive a spring demand for massive data. However, it inevitably causes infringement and misuse problems with the data, such as using unauthorized online artworks or face images to train ML models. To address this problem, many efforts have been made to audit the copyright of the model training dataset. However, existing solutions vary in auditing assumptions and capabilities, making it difficult to compare their strengths and weaknesses. In addition, robustness evaluations usually consider only part of the ML pipeline and hardly reflect the performance of algorithms in real-world ML applications. Thus, it is essential to take a practical deployment perspective on the current dataset copyright auditing tools, examining their effectiveness and limitations. Concretely, we categorize dataset copyright auditing research into two prominent strands: intrusive methods and non-intrusive methods, depending on whether they require modifications to the original dataset. Then, we break down the intrusive methods into different watermark injection options and examine the non-intrusive methods using various fingerprints. To summarize our results, we offer detailed reference tables, highlight key points, and pinpoint unresolved issues in the current literature. By combining the pipeline in ML systems and analyzing previous studies, we highlight several future directions to make auditing tools more suitable for real-world copyright protection requirements.

cs.CR↗

arXiv2025

Optimizing Large Language Model Training Using FP4 Quantization

The growing computational demands of training large language models (LLMs) necessitate more efficient methods. Quantized training presents a promising solution by enabling low-bit arithmetic operations to reduce these costs. While FP8 precision has demonstrated feasibility, leveraging FP4 remains a challenge due to significant quantization errors and limited representational capacity. This work introduces the first FP4 training framework for LLMs, addressing these challenges with two key innovations: a differentiable quantization estimator for precise weight updates and an outlier clamping and compensation strategy to prevent activation collapse. To ensure stability, the framework integrates a mixed-precision training scheme and vector-wise quantization. Experimental results demonstrate that our FP4 framework achieves accuracy comparable to BF16 and FP8, with minimal degradation, scaling effectively to 13B-parameter LLMs trained on up to 100B tokens. With the emergence of next-generation hardware supporting FP4, our framework sets a foundation for efficient ultra-low precision training.

cs.LG↗

arXiv2025

Beyond Prompt Content: Enhancing LLM Performance via Content-Format Integrated Prompt Optimization

Large Language Models (LLMs) have shown significant capability across various tasks, with their real-world effectiveness often driven by prompt design. While recent research has focused on optimizing prompt content, the role of prompt formatting, a critical but often overlooked dimension, has received limited systematic investigation. In this paper, we introduce Content-Format Integrated Prompt Optimization (CFPO), an innovative methodology that jointly optimizes both prompt content and formatting through an iterative refinement process. CFPO leverages natural language mutations to explore content variations and employs a dynamic format exploration strategy that systematically evaluates diverse format options. Our extensive evaluations across multiple tasks and open-source LLMs demonstrate that CFPO demonstrates measurable performance improvements compared to content-only optimization methods. This highlights the importance of integrated content-format optimization and offers a practical, model-agnostic approach to enhancing LLM performance. Code is available at https://github.com/HenryLau7/CFPO.

cs.CL↗

arXiv2025

Velocitune: A Velocity-based Dynamic Domain Reweighting Method for Continual Pre-training

It is well-known that a diverse corpus is critical for training large language models, which are typically constructed from a mixture of various domains. In general, previous efforts resort to sampling training data from different domains with static proportions, as well as adjusting data proportions during training. However, few methods have addressed the complexities of domain-adaptive continual pre-training. To fill this gap, we propose Velocitune, a novel framework dynamically assesses learning velocity and adjusts data proportions accordingly, favoring slower-learning domains while shunning faster-learning ones, which is guided by a scaling law to indicate the desired learning goal for each domain with less associated cost. To evaluate the effectiveness of Velocitune, we conduct experiments in a reasoning-focused dataset with CodeLlama, as well as in a corpus specialised for system command generation with Llama3 and Mistral. Velocitune achieves performance gains in both math and code reasoning tasks and command-line generation benchmarks. Further analysis reveals that key factors driving Velocitune's effectiveness include target loss prediction and data ordering.

cs.CL↗

arXiv2025

FedSaaS: Class-Consistency Federated Semantic Segmentation via Global Prototype Supervision and Local Adversarial Harmonization

Federated semantic segmentation enables pixel-level classification in images through collaborative learning while maintaining data privacy. However, existing research commonly overlooks the fine-grained class relationships within the semantic space when addressing heterogeneous problems, particularly domain shift. This oversight results in ambiguities between class representation. To overcome this challenge, we propose a novel federated segmentation framework that strikes class consistency, termed FedSaaS. Specifically, we introduce class exemplars as a criterion for both local- and global-level class representations. On the server side, the uploaded class exemplars are leveraged to model class prototypes, which supervise global branch of clients, ensuring alignment with global-level representation. On the client side, we incorporate an adversarial mechanism to harmonize contributions of global and local branches, leading to consistent output. Moreover, multilevel contrastive losses are employed on both sides to enforce consistency between two-level representations in the same semantic space. Extensive experiments on several driving scene segmentation datasets demonstrate that our framework outperforms state-of-the-art methods, significantly improving average segmentation accuracy and effectively addressing the class-consistency representation problem.

cs.CV↗

arXiv2025

Thermodynamics of the Kerr-AdS black hole from an ensemble-averaged theory

Exploring the universal structure of the gravitational path integral beyond semi-classical saddles and uncovering a compelling statistical interpretation of black hole thermodynamics have long been significant challenges. We investigate the statistical interpretation of the Kerr-AdS black hole thermodynamics through an ensemble-averaged theory. By extending the phase space to include all possible states with conical singularities in their Euclidean counterparts, we derive the probability distribution of different states inherited from the Euclidean gravitational path integral. Moreover, we can define a density matrix of all states in the phase space. By ensemble-averaging over all states, we show that the black hole phase transition naturally arises in the semi-classical limit. Away from the semi-classical regime, the ensemble-averaged theory exhibits a notable deviation from the conventional phase transition. Expanding around the classical saddles yields the subleading-order correction to the Gibbs free energy, which is half of the Hawking temperature. We demonstrate that the half Hawking temperature correction is a universal feature inherent to black holes in asymptotically AdS spacetime. With the subleading-order correction to Gibbs free energy, we also suggest that the whole black hole thermodynamic should be corrected accordingly.

hep-th↗

arXiv2025

Exploring the Frontiers of LLMs in Psychological Applications: A Comprehensive Review

This paper explores the frontiers of large language models (LLMs) in psychology applications. Psychology has undergone several theoretical changes, and the current use of Artificial Intelligence (AI) and Machine Learning, particularly LLMs, promises to open up new research directions. We provide a detailed exploration of how LLMs like ChatGPT are transforming psychological research. It discusses the impact of LLMs across various branches of psychology, including cognitive and behavioral, clinical and counseling, educational and developmental, and social and cultural psychology, highlighting their potential to simulate aspects of human cognition and behavior. The paper delves into the capabilities of these models to emulate human-like text generation, offering innovative tools for literature review, hypothesis generation, experimental design, experimental subjects, data analysis, academic writing, and peer review in psychology. While LLMs are essential in advancing research methodologies in psychology, the paper also cautions about their technical and ethical challenges. There are issues like data privacy, the ethical implications of using LLMs in psychological research, and the need for a deeper understanding of these models' limitations. Researchers should responsibly use LLMs in psychological studies, adhering to ethical standards and considering the potential consequences of deploying these technologies in sensitive areas. Overall, the article provides a comprehensive overview of the current state of LLMs in psychology, exploring potential benefits and challenges. It serves as a call to action for researchers to leverage LLMs' advantages responsibly while addressing associated risks.

cs.LG↗

arXiv2025

ArtistAuditor: Auditing Artist Style Pirate in Text-to-Image Generation Models

Text-to-image models based on diffusion processes, such as DALL-E, Stable Diffusion, and Midjourney, are capable of transforming texts into detailed images and have widespread applications in art and design. As such, amateur users can easily imitate professional-level paintings by collecting an artist's work and fine-tuning the model, leading to concerns about artworks' copyright infringement. To tackle these issues, previous studies either add visually imperceptible perturbation to the artwork to change its underlying styles (perturbation-based methods) or embed post-training detectable watermarks in the artwork (watermark-based methods). However, when the artwork or the model has been published online, i.e., modification to the original artwork or model retraining is not feasible, these strategies might not be viable. To this end, we propose a novel method for data-use auditing in the text-to-image generation model. The general idea of ArtistAuditor is to identify if a suspicious model has been finetuned using the artworks of specific artists by analyzing the features related to the style. Concretely, ArtistAuditor employs a style extractor to obtain the multi-granularity style representations and treats artworks as samplings of an artist's style. Then, ArtistAuditor queries a trained discriminator to gain the auditing decisions. The experimental results on six combinations of models and datasets show that ArtistAuditor can achieve high AUC values (> 0.937). By studying ArtistAuditor's transferability and core modules, we provide valuable insights into the practical implementation. Finally, we demonstrate the effectiveness of ArtistAuditor in real-world cases by an online platform Scenario. ArtistAuditor is open-sourced at https://github.com/Jozenn/ArtistAuditor.

cs.CV↗

arXiv2025

Black hole thermodynamics from an ensemble-averaged theory

The path integral approach to a quantum theory of gravity is widely regarded as an indispensable strategy. However, determining what additional elements, beyond black hole or AdS spacetime, should be incorporated into the path integral remains crucial yet perplexing. We argue that the spacetime with a conical singularity in its Euclidean counterpart should be the most important ingredient to append to the path integral. Therefore, physical quantities should be ensemble-averaged over all geometries since they are described by the same Lorentzian metric. When the ensemble average is introduced, the Hawking-Page transition for the Schwarzschild-AdS black hole and the small-large black hole transition for the Reissner-Nordström-AdS black hole naturally arise as semi-classical approximations, when the size of the black hole system is much larger than the Planck length. Away from the semi-classical limit, the system is a superposition of different geometries, and the averaged quantities would deviate from the black hole thermodynamics. Expanding around the classical saddles, the subleading order of the Newton constant contributions can be derived, which are half of the Hawking temperature both for the Schwarzschild and Reissner-Nordström black holes. The result may imply a universal structure. The subsubleading terms and more intriguing physics that diverge from black hole thermodynamics are revealed. The ensemble-averaged theory provides a new way of studying subleading effects and extending the traditional AdS/CFT correspondence.

gr-qc↗