arXiv subjects

Peng Ding

Publications and source records attributed to Peng Ding.

At least 55 records · Page 3Linked to original sources

arXiv2025

Unified Tool Integration for LLMs: A Protocol-Agnostic Approach to Function Calling

The proliferation of tool-augmented Large Language Models (LLMs) has created a fragmented ecosystem where developers must navigate multiple protocols, manual schema definitions, and complex execution workflows. We address this challenge by proposing a unified approach to tool integration that abstracts protocol differences while optimizing execution performance. Our solution demonstrates how protocol-agnostic design principles can significantly reduce development overhead through automated schema generation, dual-mode concurrent execution, and seamless multi-source tool management. Experimental results show 60-80% code reduction across integration scenarios, performance improvements up to 3.1x through optimized concurrency, and full compatibility with existing function calling standards. This work contributes both theoretical insights into tool integration architecture and practical solutions for real-world LLM application development.

cs.AI↗

arXiv2025

Linear Model and Extensions

I developed the lecture notes based on my ``Linear Model'' course at the University of California, Berkeley over the past ten years. This book provides an intermediate-level introduction to the linear model. It balances rigorous proofs and heuristic arguments. This book provides R code to replicate all simulation studies and case studies.

stat.ME↗

arXiv2025

Berry-Esseen bounds for design-based causal inference with possibly diverging treatment levels and varying group sizes

Neyman (1923/1990) introduced the randomization model, which contains the notation of potential outcomes to define causal effects and a framework for large-sample inference based on the design of the experiment. However, the existing theory for this framework is far from complete, especially when the number of treatment levels diverges and the treatment group sizes vary. We provide a unified discussion of statistical inference under the randomization model with general treatment group sizes. We formulate the estimator in terms of a linear permutation statistic and use results based on Stein's method to derive various Berry--Esseen bounds on the linear and quadratic functions of the estimator. These new Berry--Esseen bounds serve as the basis for design-based causal inference with possibly diverging treatment levels and a diverging number of causal parameters of interest. We also fill an important gap by proposing novel variance estimators for experiments with possibly many treatment levels without replications. Equipped with the newly developed results, design-based causal inference in general settings becomes more convenient with stronger theoretical guarantees.

stat.ME↗

arXiv2025

Causal inference in network experiments: regression-based analysis and design-based properties

Network experiments are powerful tools for studying spillover effects, which avoid endogeneity by randomly assigning treatments to units over networks. However, it is non-trivial to analyze network experiments properly without imposing strong modeling assumptions. We show that regression-based point estimators and standard errors can have strong theoretical guarantees if the regression functions and robust standard errors are carefully specified to accommodate the interference patterns under network experiments. We first recall a well-known result that the Hájek estimator is numerically identical to the coefficient from the weighted-least-squares fit based on the inverse probability of the exposure mapping. Moreover, we demonstrate that the regression-based approach offers three notable advantages: its ease of implementation, the ability to derive standard errors through the same regression fit, and the potential to integrate covariates into the analysis to improve efficiency. Recognizing that the regression-based network-robust covariance estimator can be anti-conservative under nonconstant effects, we propose an adjusted covariance estimator to improve the empirical coverage rates.

econ.EM↗

arXiv2025

Why Not Act on What You Know? Unleashing Safety Potential of LLMs via Self-Aware Guard Enhancement

Large Language Models (LLMs) have shown impressive capabilities across various tasks but remain vulnerable to meticulously crafted jailbreak attacks. In this paper, we identify a critical safety gap: while LLMs are adept at detecting jailbreak prompts, they often produce unsafe responses when directly processing these inputs. Inspired by this insight, we propose SAGE (Self-Aware Guard Enhancement), a training-free defense strategy designed to align LLMs' strong safety discrimination performance with their relatively weaker safety generation ability. SAGE consists of two core components: a Discriminative Analysis Module and a Discriminative Response Module, enhancing resilience against sophisticated jailbreak attempts through flexible safety discrimination instructions. Extensive experiments demonstrate SAGE's effectiveness and robustness across various open-source and closed-source LLMs of different sizes and architectures, achieving an average 99% defense success rate against numerous complex and covert jailbreak methods while maintaining helpfulness on general benchmarks. We further conduct mechanistic interpretability analysis through hidden states and attention distributions, revealing the underlying mechanisms of this detection-generation discrepancy. Our work thus contributes to developing future LLMs with coherent safety awareness and generation behavior. Our code and datasets are publicly available at https://github.com/NJUNLP/SAGE.

cs.CL↗

arXiv2025

A novel class of arbitrary high-order numerical schemes for fractional differential equations

A novel efficient and high accuracy numerical method for the time-fractional differential equations (TFDEs) is proposed in this work. We show the equivalence between TFDEs and the integer-order extended parametric differential equations (EPDE) by dimensional expanding, and establish the stability of EPDE. We apply BDF-$k$ formula for the temporal discretization, while we use the Jacobi spectral collocation method for the discretization of the extended direction. We analyze the stability of the proposed method and give rigorous error estimates with order $O(Δt^{k} + M^{-m})$, where $Δt$ and $M$ are time step size and number of collocation nodes in extended direction, respectively. Also, we point out that the computational cost and the storage requirement is essentially the same as the integer problems, namely, the computational cost and the storage of the present algorithm are $O(N)$ and $O(1)$, respectively, where $N$ is the total number of time step. We present several numerical examples, including both linear and nonlinear problems, to demonstrate the effectiveness of the proposed method and to validate the theoretical results

math.NA↗

arXiv2025

Design-based causal inference in bipartite experiments

Bipartite experiments arise in various fields, in which the treatments are randomized over one set of units, while the outcomes are measured over another separate set of units. However, existing methods often rely on strong model assumptions about the data-generating process. Under the potential outcomes formulation, we explore design-based causal inference in bipartite experiments under weak assumptions by leveraging the sparsity structure of the bipartite graph that connects the treatment units and outcome units. We make several contributions. First, we formulate the causal inference problem under the design-based framework that can account for the bipartite interference. Second, we propose a consistent point estimator for the total treatment effect, a policy-relevant parameter that measures the difference in the outcome means if all treatment units receive the treatment or control. Third, we establish a central limit theorem for the estimator and propose a conservative variance estimator for statistical inference. Fourth, we discuss a covariate adjustment strategy to enhance estimation efficiency.

stat.ME↗

arXiv2025

Interpretable sensitivity analysis for the Baron-Kenny approach to mediation with unmeasured confounding

Mediation analysis assesses the extent to which the exposure affects the outcome indirectly through a mediator and the extent to which it operates directly through other pathways. The popular Baron-Kenny approach estimates the indirect and direct effects of the exposure on the outcome based on linear regressions. However, when the exposure and the mediator are not randomized, the estimates may be biased due to unmeasured confounding. We first derive general omitted-variable bias formulas in linear regressions with vector responses and regressors. We then use the formulas to develop a sensitivity analysis method for the Baron-Kenny approach in the presence of unmeasured confounding. To ensure interpretability, we express the sensitivity parameters to correspond to the natural factorization of the joint distribution of the direct acyclic graph. They measure the partial correlation between the unmeasured confounder and the exposure, mediator, and outcome, respectively. We further propose a novel measure called the "robustness value for mediation" or simply the "robustness value", to assess the robustness of results based on the Baron-Kenny approach with respect to unmeasured confounding. Intuitively, the robustness value measures the minimum value of the maximum proportion of variability explained by the unmeasured confounding, for the exposure, mediator, and outcome, to overturn the results of the direct and indirect effect estimates. Importantly, we prove that all our sensitivity bounds are attainable and thus sharp.

stat.ME↗

arXiv2025

Entropy-Reinforced Planning with Large Language Models for Drug Discovery

The objective of drug discovery is to identify chemical compounds that possess specific pharmaceutical properties toward a binding target. Existing large language models (LLMS) can achieve high token matching scores in terms of likelihood for molecule generation. However, relying solely on LLM decoding often results in the generation of molecules that are either invalid due to a single misused token, or suboptimal due to unbalanced exploration and exploitation as a consequence of the LLMs prior experience. Here we propose ERP, Entropy-Reinforced Planning for Transformer Decoding, which employs an entropy-reinforced planning algorithm to enhance the Transformer decoding process and strike a balance between exploitation and exploration. ERP aims to achieve improvements in multiple properties compared to direct sampling from the Transformer. We evaluated ERP on the SARS-CoV-2 virus (3CLPro) and human cancer cell target protein (RTCB) benchmarks and demonstrated that, in both benchmarks, ERP consistently outperforms the current state-of-the-art algorithm by 1-5 percent, and baselines by 5-10 percent, respectively. Moreover, such improvement is robust across Transformer models trained with different objectives. Finally, to further illustrate the capabilities of ERP, we tested our algorithm on three code generation benchmarks and outperformed the current state-of-the-art approach as well. Our code is publicly available at: https://github.com/xuefeng-cs/ERP.

cs.LG↗

arXiv2025

Asymptotic theory of the quadratic assignment procedure for dyadic data analysis

The quadratic assignment procedure (QAP) is a popular tool for analyzing dyadic data in medical and social sciences. To test the association between two dyadic measurements represented by two symmetric matrices, QAP calculates the p-value by permuting the units, or equivalently, by permuting the rows and columns of one matrix in the same way. Its extension to the regression setting, known as the multiple regression QAP, has also gained popularity, especially in psychometrics. However, the statistics theory for QAP has not been fully established in the literature. We fill the gap in this paper. We formulate the network models underlying various QAPs. We derive (a) the asymptotic sampling distributions of some canonical test statistics and (b) the corresponding asymptotic permutation distributions induced by QAP under strong and weak null hypotheses. Task (a) relies on applying the theory of U-statistics, and task (b) relies on applying the theory of double-indexed permutation statistics. The combination of tasks (a) and (b) provides a relatively complete picture of QAP. Overall, our asymptotic theory suggests that using properly studentized statistics in QAP is a robust choice in that it is finite-sample exact under the strong null hypothesis and preserves the asymptotic type one error rate under the weak null hypothesis.

stat.ME↗

arXiv2025

Two-phase rejective sampling and its asymptotic properties

Rejective sampling improves design and estimation efficiency of single-phase sampling when auxiliary information in a finite population is available. When such auxiliary information is unavailable, we propose to use two-phase rejective sampling (TPRS), which involves measuring auxiliary variables for the sample of units in the first phase, followed by the implementation of rejective sampling for the outcome in the second phase. We explore the asymptotic design properties of double expansion and regression estimators under TPRS. We show that TPRS enhances the efficiency of the double expansion estimator, rendering it comparable to a regression estimator. We further refine the design to accommodate varying importance of covariates and extend it to multi-phase sampling. We start with the theory for the population mean and then extend the theory to parameters defined by general estimating equations. Our asymptotic results for TPRS immediately cover the existing single-phase rejective sampling, under which the asymptotic theory has not been fully established.

stat.ME↗

arXiv2025

A stronger Sylvester's criterion for positive semidefinite matrices

Sylvester's criterion characterizes positive definite (PD) and positive semidefinite (PSD) matrices without the need of eigendecomposition. It states that a symmetric matrix is PD if and only if all of its leading principal minors are positive, and a symmetric matrix is PSD if and only if all of its principal minors are nonnegative. For an $m\times m$ symmetric matrix, Sylvester's criterion requires computing $m$ and $2^m-1$ determinants to verify it is PD and PSD, respectively. Therefore, it is less useful for PSD matrices due to the exponential growth in the number of principal submatrices as the matrix dimension increases. We provide a stronger Sylvester's criterion for PSD matrices which only requires to verify the nonnegativity of $m(m+1)/2$ determinants. Based on the new criterion, we provide a method to derive elementwise criteria for PD and PSD matrices. We illustrate the applications of our results in PD or PSD matrix completion and highlight their statistics applications via nonlinear semidefinite program.

math.RA↗

arXiv2024

When is it worthwhile to jackknife? Breaking the quadratic barrier for Z-estimators

Resampling methods are especially well-suited to inference with estimators that provide only "black-box'' access. Jackknife is a form of resampling, widely used for bias correction and variance estimation, that is well-understood under classical scaling where the sample size $n$ grows for a fixed problem. We study its behavior in application to estimating functionals using high-dimensional $Z$-estimators, allowing both the sample size $n$ and problem dimension $d$ to diverge. We begin showing that the plug-in estimator based on the $Z$-estimate suffers from a quadratic breakdown: while it is $\sqrt{n}$-consistent and asymptotically normal whenever $n \gtrsim d^2$, it fails for a broad class of problems whenever $n \lesssim d^2$. We then show that under suitable regularity conditions, applying a jackknife correction yields an estimate that is $\sqrt{n}$-consistent and asymptotically normal whenever $n\gtrsim d^{3/2}$. This provides strong motivation for the use of jackknife in high-dimensional problems where the dimension is moderate relative to sample size. We illustrate consequences of our general theory for various specific $Z$-estimators, including non-linear functionals in linear models; generalized linear models; and the inverse propensity score weighting (IPW) estimate for the average treatment effect, among others.

math.ST↗

arXiv2024

Identifying and bounding the probability of necessity for causes of effects with ordinal outcomes

Although the existing causal inference literature focuses on the forward-looking perspective by estimating effects of causes, the backward-looking perspective can provide insights into causes of effects. In backward-looking causal inference, the probability of necessity measures the probability that a certain event is caused by the treatment given the observed treatment and outcome. Most existing results focus on binary outcomes. Motivated by applications with ordinal outcomes, we propose a general definition of the probability of necessity. However, identifying the probability of necessity is challenging because it involves the joint distribution of the potential outcomes. We propose a novel assumption of monotonic incremental treatment effect to identify the probability of necessity with ordinal outcomes. We also discuss the testable implications of this key identification assumption. When it fails, we derive explicit formulas of the sharp large-sample bounds on the probability of necessity.

math.ST↗

arXiv2024

With random regressors, least squares inference is robust to correlated errors with unknown correlation structure

Linear regression is arguably the most widely used statistical method. With fixed regressors and correlated errors, the conventional wisdom is to modify the variance-covariance estimator to accommodate the known correlation structure of the errors. We depart from the literature by showing that with random regressors, linear regression inference is robust to correlated errors with unknown correlation structure. The existing theoretical analyses for linear regression are no longer valid because even the asymptotic normality of the least-squares coefficients breaks down in this regime. We first prove the asymptotic normality of the t statistics by establishing their Berry-Esseen bounds based on a novel probabilistic analysis of self-normalized statistics. We then study the local power of the corresponding t tests and show that, perhaps surprisingly, error correlation can even enhance power in the regime of weak signals. Overall, our results show that linear regression is applicable more broadly than the conventional theory suggests, and further demonstrate the value of randomization to ensure robustness of inference.

math.ST↗

arXiv2024

MANGO: A Benchmark for Evaluating Mapping and Navigation Abilities of Large Language Models

Large language models such as ChatGPT and GPT-4 have recently achieved astonishing performance on a variety of natural language processing tasks. In this paper, we propose MANGO, a benchmark to evaluate their capabilities to perform text-based mapping and navigation. Our benchmark includes 53 mazes taken from a suite of textgames: each maze is paired with a walkthrough that visits every location but does not cover all possible paths. The task is question-answering: for each maze, a large language model reads the walkthrough and answers hundreds of mapping and navigation questions such as "How should you go to Attic from West of House?" and "Where are we if we go north and east from Cellar?". Although these questions are easy to humans, it turns out that even GPT-4, the best-to-date language model, performs poorly at answering them. Further, our experiments suggest that a strong mapping and navigation ability would benefit large language models in performing relevant downstream tasks, such as playing textgames. Our MANGO benchmark will facilitate future research on methods that improve the mapping and navigation capabilities of language models. We host our leaderboard, data, code, and evaluation program at https://mango.ttic.edu and https://github.com/oaklight/mango/.

cs.CL↗

arXiv2024

Hallu-PI: Evaluating Hallucination in Multi-modal Large Language Models within Perturbed Inputs

Multi-modal Large Language Models (MLLMs) have demonstrated remarkable performance on various visual-language understanding and generation tasks. However, MLLMs occasionally generate content inconsistent with the given images, which is known as "hallucination". Prior works primarily center on evaluating hallucination using standard, unperturbed benchmarks, which overlook the prevalent occurrence of perturbed inputs in real-world scenarios-such as image cropping or blurring-that are critical for a comprehensive assessment of MLLMs' hallucination. In this paper, to bridge this gap, we propose Hallu-PI, the first benchmark designed to evaluate Hallucination in MLLMs within Perturbed Inputs. Specifically, Hallu-PI consists of seven perturbed scenarios, containing 1,260 perturbed images from 11 object types. Each image is accompanied by detailed annotations, which include fine-grained hallucination types, such as existence, attribute, and relation. We equip these annotations with a rich set of questions, making Hallu-PI suitable for both discriminative and generative tasks. Extensive experiments on 12 mainstream MLLMs, such as GPT-4V and Gemini-Pro Vision, demonstrate that these models exhibit significant hallucinations on Hallu-PI, which is not observed in unperturbed scenarios. Furthermore, our research reveals a severe bias in MLLMs' ability to handle different types of hallucinations. We also design two baselines specifically for perturbed scenarios, namely Perturbed-Reminder and Perturbed-ICL. We hope that our study will bring researchers' attention to the limitations of MLLMs when dealing with perturbed inputs, and spur further investigations to address this issue. Our code and datasets are publicly available at https://github.com/NJUNLP/Hallu-PI.

cs.CV↗

arXiv2024

A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily

Large Language Models (LLMs), such as ChatGPT and GPT-4, are designed to provide useful and safe responses. However, adversarial prompts known as 'jailbreaks' can circumvent safeguards, leading LLMs to generate potentially harmful content. Exploring jailbreak prompts can help to better reveal the weaknesses of LLMs and further steer us to secure them. Unfortunately, existing jailbreak methods either suffer from intricate manual design or require optimization on other white-box models, which compromises either generalization or efficiency. In this paper, we generalize jailbreak prompt attacks into two aspects: (1) Prompt Rewriting and (2) Scenario Nesting. Based on this, we propose ReNeLLM, an automatic framework that leverages LLMs themselves to generate effective jailbreak prompts. Extensive experiments demonstrate that ReNeLLM significantly improves the attack success rate while greatly reducing the time cost compared to existing baselines. Our study also reveals the inadequacy of current defense methods in safeguarding LLMs. Finally, we analyze the failure of LLMs defense from the perspective of prompt execution priority, and propose corresponding defense strategies. We hope that our research can catalyze both the academic community and LLMs developers towards the provision of safer and more regulated LLMs. The code is available at https://github.com/NJUNLP/ReNeLLM.

cs.CL↗