arXiv subjects

Guanxun Li

Publications and source records attributed to Guanxun Li.

6 recordsLinked to original sources

arXiv2025

Adaptive Testing for Segmenting Watermarked Texts From Language Models

The rapid adoption of large language models (LLMs), such as GPT-4 and Claude 3.5, underscores the need to distinguish LLM-generated text from human-written content to mitigate the spread of misinformation and misuse in education. One promising approach to address this issue is the watermark technique, which embeds subtle statistical signals into LLM-generated text to enable reliable identification. In this paper, we first generalize the likelihood-based LLM detection method of a previous study by introducing a flexible weighted formulation, and further adapt this approach to the inverse transform sampling method. Moving beyond watermark detection, we extend this adaptive detection strategy to tackle the more challenging problem of segmenting a given text into watermarked and non-watermarked substrings. In contrast to the approach in a previous study, which relies on accurate estimation of next-token probabilities that are highly sensitive to prompt estimation, our proposed framework removes the need for precise prompt estimation. Extensive numerical experiments demonstrate that the proposed methodology is both effective and robust in accurately segmenting texts containing a mixture of watermarked and non-watermarked content.

stat.ML↗

arXiv2025

A General Framework for Multiple Testing via E-value Aggregation and Data-Dependent Weighting

Motivated by recent findings in Li and Zhang (2025), which established an equivalence between certain p-value-based multiple testing procedures and the e-Benjamini-Hochberg procedure (Wang and Ramdas, 2022), we introduce a general framework for constructing novel multiple testing methods through the aggregation and combination of e-values. Specifically, we propose methodologies for three distinct scenarios: (i) assembly of e-values obtained from different subsets of data, simultaneously controlling group-wise and overall false discovery rates; (ii) aggregation of e-values derived from different procedures or the same procedure employing different test statistics; and (iii) adaptive multiple testing methods that incorporate external structural information to enhance statistical power. A notable feature of our approach is the use of data-dependent weighting of e-values, significantly improving the efficiency of the resulting e-Benjamini-Hochberg procedures. The construction of these weights is non-trivial and inspired by leave-one-out analysis, a widely utilized technique for proving false discovery rate control in p-value-based methodologies. We theoretically establish that the proposed e-Benjamini-Hochberg procedures, when equipped with data-dependent weights, guarantee finite-sample false discovery rate control across all three considered applications. Additionally, numerical studies illustrate the efficacy and advantages of the proposed methods within each application scenario.

stat.ME↗

arXiv2024

Segmenting Watermarked Texts From Language Models

Watermarking is a technique that involves embedding nearly unnoticeable statistical signals within generated content to help trace its source. This work focuses on a scenario where an untrusted third-party user sends prompts to a trusted language model (LLM) provider, who then generates a text from their LLM with a watermark. This setup makes it possible for a detector to later identify the source of the text if the user publishes it. The user can modify the generated text by substitutions, insertions, or deletions. Our objective is to develop a statistical method to detect if a published text is LLM-generated from the perspective of a detector. We further propose a methodology to segment the published text into watermarked and non-watermarked sub-strings. The proposed approach is built upon randomization tests and change point detection techniques. We demonstrate that our method ensures Type I and Type II error control and can accurately identify watermarked sub-strings by finding the corresponding change point locations. To validate our technique, we apply it to texts generated by several language models with prompts extracted from Google's C4 dataset and obtain encouraging numerical results. We release all code publicly at https://github.com/doccstat/llm-watermark-cpd.

cs.LG↗

arXiv2024

Importance is Important: Generalized Markov Chain Importance Sampling Methods

We show that for any multiple-try Metropolis algorithm, one can always accept the proposal and evaluate the importance weight that is needed to correct for the bias without extra computational cost. This results in a general, convenient, and rejection-free Markov chain Monte Carlo (MCMC) sampling scheme. By further leveraging the importance sampling perspective on Metropolis--Hastings algorithms, we propose an alternative MCMC sampler on discrete spaces that is also outside the Metropolis--Hastings framework, along with a general theory on its complexity. Numerical examples suggest that the proposed algorithms are consistently more efficient than the original Metropolis--Hastings versions.

stat.CO↗

arXiv2023

Bayesian Multi-task Variable Selection with an Application to Differential DAG Analysis

In this paper, we study the Bayesian multi-task variable selection problem, where the goal is to select activated variables for multiple related data sets simultaneously. Our proposed method generalizes the spike-and-slab prior to multiple data sets, and we prove its posterior consistency in high-dimensional regimes. To calculate the posterior distribution, we propose a novel variational Bayes algorithm based on the recently developed "sum of single effects" model of Wang et al. (2020). Finally, motivated by differential gene network analysis in biology, we extend our method to joint learning of multiple directed acyclic graphical models. Both simulation studies and real gene expression data analysis are conducted to show the effectiveness of the proposed method.

stat.ME↗

arXiv2023

Fast Replica Exchange Stochastic Gradient Langevin Dynamics

Application of the replica exchange (i.e., parallel tempering) technique to Langevin Monte Carlo algorithms, especially stochastic gradient Langevin dynamics (SGLD), has scored great success in non-convex learning problems, but one potential limitation is the computational cost caused by running multiple chains. Upon observing that a large variance of the gradient estimator in SGLD essentially increases the temperature of the stationary distribution, we propose expediting tempering schemes for SGLD by directly estimating the bias caused by the stochastic gradient estimator. This simple idea enables us to simulate high-temperature chains at a negligible computational cost (compared to that of the low-temperature chain) while preserving the convergence to the target distribution. Our method is fundamentally different from the recently proposed m-reSGLD (multi-variance replica exchange SGLD) method in that the latter suffers from the low accuracy of the gradient estimator (e.g., the chain can fail to converge to the target) while our method benefits from it. Further, we derive a swapping rate that can be easily evaluated, providing another significant improvement over m-reSGLD. To theoretically demonstrate the advantage of our method, we develop convergence bounds in Wasserstein distances. Numerical examples for Gaussian mixture and inverse PDE models are also provided, which show that our method can converge quicker than the vanilla multi-variance replica exchange method.

math.NA↗