Searcharxiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.17655

textGrain watermarking is harder to detect: can sum-tests still do it optimally?

Swapnaneel Bhattacharyya·Soham Bonnerjee
View original source ↗Read source document ↗Export metadata ↓

Abstract

Watermarking embeds an imperceptible signal in text from large language models that is statistically detectable with a secret key. The recent textGrain watermark (Li et. al. (2026b)) calibrates this signal against the model's sampling entropy through an explicit entropy budget, combining the stochasticity of Red-Green and the unbiasedness of Gumbel-max watermarks, and its accompanying report proposes a detector that sums pivotal statistics. We study when, and how efficiently, textGrain can be detected after human edits, modeled as a sparse mixture in which a fraction $\varepsilon_n\asymp n^{-p}$ of positions retain the watermark and token uncertainty decays as $n^{-q}$. First, entropy calibration comes at a cost: the minimax detection boundary of textGrain is $p+q=1/2$, so its detectable region is strictly smaller than that of Gumbel-max and maximal-coupling watermarks, whose boundary is $2p+q=1$. This cost stems from textGrain's construction rather than from entropy calibration itself: an unbiased scheme with the same entropy loss remains detectable where textGrain is not. Second, perhaps surprisingly, this loss makes the sum detector optimal at the boundary when the number of transport columns is fixed, unlike for Gumbel-max. When the number of columns grows and the entropy budget does not vanish, the detectable region expands while that of the sum detector does not. Third, for a fixed edit fraction, we characterize the optimal exponential rate of type II error decay and show that the sum detector falls strictly short of it at every fixed alternative. Finally, we propose a weighted multiscale binomial scan, valid in finite samples and needing no knowledge of the next-token distributions, entropy budget, or edit fraction; it attains every detection boundary above and the optimal exponent.

Explore related subjects

stat.MLcs.ITcs.LGmath.ITmath.STstat.MEstat.TH

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Swapnaneel Bhattacharyya, Soham Bonnerjee. 2026-10-13. textGrain watermarking is harder to detect: can sum-tests still do it optimally?. https://arxiv.org/abs/2610.17655

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Bayesian Nonparametric Thompson Sampling for Multi-Armed Bandits

In stochastic multi-armed bandits, rewards are generated by an unknown distribution; the world's reward uncertainty induces the exploration-exploitation trade-off. Thompson sampling (TS) addresses this trade-off elegantly, but requires knowing the reward model's parametric form to sample from its posterior and compute per-arm expected rewards ---which limits its applicability as it demands per-scenario model specification and hyperparameter tuning. We here adopt Bayesian nonparametric (BNP) mixture models to remove this requirement, extending TS to arbitrary reward distributions. We model each arm's unknown reward distribution with an independent BNP Gaussian mixture, which can approximate any continuous reward density, and adapts its complexity to sequentially observed data. Embedding the BNP mixture model into TS yields Nonparametric TS, an algorithm that sequentially learns the reward model that best approximates the true, yet unknown, per-arm reward distribution, avoiding case-by-case model specification and hyperparameter tuning. We empirically demonstrate that ($i$) Nonparametric TS matches the regret performance of Oracle TS policies that know the true reward model; ($ii$) it favorably compares (in cumulative regret and volatility) to a suite of state-of-the-art alternatives across multimodal, outlier-contaminated, and non-exponential-family reward distributions; and ($iii$) its Bayesian nonparametric inference remains computationally practical and accurate for regret minimization.

stat.ML↗
arXiv2026

OSCAR: Order-aware Sensitivity Correction for AI Rankings

LLM judges evaluate candidate language models through pairwise comparisons of their responses. Judge-aware Bradley--Terry models aggregate pairwise comparisons by jointly estimating candidate scores and judge-specific sensitivities (how strongly each judge responds to quality differences), often interpreted as reliability. We show that this reading fails when judges prefer a display position. Even under balanced display orders or swap-and-average, omitting position attenuates fitted judge sensitivity by approximately the factor $1-D^2$ near equal quality, where $D$ is the judge's net first-slot preference between equally good responses. We propose OSCAR, which jointly estimates candidate scores, judge sensitivities and judge-specific position, length and family effects, and provides confidence intervals for pairwise win probabilities that account for dependence within prompts. On held-out data, observed attenuation closely matches the prediction $1-D^2$ (correlation of log ratios 0.99). Across four datasets, position adjustment lowers held-out log loss by 14-23% and changes which judges would be dropped for low sensitivity on three datasets. On MT-Bench, it ranks GPT-4 first instead of claude-v1, matching human votes. Simulations with known parameters show improved recovery of judge sensitivities and candidate rankings, with 94.4-95.2% confidence interval coverage. Low estimated sensitivity alone is therefore not evidence of an unreliable judge.

stat.ML↗
arXiv2026

PICPIs: Prediction-Interval-Conditional Prediction Intervals

A classical question in statistics is which observable quantities to condition on when drawing inferences about unobservable targets. For conformal prediction in nonparametric uncertainty quantification, standard marginal validity offers limited resolution at the prediction values on which decisions are based, and fully conditional guarantees with respect to the covariates are provably unattainable. We address this gap by introducing a prediction-based conditioning framework that we refer to as Prediction-Interval-Conditional Prediction Intervals (PICPIs). Formally, a PICPI is an interval $I$ satisfying a self-consistency condition: $$\mathbb{E} [Y \mid p(X) \in I] \in I,$$ for predictive model $p$, contextual covariate $X$, and outcome $Y$. Thus, an interval simultaneously defines a stratum of prediction values and certifies that the mean outcome in that stratum lies in the same interval. This self-consistency condition yields data-adaptive strata without altering the original prediction. Such intervals can be constructed using practical algorithms. Under regularity of the prediction distribution, the constructed intervals cover all but an arbitrarily small fraction of prediction values and have widths that decrease at rate $n^{-1/3}$, up to logarithmic factors and the prediction error. Moreover, identifying these locally calibrated intervals can, in turn, inform downstream decision-making. We derive inference procedures for PICPIs in probabilistic prediction and multi-class classification, accompanied by theoretical guarantees. Empirical results are provided that compare PICPIs with existing interval-based baselines.

stat.ML↗