Searcharxiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.23570

A general recipe for data-adaptive anytime-valid envelopes for iterative algorithms

Soham Bonnerjee·Aaditya Ramdas
View original source ↗Read source document ↗Export metadata ↓

Abstract

Most convergence guarantees for iterative stochastic algorithms are stated at a fixed iteration count and do not justify stopping at a data-dependent time. Recent work has constructed anytime-valid envelopes that do, but deterministic envelopes are typically too conservative for practical stopping, and existing data-adaptive ones are derived case-by-case, each coupling a specific algorithm to a specific noise model and boundary construction. This paper provides a general recipe for constructing such anytime-valid data-adaptive boundaries for a large class of iterative algorithms. Performance measures of many of these algorithms satisfy a one-step recursion that reduces the entire error analysis to a single general requirement: time-uniform concentration for predictably weighted sums of noise. We formalize this through a strongly adapted sub-$ψ$ condition, and derive off-the-shelf boundary constructions via continuous and discrete mixtures, stitching, and product betting martingales. We show how these boundaries can be directly coupled with a separate algorithmic analysis that derives the one-step recursion to provide a class of readymade anytime-valid envelopes across diverse settings including stochastic gradient descent (SGD), temporal difference learning, and Bures-Wasserstein SGD. In particular, for SGD, our framework either recovers or significantly sharpens every existing envelope we are aware of. Our constructions come with explicit implementation details and are substantiated by numerical experiments.

Explore related subjects

stat.MLcs.LGmath.OCstat.ME

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Soham Bonnerjee, Aaditya Ramdas. 2026-10-16. A general recipe for data-adaptive anytime-valid envelopes for iterative algorithms. https://arxiv.org/abs/2610.23570

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Deep conditional distribution learning via conditional Föllmer flow

We introduce an ordinary differential equation (ODE) based deep generative method for learning conditional distributions, named Conditional Föllmer Flow. Starting from a standard Gaussian distribution, the proposed flow could approximate the target conditional distribution very well when the time is close to 1. For effective implementation, we discretize the flow with Euler's method where we estimate the velocity field nonparametrically using a deep neural network. Furthermore, we also establish the convergence result for the Wasserstein-2 distance between the distribution of the learned samples and the target conditional distribution, providing the first comprehensive end-to-end error analysis for conditional distribution learning via ODE flow. Our numerical experiments showcase its effectiveness across a range of scenarios, from standard nonparametric conditional density estimation problems to more intricate challenges involving image data, illustrating its superiority over various existing conditional density estimation methods.

stat.ML↗
arXiv2026

What is causal about causal models and representations?

Causal Bayesian networks are often described as causal because they support reasoning about interventions. Yet interventions are defined in the mathematical model, whereas the resulting model predictions concern the effects of actions in the real world. Consequently, whether and how a causal model makes falsifiable predictions about the effects of actions depends on how actions are interpreted as interventions. This requires specifying which actions correspond to which interventions, a step that is frequently left implicit. We develop a formal framework that separates a causal model from the data-generating process it aims to describe and formalizes interpretations connecting actions to interventions. Within this framework, we prove an impossibility result: no interpretation can simultaneously satisfy a collection of desirable properties while remaining non-circular. Under a circular interpretation, every model that fits the observational distribution is interventionally valid, so no experiment can falsify it. We analyze several non-circular alternatives and discuss implications for falsification, causal discovery, causal representation learning, and causal abstraction. More broadly, our framework provides a way to make assumptions about the connections between actions and interventions explicit and to reason formally about the empirical content of causal models.

stat.ML↗
arXiv2026

Optimal estimates for the $\ell^p$-Lipschitz constants of deep random ReLU neural networks

This paper studies the $\ell^p$-Lipschitz constants of ReLU neural networks $Φ: \mathbb{R}^d \to \mathbb{R}$ with random parameters for $p \in [1,\infty]$. The distribution of the weights follows a variant of the He initialization. In the case of zero-bias networks, we derive matching high-probability upper and lower bounds for wide networks that differ at most by an absolute constant. The bounds reveal two remarkable phenomena. First, the bounds grow root-logarithmically in terms of the depth of the net. Second, the behavior of the $\ell^p$-Lipschitz constants varies significantly between the regimes $p \in [1,2)$ and $p \in [2,\infty]$. For $p \in [2,\infty]$, the $\ell^p$-Lipschitz constant behaves similarly to $\|g\|_{p'}$, where $g \in \mathbb{R}^d$ is a standard Gaussian vector and $1/p + 1/p' = 1$. In contrast, for $p \in [1,2)$, the $\ell^p$-Lipschitz constant aligns more closely with $\|g\|_2$. For networks with possibly nonzero biases, we obtain the same lower bounds without assumptions on the bias distributions. For symmetric bias distributions with bounded densities, we also derive high-probability upper bounds for the $\ell^p$-Lipschitz constants. For fixed sub-exponential bias distributions, these upper bounds differ from the lower bounds by a factor that has square-root dependence on the depth and logarithmic dependence on the width.

stat.ML↗