arXiv subjects

Adrien Vandenbroucque

Publications and source records attributed to Adrien Vandenbroucque.

7 recordsLinked to original sources

arXiv2026

A Finite-Sample Strong Converse for Binary Hypothesis Testing via (Reverse) Rényi Divergence

This work investigates binary hypothesis testing between $H_0\sim P_0$ and $H_1\sim P_1$ in the finite-sample regime under asymmetric error constraints. By employing the ``reverse" Rényi divergence, we derive novel non-asymptotic bounds on the Type II error probability which naturally establish a strong converse result. Furthermore, when the Type I error is constrained to decay exponentially with a rate $c$, we show that the Type II error converges to 1 exponentially fast if $c$ exceeds the Kullback-Leibler divergence $D(P_1\|P_0)$, and vanishes exponentially fast if $c$ is smaller. Finally, we present numerical examples demonstrating that the proposed converse bounds strictly improve upon existing finite-sample results in the literature.

cs.IT↗

arXiv2026

Contraction of Rényi Divergences for Discrete Channels: Properties and Applications

This work explores properties of Strong Data-Processing constants for Rényi Divergences. Parallels are made with the well-studied $φ$-Divergences, and it is shown that the order $α$ of Rényi Divergences dictates whether certain properties of the contraction of $φ$-Divergences are mirrored or not. In particular, we demonstrate that when $α>1$, the contraction properties can deviate quite strikingly from those of $φ$-Divergences. We also uncover specific characteristics of contraction for the $\infty$-Rényi Divergence and relate it to $\varepsilon$-Local Differential Privacy. The results are then applied to bound the speed of convergence of Markov chains, where we argue that the contraction of Rényi Divergences offers a new perspective on the contraction of $L^α$-norms commonly studied in the literature.

cs.IT↗

arXiv2024

Analysis of Bootstrap and Subsampling in High-dimensional Regularized Regression

We investigate popular resampling methods for estimating the uncertainty of statistical models, such as subsampling, bootstrap and the jackknife, and their performance in high-dimensional supervised regression tasks. We provide a tight asymptotic description of the biases and variances estimated by these methods in the context of generalized linear models, such as ridge and logistic regression, taking the limit where the number of samples $n$ and dimension $d$ of the covariates grow at a comparable fixed rate $α\!=\! n/d$. Our findings are three-fold: i) resampling methods are fraught with problems in high dimensions and exhibit the double-descent-like behavior typical of these situations; ii) only when $α$ is large enough do they provide consistent and reliable error estimations (we give convergence rates); iii) in the over-parametrized regime $α\!<\!1$ relevant to modern machine learning practice, their predictions are not consistent, even with optimal regularization.

stat.ML↗

arXiv2023

Lower Bounds on the Bayesian Risk via Information Measures

This paper focuses on parameter estimation and introduces a new method for lower bounding the Bayesian risk. The method allows for the use of virtually \emph{any} information measure, including Rényi's $α$, $φ$-Divergences, and Sibson's $α$-Mutual Information. The approach considers divergences as functionals of measures and exploits the duality between spaces of measures and spaces of functions. In particular, we show that one can lower bound the risk with any information measure by upper bounding its dual via Markov's inequality. We are thus able to provide estimator-independent impossibility results thanks to the Data-Processing Inequalities that divergences satisfy. The results are then applied to settings of interest involving both discrete and continuous parameters, including the ``Hide-and-Seek'' problem, and compared to the state-of-the-art techniques. An important observation is that the behaviour of the lower bound in the number of samples is influenced by the choice of the information measure. We leverage this by introducing a new divergence inspired by the ``Hockey-Stick'' Divergence, which is demonstrated empirically to provide the largest lower-bound across all considered settings. If the observations are subject to privatisation, stronger impossibility results can be obtained via Strong Data-Processing Inequalities. The paper also discusses some generalisations and alternative directions.

cs.IT↗

arXiv2022

The Houdayer Algorithm: Overview, Extensions, and Applications

The study of spin systems with disorder and frustration is known to be a computationally hard task. Standard heuristics developed for optimizing and sampling from general Ising Hamiltonians tend to produce correlated solutions due to their locality, resulting in a suboptimal exploration of the search space. To mitigate these effects, cluster Monte-Carlo methods are often employed as they provide ways to perform non-local transformations on the system. In this work, we investigate the Houdayer algorithm, a cluster Monte-Carlo method with small numerical overhead which improves the exploration of configurations by preserving the energy of the system. We propose a generalization capable of reaching exponentially many configurations at the same energy, while offering a high level of adaptability to ensure that no biased choice is made. We discuss its applicability in various contexts, including Markov chain Monte-Carlo sampling and as part of a genetic algorithm. The performance of our generalization in these settings is illustrated by sampling for the Ising model across different graph connectivities and by solving instances of well-known binary optimization problems. We expect our results to be of theoretical and practical relevance in the study of spin glasses but also more broadly in discrete optimization, where a multitude of problems follow the structure of Ising spin systems.

cond-mat.dis-nn↗

arXiv2022

Lower-bounds on the Bayesian Risk in Estimation Procedures via $f$-Divergences

We consider the problem of parameter estimation in a Bayesian setting and propose a general lower-bound that includes part of the family of $f$-Divergences. The results are then applied to specific settings of interest and compared to other notable results in the literature. In particular, we show that the known bounds using Mutual Information can be improved by using, for example, Maximal Leakage, Hellinger divergence, or generalizations of the Hockey-Stick divergence.

cs.IT↗

arXiv2022

On Sibson's $α$-Mutual Information

We explore a family of information measures that stems from Rényi's $α$-Divergences with $α<0$. In particular, we extend the definition of Sibson's $α$-Mutual Information to negative values of $α$ and show several properties of these objects. Moreover, we highlight how this family of information measures is related to functional inequalities that can be employed in a variety of fields, including lower-bounds on the Risk in Bayesian Estimation Procedures.

cs.IT↗