arXiv subjects

Victor-Emmanuel Brunel

Publications and source records attributed to Victor-Emmanuel Brunel.

At least 19 recordsLinked to original sources

arXiv2026

Functional Central Limit Theorem for Stochastic Gradient Descent

We study the asymptotic shape of the trajectory of the stochastic gradient descent algorithm applied to a convex objective function. Under mild regularity assumptions, we prove a functional central limit theorem for the properly rescaled trajectory. Our result characterizes the long-term fluctuations of the algorithm around the minimizer by providing a diffusion limit for the trajectory. In contrast with classical central limit theorems for the last iterate or Polyak-Ruppert averages, this functional result captures the temporal structure of the fluctuations and applies to non-smooth settings such as robust location estimation, including the geometric median.

stat.ML↗

arXiv2026

Bernstein-von Mises theorem for log-concave posteriors

We prove new, general versions of Bernstein-von Mises theorem for both well-specified and misspecified models when the log-likelihood is concave in the parameter and the prior distribution is log-concave. Unlike classical versions of Bernstein-von Mises theorem, our versions do not require technical smoothness assumptions, and they solely rely on convex analysis.

math.ST↗

arXiv2025

On the continuity of geodesically convex functions on Riemannian manifolds

In this short note, we prove that all geodesically convex functions defined on a Riemannian manifold are continuous in the interior of their domain. This is a folklore result, but to the best of our knowledge, there is only one available proof, which is largely cited. However, it contains a significant gap, which we fill here. We also discuss extensions of this result beyond the Riemannian setting.

math.DG↗

arXiv2025

Asymptotics of constrained $M$-estimation under convexity

M-estimation, aka empirical risk minimization, is at the heart of statistics and machine learning: Classification, regression, location estimation, etc. Asymptotic theory is well understood when the loss satisfies some smoothness assumptions and its derivatives are dominated locally. However, these conditions are typically technical and can be too restrictive or heavy to check. Here, we consider the case of a convex loss function, which may not even be differentiable: We establish an asymptotic theory for M-estimation with convex loss (which needs not be differentiable) under convex constraints. We show that the asymptotic distributions of the corresponding M-estimators depend on an interplay between the loss function and the boundary structure of the set of constraints. We extend our results to U-estimators, building on the asymptotic theory of U-statistics. Applications of our work include, among other, robust location/scatter estimation, estimation of deepest points relative to depth functions such as Oja's depth, etc.

math.ST↗

arXiv2025

Bayesian Off-Policy Evaluation and Learning for Large Action Spaces

In interactive systems, actions are often correlated, presenting an opportunity for more sample-efficient off-policy evaluation (OPE) and learning (OPL) in large action spaces. We introduce a unified Bayesian framework to capture these correlations through structured and informative priors. In this framework, we propose sDM, a generic Bayesian approach for OPE and OPL, grounded in both algorithmic and theoretical foundations. Notably, sDM leverages action correlations without compromising computational efficiency. Moreover, inspired by online Bayesian bandits, we introduce Bayesian metrics that assess the average performance of algorithms across multiple problem instances, deviating from the conventional worst-case assessments. We analyze sDM in OPE and OPL, highlighting the benefits of leveraging action correlations. Empirical evidence showcases the strong performance of sDM.

cs.LG↗

arXiv2025

Finite sample bounds for barycenter estimation in geodesic spaces

We study the problem of estimating the barycenter of a distribution given i.i.d. data in a geodesic space. Assuming an upper curvature bound in Alexandrov's sense and a support condition ensuring the strong geodesic convexity of the barycenter problem, we establish finite-sample error bounds in expectation and with high probability. Our results generalize Hoeffding- and Bernstein-type concentration inequalities from Euclidean to geodesic spaces. Building on these concentration inequalities, we derive statistical guarantees for two efficient algorithms for the computation of barycenters.

math.ST↗

arXiv2024

Recovering a Magnitude-Symmetric Matrix from its Principal Minors

We consider the inverse problem of finding a magnitude-symmetric matrix (matrix with opposing off-diagonal entries equal in magnitude) with a prescribed set of principal minors. This problem is closely related to the theory of recognizing and learning signed determinantal point processes in machine learning, as kernels of these point processes are magnitude-symmetric matrices. In this work, we prove a number of properties regarding sparse and generic magnitude-symmetric matrices. We show that principal minors of order at most $\ell$, for some invariant $\ell$ depending only on principal minors of order at most two, uniquely determines principal minors of all orders. In addition, we produce a polynomial-time algorithm that, given access to principal minors, recovers a matrix with those principal minors using only a quadratic number of queries. Furthermore, when principal minors are known only approximately, we present an algorithm that approximately recovers a matrix, and show that the approximation guarantee of this algorithm cannot be improved in general.

math.CO↗

arXiv2024

A generalization of Grünbaum's inequality in RCD$(0,N)$-spaces

We generalize Grünbaum's classical inequality in convex geometry to curved spaces with nonnegative Ricci curvature, precisely, to $\mathrm{RCD}(0,N)$-spaces with $N \in (1,\infty)$ as well as weighted Riemannian manifolds of $\mathrm{Ric}_N \ge 0$ for $N \in (-\infty,-1) \cup \{\infty\}$. Our formulation makes use of the isometric splitting theorem; given a convex set $Ω$ and the Busemann function associated with any straight line, the volume of the intersection of $Ω$ and any sublevel set of the Busemann function that contains a barycenter of $Ω$ is bounded from below in terms of $N$. We also extend this inequality beyond uniform distributions on convex sets. Moreover, we establish some rigidity results by using the localization method, and the stability problem is also studied.

math.MG↗

arXiv2024

Unified PAC-Bayesian Study of Pessimism for Offline Policy Learning with Regularized Importance Sampling

Off-policy learning (OPL) often involves minimizing a risk estimator based on importance weighting to correct bias from the logging policy used to collect data. However, this method can produce an estimator with a high variance. A common solution is to regularize the importance weights and learn the policy by minimizing an estimator with penalties derived from generalization bounds specific to the estimator. This approach, known as pessimism, has gained recent attention but lacks a unified framework for analysis. To address this gap, we introduce a comprehensive PAC-Bayesian framework to examine pessimism with regularized importance weighting. We derive a tractable PAC-Bayesian generalization bound that universally applies to common importance weight regularizations, enabling their comparison within a single framework. Our empirical results challenge common understanding, demonstrating the effectiveness of standard IW regularization techniques.

cs.LG↗

arXiv2023

Convex generalized Fr\'echet means in a metric tree

We are interested in measures of central tendency for a population $\mu$ on a network, which is modeled by a metric tree. The location parameters that we study are generalized Fr\'echet means defined as minimizers of the objective function $\alpha \mapsto \mathbb E[\ell(d(\alpha,X)) - \ell(d(o,X))]$, where $\ell$ is a convex, strictly increasing loss, $X\sim \mu$ and $o$ is an arbitrary origin. We leverage the geometry of the tree and the geodesic convexity of the objective to develop a notion of directional derivative in the tree, which helps us locate and characterize the minimizers. We then extend to a metric tree the concept of stickiness defined by Hotz et al. (2013). When $\mu$ is nondegenerate, any Fr\'echet $\ell$-mean of $\mu$ is either sticky, one-sided partly sticky or two-sided partly sticky, depending on the number of vanishing directional derivatives. Estimation is performed using a sample analog. We establish laws of large numbers and central limit theorems that reveal distinct asymptotic behaviors of empirical means across these three regimes: collapse onto the population mean in the sticky case, partial collapse accompanied by one-sided fluctuations in the one-sided case, and two-sided fluctuations without collapse in the two-sided case. We further develop two consistent estimators of the stickiness regime, based respectively on pairwise collisions among empirical means and on empirical difference quotients. For the particular case of the Fr\'echet median, we develop distribution-free non-asymptotic confidence regions for the entire set of minimizers.

math.ST↗

arXiv2023

Exponential Smoothing for Off-Policy Learning

Off-policy learning (OPL) aims at finding improved policies from logged bandit data, often by minimizing the inverse propensity scoring (IPS) estimator of the risk. In this work, we investigate a smooth regularization for IPS, for which we derive a two-sided PAC-Bayes generalization bound. The bound is tractable, scalable, interpretable and provides learning certificates. In particular, it is also valid for standard IPS without making the assumption that the importance weights are bounded. We demonstrate the relevance of our approach and its favorable performance through a set of learning tasks. Since our bound holds for standard IPS, we are able to provide insight into when regularizing IPS is useful. Namely, we identify cases where regularization might not be needed. This goes against the belief that, in practice, clipped IPS often enjoys favorable performance than standard IPS in OPL.

cs.LG↗

arXiv2023

Geodesically convex $M$-estimation in metric spaces

We study the asymptotic properties of geodesically convex $M$-estimation on non-linear spaces. Namely, we prove that under very minimal assumptions besides geodesic convexity of the cost function, one can obtain consistency and asymptotic normality, which are fundamental properties in statistical inference. Our results extend the Euclidean theory of convex $M$-estimation; They also generalize limit theorems on non-linear spaces which, essentially, were only known for barycenters, allowing to consider robust alternatives that are defined through non-smooth $M$-estimation procedures.

math.ST↗

arXiv2023

Concentration of empirical barycenters in metric spaces

Barycenters (aka Fréchet means) were introduced in statistics in the 1940's and popularized in the fields of shape statistics and, later, in optimal transport and matrix analysis. They provide the most natural extension of linear averaging to non-Euclidean geometries, which is perhaps the most basic and widely used tool in data science. In various setups, their asymptotic properties, such as laws of large numbers and central limit theorems, have been established, but their non-asymptotic behaviour is still not well understood. In this work, we prove finite sample concentration inequalities (namely, generalizations of Hoeffding's and Bernstein's inequalities) for barycenters of i.i.d. random variables in metric spaces with non-positive curvature in Alexandrov's sense. As a byproduct, we also obtain PAC guarantees for a stochastic online algorithm that computes the barycenter of a finite collection of points in a non-positively curved space. We also discuss extensions of our results to spaces with possibly positive curvature.

math.ST↗

arXiv2021

Scalable Learning and MAP Inference for Nonsymmetric Determinantal Point Processes

Determinantal point processes (DPPs) have attracted significant attention in machine learning for their ability to model subsets drawn from a large item collection. Recent work shows that nonsymmetric DPP (NDPP) kernels have significant advantages over symmetric kernels in terms of modeling power and predictive performance. However, for an item collection of size $M$, existing NDPP learning and inference algorithms require memory quadratic in $M$ and runtime cubic (for learning) or quadratic (for inference) in $M$, making them impractical for many typical subset selection tasks. In this work, we develop a learning algorithm with space and time requirements linear in $M$ by introducing a new NDPP kernel decomposition. We also derive a linear-complexity NDPP maximum a posteriori (MAP) inference algorithm that applies not only to our new kernel but also to that of prior work. Through evaluation on real-world datasets, we show that our algorithms scale significantly better, and can match the predictive performance of prior work.

cs.LG↗

arXiv2020

Learning Nonsymmetric Determinantal Point Processes

Determinantal point processes (DPPs) have attracted substantial attention as an elegant probabilistic model that captures the balance between quality and diversity within sets. DPPs are conventionally parameterized by a positive semi-definite kernel matrix, and this symmetric kernel encodes only repulsive interactions between items. These so-called symmetric DPPs have significant expressive power, and have been successfully applied to a variety of machine learning tasks, including recommendation systems, information retrieval, and automatic summarization, among many others. Efficient algorithms for learning symmetric DPPs and sampling from these models have been reasonably well studied. However, relatively little attention has been given to nonsymmetric DPPs, which relax the symmetric constraint on the kernel. Nonsymmetric DPPs allow for both repulsive and attractive item interactions, which can significantly improve modeling power, resulting in a model that may better fit for some applications. We present a method that enables a tractable algorithm, based on maximum likelihood estimation, for learning nonsymmetric DPPs from data composed of observed subsets. Our method imposes a particular decomposition of the nonsymmetric kernel that enables such tractable learning algorithms, which we analyze both theoretically and experimentally. We evaluate our model on synthetic and real-world datasets, demonstrating improved predictive performance compared to symmetric DPPs, which have previously shown strong performance on modeling tasks associated with these datasets.

cs.LG↗

arXiv2020

Statistical guarantees for generative models without domination

In this paper, we introduce a convenient framework for studying (adversarial) generative models from a statistical perspective. It consists in modeling the generative device as a smooth transformation of the unit hypercube of a dimension that is much smaller than that of the ambient space and measuring the quality of the generative model by means of an integral probability metric. In the particular case of integral probability metric defined through a smoothness class, we establish a risk bound quantifying the role of various parameters. In particular, it clearly shows the impact of dimension reduction on the error of the generative model.

math.ST↗

arXiv2020

Propose, Test, Release: Differentially private estimation with high probability

We derive concentration inequalities for differentially private median and mean estimators building on the "Propose, Test, Release" (PTR) mechanism introduced by Dwork and Lei (2009). We introduce a new general version of the PTR mechanism that allows us to derive high probability error bounds for differentially private estimators. Our algorithms provide the first statistical guarantees for differentially private estimation of the median and mean without any boundedness assumptions on the data, and without assuming that the target population parameter lies in some known bounded interval. Our procedures do not rely on any truncation of the data and provide the first sub-Gaussian high probability bounds for differentially private median and mean estimation, for possibly heavy tailed random variables.

stat.ML↗

arXiv2019

Differentially private sub-Gaussian location estimators

We tackle the problem of estimating a location parameter with differential privacy guarantees and sub-Gaussian deviations. Recent work in statistics has focused on the study of estimators that achieve sub-Gaussian type deviations even for heavy tailed data. We revisit some of these estimators through the lens of differential privacy and show that a naive application of the Laplace mechanism can lead to sub-optimal results. We design two private algorithms for estimating the median that lead to estimators with sub-Gaussian type errors. Unlike most existing differentially private median estimators, both algorithms are well defined for unbounded random variables that are not even required to have finite moments. We then turn to the problem of sub-Gaussian mean estimation and show that under heavy tails natural differentially private alternatives lead to strictly worse deviations than their non-private sub-Gaussian counterparts. This is in sharp contrast with recent results that show that from an asymptotic perspective the cost of differential privacy is negligible.

math.ST↗