arXiv subjects

François Bachoc

Publications and source records attributed to François Bachoc.

At least 19 recordsLinked to original sources

arXiv2026

When majority rules, minority loses: bias amplification of gradient descent

Despite growing empirical evidence of bias amplification in machine learning, its theoretical foundations remain poorly understood. We develop a formal framework for majority-minority learning tasks, showing how standard training can favor majority groups and produce stereotypical predictors that neglect minority-specific features. Assuming population and variance imbalance, our analysis reveals three key findings: (i) the close proximity between ``full-data'' and stereotypical predictors, (ii) the dominance of a region where training the entire model tends to merely learn the majority traits, and (iii) a lower bound on the additional training required. Our results are illustrated through experiments in deep learning for tabular and image classification tasks.

cs.LG↗

arXiv2026

High-dimensional reliability-oriented Shapley effect estimation with Normalizing Flows

This article presents a new estimation scheme for the reliability-oriented Shapley effects when there is a large number of correlated input variables in the model, using a unique sample of failure points. To do so, we first propose a new writing of the reliability-oriented closed Sobol indices involving the marginal densities conditionally to the failure, which may be high-dimensional. Then, we propose to estimate these densities with the available failing samples using Normalizing Flows, powerful tools from generative modeling that enable the estimation of complex high-dimensional densities. In addition, we provide an error estimation procedure relying on the same sample of failing points, which constitutes a new contribution for the estimation of target Shapley effects. Finally, we illustrate our methodology on numerical use-cases, discuss insightful features of our approach and provide prospects for the future.

stat.ME↗

arXiv2026

Repeated Bilateral Trade: The Quest for Fairness

We study repeated bilateral trade from a fairness perspective. At each round, a fresh seller-buyer pair arrives, and the platform posts a price before observing the traders' valuations. Trade occurs only if both agents accept the price. Rather than maximizing only the gain from trade, we consider platforms that seek balanced divisions of the generated surplus. We show that natural fairness desiderata lead to a one-parameter Rawls-to-Nash family of fair-gain objectives, obtained by aggregating the seller's and buyer's net gains through nonpositive Hölder means. Unlike the standard gain-from-trade objective and the Rawlsian fair-gain objective studied in prior work, our proposed objectives induce a new statistical structure in which expected rewards are recovered from threshold feedback through a two-dimensional singular-kernel integral identity. This leads to a nonstandard pure-exploration problem whose natural estimators are rectangular double sums with row-column dependence and singular weights. Assuming independent i.i.d. seller and buyer valuation sequences with arbitrary unknown marginals, we characterize the optimal learning rates for the whole Rawls-to-Nash family of fair-gain objectives, giving matching fixed-confidence sample-complexity and regret bounds up to polylogarithmic factors.

cs.LG↗

arXiv2026

Wasserstein Spatial Depth

Modeling observations as random distributions embedded within Wasserstein spaces is becoming increasingly popular across scientific fields, as it captures the variability and geometric structure of the data more effectively. However, the distinct geometry and unique properties of Wasserstein spaces pose challenges to the application of conventional statistical tools, which are primarily designed for Euclidean spaces. Consequently, adapting and developing new methodologies for analysis within Wasserstein spaces has become essential. The space of distributions on $\mathbb{R}^d$ with $d>1$ is not linear, and "mimic" the geometry of a Riemannian manifold. In this paper, we extend the concept of statistical depth to distribution-valued data, introducing the notion of Wasserstein spatial depth. This new measure provides a way to rank and order distributions, enabling the development of order-based clustering techniques and inferential tools. We show that Wasserstein spatial depth (WSD) preserves critical properties of conventional statistical depths, notably, ranging within $[0,1]$, transformation and geodesic invariance, vanishing at infinity, reaching a maximum at the geometric median, and continuity. Regarding robustness, we characterize the breakdown points of the empirical depth regions and the influence function of the WSD. Additionally, the population WSD has a straightforward plug-in estimator based on sampled empirical distributions. We establish the estimator's consistency and asymptotic normality. We also provide a two-sample test for populations of distributions based on the WSD. Finally, extensive simulations and a real-data application showcase the practical efficacy of the WSD.

math.ST↗

arXiv2026

A Parametric Contextual Online Learning Theory of Brokerage

We study the role of contextual information in the online learning problem of brokerage between traders. In this sequential problem, at each time step, two traders arrive with secret valuations about an asset they wish to trade. The learner (a broker) suggests a trading (or brokerage) price based on contextual data about the asset and the market conditions. Then, the traders reveal their willingness to buy or sell based on whether their valuations are higher or lower than the brokerage price. A trade occurs if one of the two traders decides to buy and the other to sell, i.e., if the broker's proposed price falls between the smallest and the largest of their two valuations. We design algorithms for this problem and prove optimal theoretical regret guarantees under various standard assumptions.

q-fin.CP↗

arXiv2026

Online Budget Allocation with Censored Semi-Bandit Feedback

We study a stochastic budget-allocation problem over $K$ tasks. At each round $t$, the learner chooses an allocation $X_t \in Δ_K$. Task $k$ succeeds with probability $F_k(X_{t,k})$, where $F_1,\dots,F_K$ are nondecreasing budget-to-success curves, and upon success yields a random reward with unknown mean $μ_k$. The learner observes which tasks succeed, and observes a task's reward only upon success (censored semi-bandit feedback). This model captures, for instance, splitting payments across crowdsourcing workers or distributing bids across simultaneous auctions, and subsumes stochastic multi-armed bandits and semi-bandits. We design an optimism-based algorithm that operates under censored semi-bandit feedback. Our main result shows that in diminishing-returns regimes, the regret of this algorithm scales polylogarithmically with the horizon $T$ without any ad hoc tuning. For general nondecreasing curves, we prove that the same algorithm (with the same tuning) achieves a worst-case regret upper bound of $\tilde O(K\sqrt{T})$. Finally, we establish a matching worst-case regret lower bound of $Ω(K\sqrt{T})$ that holds even for full-feedback algorithms, highlighting the intrinsic hardness of our problem outside diminishing returns.

cs.GT↗

arXiv2026

Geometry-induced Regularization in Deep ReLU Neural Networks

Neural networks with a large number of parameters often do not overfit, owing to implicit regularization that favors \lq good\rq{} networks. Other related and puzzling phenomena include properties of flat minima, saddle-to-saddle dynamics, and neuron alignment. To investigate these phenomena, we study the local geometry of deep ReLU neural networks. We show that, for a fixed architecture, as the weights vary, the image of a sample $X$ forms a set whose local dimension changes. The parameter space is partitioned into regions where this local dimension remains constant. The local dimension is invariant under the natural symmetries of ReLU networks (i.e., positive rescalings and neuron permutations). We establish then that the network's geometry induces a regularization, with the local dimension serving as a key measure of regularity. Moreover, we relate the local dimension to a new notion of flatness of minima and to saddle-to-saddle dynamics. For shallow networks, we also show that the local dimension is connected to the number of linear regions perceived by $X$, offering insight into the effects of regularization. This is further supported by experiments and linked to neuron alignment. Our analysis offers, for the first time, a simple and unified geometric explanation that applies to all learning contexts for these phenomena, which are usually studied in isolation. Finally, we explore the practical computation of the local dimension and present experiments on the MNIST dataset, which highlight geometry-induced regularization in this setting.

cs.AI↗

arXiv2025

Selective inference after convex clustering with $\ell_1$ penalization

Classical inference methods notoriously fail when applied to data-driven test hypotheses or inference targets. Instead, dedicated methodologies are required to obtain statistical guarantees for these selective inference problems. Selective inference is particularly relevant post-clustering, typically when testing a difference in mean between two clusters. In this paper, we address convex clustering with $\ell_1$ penalization, by leveraging related selective inference tools for regression, based on Gaussian vectors conditioned to polyhedral sets. In the one-dimensional case, we prove a polyhedral characterization of obtaining given clusters, than enables us to suggest a test procedure with statistical guarantees. This characterization also allows us to provide a computationally efficient regularization path algorithm. Then, we extend the above test procedure and guarantees to multi-dimensional clustering with $\ell_1$ penalization, and also to more general multi-dimensional clusterings that aggregate one-dimensional ones. With various numerical experiments, we validate our statistical guarantees and we demonstrate the power of our methods to detect differences in mean between clusters. Our methods are implemented in the R package poclin.

stat.ME↗

arXiv2025

Scale estimation and rate-unbiasedness for Gaussian processes under smoothness misspecification

Gaussian process regression is used throughout statistics and machine learning for prediction and uncertainty quantification. A Gaussian process is specified by its mean and covariance functions. Many covariance functions, including Matérns, have a smoothness parameter that is notoriously difficult to specify correctly or estimate from the data. In practice, the smoothness parameter is often selected more or less arbitrarily. We introduce rate-unbiasedness, a relaxed notion of asymptotic optimality which requires that the expected ratio of the mean-square error presumed by a potentially misspecified model and the true, but unknown, mean-square error remain bounded away from zero and infinity as more data are obtained. A rate-unbiased model provides uncertainty quantification that is of correct order of magnitude. We then prove that scale estimation suffices for rate-unbiasedness in a variety of common settings. As estimation of the scale of a Gaussian process is routine and requires no optimisation, rate-unbiasedness can be achieved in many applications.

math.ST↗

arXiv2025

Kriging measure-valued data with sparse observations: application to nuclear safety studies

This work addresses the interpolation of probability measures within a spatial statistics framework. We develop a Kriging approach in the Wasserstein space, leveraging the quantile function representation of the one-dimensional Wasserstein distance. To mitigate the inaccuracies in semivariogram estimation that arise from sparse datasets, we combine this formulation with cross-validation techniques. In particular, we introduce a variant of the virtual cross-validation formulas tailored to quantile functions. The effectiveness of the proposed method is demonstrated on a controlled toy problem as well as on a real-world application from nuclear safety.

math.ST↗

arXiv2025

Improved learning theory for kernel distribution regression with two-stage sampling

The distribution regression problem encompasses many important statistics and machine learning tasks, and arises in a large range of applications. Among various existing approaches to tackle this problem, kernel methods have become a method of choice. Indeed, kernel distribution regression is both computationally favorable, and supported by a recent learning theory. This theory also tackles the two-stage sampling setting, where only samples from the input distributions are available. In this paper, we improve the learning theory of kernel distribution regression. We address kernels based on Hilbertian embeddings, that encompass most, if not all, of the existing approaches. We introduce the novel near-unbiased condition on the Hilbertian embeddings, that enables us to provide new error bounds on the effect of the two-stage sampling, thanks to a new analysis. We show that this near-unbiased condition holds for three important classes of kernels, based on optimal transport and mean embedding. As a consequence, we strictly improve the existing convergence rates for these kernels. Our setting and results are illustrated by numerical experiments.

math.ST↗

arXiv2025

Contraction rates and projection subspace estimation with Gaussian process priors in high dimension

This work explores the dimension reduction problem for Bayesian nonparametric regression and density estimation. More precisely, we are interested in estimating a functional parameter $f$ over the unit ball in $\mathbb{R}^d$, which depends only on a $d^*$-dimensional subspace of $\mathbb{R}^d$, with $d^* < d$. It is well-known that rescaled Gaussian process priors over the function space achieve smoothness adaptation and posterior contraction with near minimax-optimal rates. Moreover, hierarchical extensions of this approach, equipped with subspace projection, can also adapt to the intrinsic dimension $d^*$ (\cite{Tokdar2011DimensionAdapt}). When the ambient dimension $d$ does not vary with $n$, the minimax rate remains of the order $n^{-β/(2β+d^*)}$, where $β$ denotes the smoothnes of $f$. However, this is up to multiplicative constants that can become prohibitively large when $d$ grows. The dependences between the contraction rate and the ambient dimension have not been fully explored yet and this work provides a first insight: we let the dimension $d$ grow with $n$ and, by combining the arguments of \cite{Tokdar2011DimensionAdapt} and \cite{Jiang2021VariableSelection}, we derive a growth rate for $d$ that still leads to posterior consistency with minimax rate. The optimality of this growth rate is then discussed. Additionally, we provide a set of assumptions under which consistent estimation of $f$ leads to a correct estimation of the subspace projection, assuming that $d^*$ is known.

math.ST↗

arXiv2025

Error Bounds for a Kernel-Based Constrained Optimal Smoothing Approximation

This paper establishes error bounds for the convergence of a piecewise linear approximation of the constrained optimal smoothing problem posed in a reproducing kernel Hilbert space (RKHS). This problem can be reformulated as a Bayesian estimation problem involving a Gaussian process related to the kernel of the RKHS. Consequently, error bounds can be interpreted as a quantification of the maximum a posteriori (MAP) accuracy. To our knowledge, no error bounds have been proposed for this type of problem so far. The convergence results are provided as a function of the grid size, the regularity of the kernel, and the distance from the kernel interpolant of the approximation to the set of constraints. Inspired by the MaxMod algorithm from recent literature, which sequentially allocates knots for the piecewise linear approximation, we conduct our analysis for non-equispaced knots. These knots are even allowed to be non-dense, which impacts the definition of the optimal smoothing solution and our error bound quantifiers. Finally, we illustrate our theorems through several numerical experiments involving constraints such as boundedness and monotonicity.

math.ST↗

arXiv2025

Inference post region selection

Post-selection inference consists in providing statistical guarantees, based on a data set, that are robust to a prior model selection step on the same data set. In this paper, we address an instance of the post-selection-inference problem, where the model selection step consists in selecting a rectangular region in a spatial domain. The inference step then consists in constructing confidence intervals on the average signal of this region. This is motivated by applications such as genetics or brain imaging. Our confidence intervals are constructed in dimension one, and then extended to higher dimension. They are based on the process mapping all possible selected regions to their corresponding estimation errors on the average signal. We prove the functional convergence of this process to a limiting Gaussian process with explicit covariance. This enables us to provide confidence intervals with asymptotic guarantees. In numerical experiments with simulated data, we show that our coverage proportions are fairly close to the nominal level already for small to moderate data-set size. We also highlight the impact of various possible noise distributions and the robustness of our intervals. Finally, we illustrate the relevance of our method to a segmentation problem inspired by the analysis of DNA copy number data in cancerology.

math.ST↗

arXiv2025

A Tight Regret Analysis of Non-Parametric Repeated Contextual Brokerage

We study a contextual version of the repeated brokerage problem. In each interaction, two traders with private valuations for an item seek to buy or sell based on the learner's-a broker-proposed price, which is informed by some contextual information. The broker's goal is to maximize the traders' net utility-also known as the gain from trade-by minimizing regret compared to an oracle with perfect knowledge of traders' valuation distributions. We assume that traders' valuations are zero-mean perturbations of the unknown item's current market value-which can change arbitrarily from one interaction to the next-and that similar contexts will correspond to similar market prices. We analyze two feedback settings: full-feedback, where after each interaction the traders' valuations are revealed to the broker, and limited-feedback, where only transaction attempts are revealed. For both feedback types, we propose algorithms achieving tight regret bounds. We further strengthen our performance guarantees by providing a tight 1/2-approximation result showing that the oracle that knows the traders' valuation distributions achieves at least 1/2 of the gain from trade of the omniscient oracle that knows in advance the actual realized traders' valuations.

stat.ML↗

arXiv2024

Fair Online Bilateral Trade

In online bilateral trade, a platform posts prices to incoming pairs of buyers and sellers that have private valuations for a certain good. If the price is lower than the buyers' valuation and higher than the sellers' valuation, then a trade takes place. Previous work focused on the platform perspective, with the goal of setting prices maximizing the gain from trade (the sum of sellers' and buyers' utilities). Gain from trade is, however, potentially unfair to traders, as they may receive highly uneven shares of the total utility. In this work we enforce fairness by rewarding the platform with the fair gain from trade, defined as the minimum between sellers' and buyers' utilities. After showing that any no-regret learning algorithm designed to maximize the sum of the utilities may fail badly with fair gain from trade, we present our main contribution: a complete characterization of the regret regimes for fair gain from trade when, after each interaction, the platform only learns whether each trader accepted the current price. Specifically, we prove the following regret bounds: $Θ(\ln T)$ in the deterministic setting, $Ω(T)$ in the stochastic setting, and $\tildeΘ(T^{2/3})$ in the stochastic setting when sellers' and buyers' valuations are independent of each other. We conclude by providing tight regret bounds when, after each interaction, the platform is allowed to observe the true traders' valuations.

cs.GT↗

arXiv2024

Asymptotic analysis for covariance parameter estimation of Gaussian processes with functional inputs

We consider covariance parameter estimation for Gaussian processes with functional inputs. From an increasing-domain asymptotics perspective, we prove the asymptotic consistency and normality of the maximum likelihood estimator. We extend these theoretical guarantees to encompass scenarios accounting for approximation errors in the inputs, which allows robustness of practical implementations relying on conventional sampling methods or projections onto a functional basis. Loosely speaking, both consistency and normality hold when the approximation error becomes negligible, a condition that is often achieved as the number of samples or basis functions becomes large. These later asymptotic properties are illustrated through analytical examples, including one that covers the case of non-randomly perturbed grids, as well as several numerical illustrations.

math.ST↗

arXiv2023

Variational autoencoder with weighted samples for high-dimensional non-parametric adaptive importance sampling

Probability density function estimation with weighted samples is the main foundation of all adaptive importance sampling algorithms. Classically, a target distribution is approximated either by a non-parametric model or within a parametric family. However, these models suffer from the curse of dimensionality or from their lack of flexibility. In this contribution, we suggest to use as the approximating model a distribution parameterised by a variational autoencoder. We extend the existing framework to the case of weighted samples by introducing a new objective function. The flexibility of the obtained family of distributions makes it as expressive as a non-parametric model, and despite the very high number of parameters to estimate, this family is much more efficient in high dimension than the classical Gaussian or Gaussian mixture families. Moreover, in order to add flexibility to the model and to be able to learn multimodal distributions, we consider a learnable prior distribution for the variational autoencoder latent variables. We also introduce a new pre-training procedure for the variational autoencoder to find good starting weights of the neural networks to prevent as much as possible the posterior collapse phenomenon to happen. At last, we explicit how the resulting distribution can be combined with importance sampling, and we exploit the proposed procedure in existing adaptive importance sampling algorithms to draw points from a target distribution and to estimate a rare event probability in high dimension on two multimodal problems.

cs.LG↗