arXiv subjects

François Roueff

Publications and source records attributed to François Roueff.

At least 19 recordsLinked to original sources

arXiv2026

On the privacy cost for dependent Gaussian data: spectral density estimation under local differential privacy

We study the fundamental problem of estimating the dependence structure of a centered stationary Gaussian process under local differential privacy (LDP). In this setting, the spectral density characterizes the dependence structure of the data and is the quantity to be estimated. Our main contribution is to close the open $α^2$-versus-$α^4$ gap between the previously known lower and upper bounds on the minimax rate. Specifically, we establish a minimax lower bound showing that, over Sobolev-type classes of spectral densities, the effective sample size in the high-privacy regime is $Nα^4$, rather than the usual $Nα^2$ arising for independent observations. This additional privacy cost is caused by the temporal dependence between the observations rather than by their marginal distributions. The proof relies on a contraction bound for privatized dependent Gaussian observations. Our second contribution is a matching upper bound, free of the polylogarithmic losses present in previous work. Rather than applying a generic privatization scheme to classical estimators, we construct a problem-specific procedure attaining the rate identified by our lower bound. Beyond closing the gaps in spectral density estimation, we apply the tools developed for this problem to several related questions. We (i) close the logarithmic gap for fixed-lag autocovariance estimation, (ii) show that the $α^4$ cost arises locally around every spectral density bounded away from zero, and (iii) establish that classical asymptotic equivalence with an independent Gaussian experiment generally fails under LDP.

math.ST↗

arXiv2026

Learning with Importance Weighted Variational Inference

Several variational bounds involving importance weighting ideas generalize the Evidence Lower BOund (ELBO) for marginal likelihood optimization, such as the Importance-weighted Auto-Encoder (IWAE), Variational Rényi (VR) and VR-IWAE bounds. Yet, it remains unclear how the joint choice of bound and gradient estimator impacts the behavior of the resulting variational inference (VI) algorithms. This paper provides a unified theoretical comparison of reparameterized (REP) and doubly-reparameterized (DREP) gradient estimators tied to the IWAE, VR and VR-IWAE bounds. Through asymptotic analyses of the Signal-to-Noise Ratio as the number of Monter Carlo samples $N$ goes to infinity, we identify a bias-variance tradeoff in these gradient estimators and we formally justify the superiority of DREP over REP in importance-weighted VI. An additional asymptotic analysis for challenging regimes, where both $N$ and the Kullback-Leibler divergence between the variational and posterior densities go to infinity, indicates that importance-weighted VI gradient estimators point in a well-founded direction even when the variational approximation deteriorates. Together, these complementary results characterize the optimization trajectory in importance-weighted VI from poor initialization to final convergence. Importantly, our proof techniques establish general theoretical tools for the study of sample means ratios whose scope extend beyond VI and constitute an independent contribution to the field of Monte Carlo methods.

stat.ML↗

arXiv2023

Monotonic Alpha-divergence Minimisation for Variational Inference

In this paper, we introduce a novel family of iterative algorithms which carry out $α$-divergence minimisation in a Variational Inference context. They do so by ensuring a systematic decrease at each step in the $α$-divergence between the variational and the posterior distributions. In its most general form, the variational distribution is a mixture model and our framework allows us to simultaneously optimise the weights and components parameters of this mixture model. Our approach permits us to build on various methods previously proposed for $α$-divergence minimisation such as Gradient or Power Descent schemes and we also shed a new light on an integrated Expectation Maximization algorithm. Lastly, we provide empirical evidence that our methodology yields improved results on several multimodal target distributions and on a real data example.

stat.CO↗

arXiv2022

Hilbert valued fractionally integrated autoregressive moving average processes with long memory operators

Fractionally integrated autoregressive moving average (FIARMA) processes have been widely and successfully used to model and predict univariate time series exhibiting long range dependence. Vector and functional extensions of these processes have also been considered more recently. Here we study these processes by relying on a spectral domain approach in the case where the processes are valued in a separable Hilbert space H0. In this framework, the usual univariate long memory parameter d is replaced by a long memory operator D acting on H0, leading to a class of H0-valued FIARMA(D, p, q) processes, where p and q are the degrees of the AR and MA polynomials. When D is a normal operator, we provide a necessary and sufficient condition for the D-fractional integration of an H0-valued ARMA(p, q) process to be well defined. Then, we derive the best predictor for a class of causal FIARMA processes and study how this best predictor can be consistently estimated from a finite sample of the process. To this end, we provide a general result on quadratic functionals of the periodogram, which incidentally yields a result of independent interest. Namely, for any ergodic stationary process valued in H0 with finite second moment, the empirical autocovariance operator converges, in trace-norm, to the true autocovariance operator almost surely at each lag.

math.FA↗

arXiv2022

Weakly stationary stochastic processes valued in a separable Hilbert space: Gramian-Cramér representations and applications

The spectral theory for weakly stationary processes valued in a separable Hilbert space has known renewed interest in the past decade. Here we follow earlier approaches which fully exploit the normal Hilbert module property of the time domain. The key point is to build the Gramian-Cramér representation as an isomorphic mapping from the modular spectral domain to the modular time domain. We also discuss the general Bochner theorem and provide useful results on the composition and inversion of lag-invariant linear filters. Finally, we derive the Cramér-Karhunen-Loève decomposition and harmonic functional principal component analysis, which are established without relying on additional assumptions.

math.ST↗

arXiv2022

New penalized criteria for smooth non-negative tensor factorization with missing entries

Tensor factorization models are widely used in many applied fields such as chemometrics, psychometrics, computer vision or communication networks. Real life data collection is often subject to errors, resulting in missing data. Here we focus in understanding how this issue should be dealt with for nonnegative tensor factorization. We investigate several criteria used for non-negative tensor factorization in the case where some entries are missing. In particular we show how smoothness penalties can compensate the presence of missing values in order to ensure the existence of an optimum. This lead us to propose new criteria with efficient numerical optimization algorithms. Numerical experiments are conducted to support our claims.

eess.SP↗

arXiv2021

Smooth nonnegative tensor factorization for multi-sites electrical load monitoring

The analysis of load curves collected from smart meters is a key step for many energy management tasks ranging from consumption forecasting to customers characterization and load monitoring. In this contribution, we propose a model based on a functional formulation of nonnegative tensor factorization and derive updates for the corresponding optimization problem. We show on the concrete example of multi-sites load curves disaggregation how this formulation is helpful for 1) exhibiting smooth intraday consumption patterns and 2) taking into account external variables such as the outside temperature. The benefits are demonstrated on simulated and real data by exhibiting a meaningful clustering of the observed sites based on the obtained decomposition.

eess.SP↗

arXiv2021

General-order observation-driven models: ergodicity and consistency of the maximum likelihood estimator

The class of observation-driven models (ODMs) includes many models of non-linear time series which, in a fashion similar to, yet different from, hidden Markov models (HMMs), involve hidden variables. Interestingly, in contrast to most HMMs, ODMs enjoy likelihoods that can be computed exactly with computational complexity of the same order as the number of observations, making maximum likelihood estimation the privileged approach for statistical inference for these models. A celebrated example of general order ODMs is the GARCH$(p,q)$ model, for which ergodicity and inference has been studied extensively. However little is known on more general models, in particular integer-valued ones, such as the log-linear Poisson GARCH or the NBIN-GARCH of order $(p,q)$ about which most of the existing results seem restricted to the case $p=q=1$. Here we fill this gap and derive ergodicity conditions for general ODMs. The consistency and the asymptotic normality of the maximum likelihood estimator (MLE) can then be derived using the method already developed for first order ODMs.

math.ST↗

arXiv2021

The $f$-Divergence Expectation Iteration Scheme

This paper introduces the $f$-EI$(ϕ)$ algorithm, a novel iterative algorithm which operates on measures and performs $f$-divergence minimisation in a Bayesian framework. We prove that for a rich family of values of $(f,ϕ)$ this algorithm leads at each step to a systematic decrease in the $f$-divergence and show that we achieve an optimum. In the particular case where we consider a weighted sum of Dirac measures and the $α$-divergence, we obtain that the calculations involved in the $f$-EI$(ϕ)$ algorithm simplify to gradient-based computations. Empirical results support the claim that the $f$-EI$(ϕ)$ algorithm serves as a powerful tool to assist Variational methods.

math.ST↗

arXiv2021

Nonlinear Functional Output Regression: a Dictionary Approach

To address functional-output regression, we introduce projection learning (PL), a novel dictionary-based approach that learns to predict a function that is expanded on a dictionary while minimizing an empirical risk based on a functional loss. PL makes it possible to use non orthogonal dictionaries and can then be combined with dictionary learning; it is thus much more flexible than expansion-based approaches relying on vectorial losses. This general method is instantiated with reproducing kernel Hilbert spaces of vector-valued functions as kernel-based projection learning (KPL). For the functional square loss, two closed-form estimators are proposed, one for fully observed output functions and the other for partially observed ones. Both are backed theoretically by an excess risk analysis. Then, in the more general setting of integral losses based on differentiable ground losses, KPL is implemented using first-order optimization for both fully and partially observed output functions. Eventually, several robustness aspects of the proposed algorithms are highlighted on a toy dataset; and a study on two real datasets shows that they are competitive compared to other nonlinear approaches. Notably, using the square loss and a learnt dictionary, KPL enjoys a particularily attractive trade-off between computational cost and performances.

stat.ML↗

arXiv2020

Necessary and sufficient conditions for the identifiability of observation-driven models

In this contribution we are interested in proving that a given observation-driven model is identifiable. In the case of a GARCH(p, q) model, a simple sufficient condition has been established in [1] for showing the consistency of the quasi-maximum likelihood estimator. It turns out that this condition applies for a much larger class of observation-driven models, that we call the class of linearly observation-driven models. This class includes standard integer valued observation-driven time series, such as the log-linear Poisson GARCH or the NBIN-GARCH models.

math.ST↗

arXiv2020

Spectral estimation for non-linear long range dependent discrete time trawl processes

Discrete time trawl processes constitute a large class of time series parameterized by a trawl sequence (a j) j$\in$N and defined though a sequence of independent and identically distributed (i.i.d.) copies of a continuous time process ($γ$(t)) t$\in$R called the seed process. They provide a general framework for modeling linear or non-linear long range dependent time series. We investigate the spectral estimation, either pointwise or broadband, of long range dependent discrete-time trawl processes. The difficulty arising from the variety of seed processes and of trawl sequences is twofold. First, the spectral density may take different forms, often including smooth additive correction terms. Second, trawl processes with similar spectral densities may exhibit very different statistical behaviors. We prove the consistency of our estimators under very general conditions and we show that a wide class of trawl processes satisfy them. This is done in particular by introducing a weighted weak dependence index that can be of independent interest. The broadband spectral estimator includes an estimator of the long memory parameter. We complete this work with numerical experiments to evaluate the finite sample size performance of this estimator for various integer valued discrete time trawl processes.

math.ST↗

arXiv2018

Time-frequency analysis of locally stationary Hawkes processes

Locally stationary Hawkes processes have been introduced in order to generalise classical Hawkes processes away from stationarity by allowing for a time-varying second-order structure. This class of self-exciting point processes has recently attracted a lot of interest in applications in the life sciences (seismology, genomics, neuro-science,...), but also in the modelling of high-frequency financial data. In this contribution we provide a fully developed nonparametric estimation theory of both local mean density and local Bartlett spectra of a locally stationary Hawkes process. In particular we apply our kernel estimation of the spectrum localised both in time and frequency to two data sets of transaction times revealing pertinent features in the data that had not been made visible by classical non-localised approaches based on models with constant fertility functions over time.

math.ST↗

arXiv2018

Prediction of weakly locally stationary processes by auto-regression

In this contribution we introduce weakly locally stationary time series through the local approximation of the non-stationary covariance structure by a stationary one. This allows us to define autoregression coefficients in a non-stationary context, which, in the particular case of a locally stationary Time Varying Autoregressive (TVAR) process, coincide with the generating coefficients. We provide and study an estimator of the time varying autoregression coefficients in a general setting. The proposed estimator of these coefficients enjoys an optimal minimax convergence rate under limited smoothness conditions. In a second step, using a bias reduction technique, we derive a minimax-rate estimator for arbitrarily smooth time-evolving coefficients, which outperforms the previous one for large data sets. In turn, for TVAR processes, the predictor derived from the estimator exhibits an optimal minimax prediction rate.

math.ST↗

arXiv2017

New results on approximate Hilbert pairs of wavelet filters with common factors

In this paper, we consider the design of wavelet filters based on the Thiran common-factor approach proposed in Selesnick [2001]. This approach aims at building finite impulseresponse filters of a Hilbert-pair of wavelets serving as real and imaginary part of a complexwavelet. Unfortunately it is not possible to construct wavelets which are both finitelysupported and analytic. The wavelet filters constructed using the common-factor approachare then approximately analytic. Thus, it is of interest to control their analyticity. Thepurpose of this paper is to first provide precise and explicit expressions as well as easilyexploitable bounds for quantifying the analytic approximation of this complex wavelet.Then, we prove the existence of such filters enjoying the classical perfect reconstructionconditions, with arbitrarily many vanishing moments.

stat.ME↗

arXiv2016

Anomaly Detection and Localisation using Mixed Graphical Models

We propose a method that performs anomaly detection and localisation within heterogeneous data using a pairwise undirected mixed graphical model. The data are a mixture of categorical and quantitative variables, and the model is learned over a dataset that is supposed not to contain any anomaly. We then use the model over temporal data, potentially a data stream, using a version of the two-sided CUSUM algorithm. The proposed decision statistic is based on a conditional likelihood ratio computed for each variable given the others. Our results show that this function allows to detect anomalies variable by variable, and thus to localise the variables involved in the anomalies more precisely than univariate methods based on simple marginals.

stat.ML↗

arXiv2016

Nonparametric estimation of mark's distribution of an exponential Shot-noise process

In this paper, we consider a nonlinear inverse problem occurring in nuclear science. Gamma rays randomly hit a semiconductor detector which produces an impulse response of electric current. Because the sampling period of the measured current is larger than the mean inter arrival time of photons, the impulse responses associated to different gamma rays can overlap: this phenomenon is known as pileup. In this work, it is assumed that the impulse response is an exponentially decaying function. We propose a novel method to infer the distribution of gamma photon energies from the indirect measurements obtained from the detector. This technique is based on a formula linking the characteristic function of the photon density to a function involving the characteristic function and its derivative of the observations. We establish that our estimator converges to the mark density in uniform norm at a logarithmic rate. A limited Monte-Carlo experiment is provided to support our findings.

stat.AP↗

arXiv2015

Aggregation of predictors for nonstationary sub-linear processes and online adaptive forecasting of time varying autoregressive processes

In this work, we study the problem of aggregating a finite number of predictors for nonstationary sub-linear processes. We provide oracle inequalities relying essentially on three ingredients: (1) a uniform bound of the $\ell^1$ norm of the time varying sub-linear coefficients, (2) a Lipschitz assumption on the predictors and (3) moment conditions on the noise appearing in the linear representation. Two kinds of aggregations are considered giving rise to different moment conditions on the noise and more or less sharp oracle inequalities. We apply this approach for deriving an adaptive predictor for locally stationary time varying autoregressive (TVAR) processes. It is obtained by aggregating a finite number of well chosen predictors, each of them enjoying an optimal minimax convergence rate under specific smoothness conditions on the TVAR coefficients. We show that the obtained aggregated predictor achieves a minimax rate while adapting to the unknown smoothness. To prove this result, a lower bound is established for the minimax rate of the prediction risk for the TVAR process. Numerical experiments complete this study. An important feature of this approach is that the aggregated predictor can be computed recursively and is thus applicable in an online prediction context.

math.ST↗