arXiv subjects

Olivier Wintenberger

Publications and source records attributed to Olivier Wintenberger.

At least 55 records · Page 3Linked to original sources

arXiv2021

Consistent Regression using Data-Dependent Coverings

In this paper, we introduce a novel method to generate interpretable regression function estimators. The idea is based on called data-dependent coverings. The aim is to extract from the data a covering of the feature space instead of a partition. The estimator predicts the empirical conditional expectation over the cells of the partitions generated from the coverings. Thus, such estimator has the same form as those issued from data-dependent partitioning algorithms. We give sufficient conditions to ensure the consistency, avoiding the sufficient condition of shrinkage of the cells that appears in the former literature. Doing so, we reduce the number of covering elements. We show that such coverings are interpretable and each element of the covering is tagged as significant or insignificant. The proof of the consistency is based on a control of the error of the empirical estimation of conditional expectations which is interesting on its own.

math.ST↗

arXiv2021

AdaVol: An Adaptive Recursive Volatility Prediction Method

Quasi-Maximum Likelihood (QML) procedures are theoretically appealing and widely used for statistical inference. While there are extensive references on QML estimation in batch settings, it has attracted little attention in streaming settings until recently. An investigation of the convergence properties of the QML procedure in a general conditionally heteroscedastic time series model is conducted, and the classical batch optimization routines extended to the framework of streaming and large-scale problems. An adaptive recursive estimation routine for GARCH models named AdaVol is presented. The AdaVol procedure relies on stochastic approximations combined with the technique of Variance Targeting Estimation (VTE). This recursive method has computationally efficient properties, while VTE alleviates some convergence difficulties encountered by the usual QML estimation due to a lack of convexity. Empirical results demonstrate a favorable trade-off between AdaVol's stability and the ability to adapt to time-varying estimates for real-life data.

q-fin.ST↗

arXiv2020

Stochastic Online Optimization using Kalman Recursion

We study the Extended Kalman Filter in constant dynamics, offering a bayesian perspective of stochastic optimization. We obtain high probability bounds on the cumulative excess risk in an unconstrained setting. In order to avoid any projection step we propose a two-phase analysis. First, for linear and logistic regressions, we prove that the algorithm enters a local phase where the estimate stays in a small region around the optimum. We provide explicit bounds with high probability on this convergence time. Second, for generalized linear regressions, we provide a martingale analysis of the excess risk in the local phase, improving existing ones in bounded stochastic optimization. The EKF appears as a parameter-free online algorithm with O(d^2) cost per iteration that optimally solves some unconstrained optimization problems.

cs.LG↗

arXiv2020

Kalman Recursions Aggregated Online

In this article, we aim at improving the prediction of expert aggregation by using the underlying properties of the models that provide expert predictions. We restrict ourselves to the case where expert predictions come from Kalman recursions, fitting state-space models. By using exponential weights, we construct different algorithms of Kalman recursions Aggregated Online (KAO) that compete with the best expert or the best convex combination of experts in a more or less adaptive way. We improve the existing results on expert aggregation literature when the experts are Kalman recursions by taking advantage of the second-order properties of the Kalman recursions. We apply our approach to Kalman recursions and extend it to the general adversarial expert setting by state-space modeling the errors of the experts. We apply these new algorithms to a real dataset of electricity consumption and show how it can improve forecast performances comparing to other exponentially weighted average procedures.

cs.LG↗

arXiv2019

On total claim amount for marked Poisson cluster models

We study the asymptotic distribution of the total claim amount for marked Poisson cluster models. The marks determine the size and other characteristics of the individual claims and potentially influence arrival rate of the future claims. We find sufficient conditions under which the total claim amount satisfies the central limit theorem or alternatively tends in distribution to an infinite variance stable random variable. We discuss several Poisson cluster models in detail, paying special attention to the marked Hawkes processes as our key example.

math.PR↗

arXiv2019

Logarithmic Regret for parameter-free Online Logistic Regression

We consider online optimization procedures in the context of logistic regression, focusing on the Extended Kalman Filter (EKF). We introduce a second-order algorithm close to the EKF, named Semi-Online Step (SOS), for which we prove a O(log(n)) regret in the adversarial setting, paving the way to similar results for the EKF. This regret bound on SOS is the first for such parameter-free algorithm in the adversarial logistic regression. We prove for the EKF in constant dynamics a O(log(n)) regret in expectation and in the well-specified logistic regression model.

cs.LG↗

arXiv2018

The tail empirical process of regularly varying functions of geometrically ergodic Markov chains

We consider a stationary regularly varying time series which can be expressedas a function of a geometrically ergodic Markov chain. We obtain practical conditionsfor the weak convergence of the tail array sums and feasible estimators ofcluster statistics. These conditions include the so-called geometric drift or Foster-Lyapunovcondition and can be easily checked for most usual time series models witha Markovian structure. We illustrate these conditions on several models and statisticalapplications. A counterexample is given to show a different limiting behaviorwhen the geometric drift condition is not fulfilled.

math.ST↗

arXiv2018

Rule Induction Partitioning Estimator

RIPE is a novel deterministic and easily understandable prediction algorithm developed for continuous and discrete ordered data. It infers a model, from a sample, to predict and to explain a real variable $Y$ given an input variable $X \in \mathcal X$ (features). The algorithm extracts a sparse set of hyperrectangles $\mathbf r \subset \mathcal X$, which can be thought of as rules of the form If-Then. This set is then turned into a partition of the features space $\mathcal X$ of which each cell is explained as a list of rules with satisfied their If conditions. The process of RIPE is illustrated on simulated datasets and its efficiency compared with that of other usual algorithms.

stat.ML↗

arXiv2018

Efficient online algorithms for fast-rate regret bounds under sparsity

We consider the online convex optimization problem. In the setting of arbitrary sequences and finite set of parameters, we establish a new fast-rate quantile regret bound. Then we investigate the optimization into the L1-ball by discretizing the parameter space. Our algorithm is projection free and we propose an efficient solution by restarting the algorithm on adaptive discretization grids. In the adversarial setting, we develop an algorithm that achieves several rates of convergence with different dependencies on the sparsity of the objective. In the i.i.d. setting, we establish new risk bounds that are adaptive to the sparsity of the problem and to the regularity of the risk (ranging from a rate 1 / $\sqrt T$ for general convex risk to 1 /T for strongly convex risk). These results generalize previous works on sparse online learning. They are obtained under a weak assumption on the risk (Łojasiewicz's assumption) that allows multiple optima which is crucial when dealing with degenerate situations.

math.ST↗

arXiv2017

On the tail behavior of a class of multivariate conditionally heteroskedastic processes

Conditions for geometric ergodicity of multivariate autoregressive conditional heteroskedasticity (ARCH) processes, with the so-called BEKK (Baba, Engle, Kraft, and Kroner) parametrization, are considered. We show for a class of BEKK-ARCH processes that the invariant distribution is regularly varying. In order to account for the possibility of different tail indices of the marginals, we consider the notion of vector scaling regular variation, in the spirit of Perfekt (1997, Advances in Applied Probability, 29, pp. 138-164). The characterization of the tail behavior of the processes is used for deriving the asymptotic properties of the sample covariance matrices.

math.ST↗

arXiv2017

A Strongly Quasiconvex PAC-Bayesian Bound

We propose a new PAC-Bayesian bound and a way of constructing a hypothesis space, so that the bound is convex in the posterior distribution and also convex in a trade-off parameter between empirical performance of the posterior distribution and its complexity. The complexity is measured by the Kullback-Leibler divergence to a prior. We derive an alternating procedure for minimizing the bound. We show that the bound can be rewritten as a one-dimensional function of the trade-off parameter and provide sufficient conditions under which the function has a single global minimum. When the conditions are satisfied the alternating minimization is guaranteed to converge to the global minimum of the bound. We provide experimental results demonstrating that rigorous minimization of the bound is competitive with cross-validation in tuning the trade-off between complexity and empirical performance. In all our experiments the trade-off turned to be quasiconvex even when the sufficient conditions were violated.

cs.LG↗

arXiv2017

Heavy Tails for an Alternative Stochastic Perpetuity Model

In this paper we consider a stochastic model of perpetuity-type. In contrast to the classical affine perpetuity model of Kesten [12] and Goldie [8] all discount factors in the model are mutually independent. We prove that the tails of the distribution of this model are regularly varying both in the univariate and multivariate cases. Due to the additional randomness in the model the tails are not pure power laws as in the Kesten-Goldie setting but involve a logarithmic term.

math.PR↗

arXiv2016

Sparse Accelerated Exponential Weights

We consider the stochastic optimization problem where a convex function is minimized observing recursively the gradients. We introduce SAEW, a new procedure that accelerates exponential weights procedures with the slow rate $1/\sqrt{T}$ to procedures achieving the fast rate $1/T$. Under the strong convexity of the risk, we achieve the optimal rate of convergence for approximating sparse parameters in $\mathbb{R}^d$. The acceleration is achieved by using successive averaging steps in an online fashion. The procedure also produces sparse estimators thanks to additional hard threshold steps.

math.ST↗

arXiv2016

Optimal learning with Bernstein Online Aggregation

We introduce a new recursive aggregation procedure called Bernstein Online Aggregation (BOA). The exponential weights include an accuracy term and a second order term that is a proxy of the quadratic variation as in Hazan and Kale (2010). This second term stabilizes the procedure that is optimal in different senses. We first obtain optimal regret bounds in the deterministic context. Then, an adaptive version is the first exponential weights algorithm that exhibits a second order bound with excess losses that appears first in Gaillard et al. (2014). The second order bounds in the deterministic context are extended to a general stochastic context using the cumulative predictive risk. Such conversion provides the main result of the paper, an inequality of a novel type comparing the procedure with any deterministic aggregation procedure for an integrated criteria. Then we obtain an observable estimate of the excess of risk of the BOA procedure. To assert the optimality, we consider finally the iid case for strongly convex and Lipschitz continuous losses and we prove that the optimal rate of aggregation of Tsybakov (2003) is achieved. The batch version of the BOA procedure is then the first adaptive explicit algorithm that satisfies an optimal oracle inequality with high probability.

stat.ML↗

arXiv2016

Exponential inequalities for unbounded functions of geometrically ergodic Markov chains. Applications to quantitative error bounds for regenerative Metropolis algorithms

The aim of this note is to investigate the concentration properties of unbounded functions of geometrically ergodic Markov chains. We derive concentration properties of centered functions with respect to the square of the Lyapunov's function in the drift condition satisfied by the Markov chain. We apply the new exponential inequalities to derive confidence intervals for MCMC algorithms. Quantitative error bounds are providing for the regenerative Metropolis algorithm of [5].

math.ST↗

arXiv2016

Regular variation of a random length sequence of random variables and application to risk assessment

When assessing risks on a finite-time horizon, the problem can often be reduced to the study of a random sequence $C(N)=(C_1,\ldots,C_N)$ of random length $N$, where $C(N)$ comes from the product of a matrix $A(N)$ of random size $N \times N$ and a random sequence $X(N)$ of random length $N$. Our aim is to build a regular variation framework for such random sequences of random length, to study their spectral properties and, subsequently, to develop risk measures. In several applications, many risk indicators can be expressed from the asymptotic behavior of $\vert \vert C(N)\vert\vert$, for some norm $\Vert \cdot \Vert$. We propose a generalization of Breiman Lemma that gives way to an asymptotic equivalent to $\Vert C(N) \Vert$ and provides risk indicators such as the ruin probability and the tail index for Shot Noise Processes on a finite-time horizon. Lastly, we apply our final result to a model used in dietary risk assessment and in non-life insurance mathematics to illustrate the applicability of our method.

math.PR↗

arXiv2016

Goodness-of-fit tests for extended Log-GARCH models

This paper studies goodness of fit tests and specification tests for an extension of the log-GARCH model which is stable by scaling. A Lagrange-Multiplier test is derived for testing the null assumption of extended log-GARCH against more general formulations including the Exponential GARCH (EGARCH). The null assumption of an EGARCH is also tested. Portmanteau goodness-of-fit tests are developed for the extended log-GARCH. Simulations illustrating the theoretical results and an application to real financial data are proposed.

math.ST↗

arXiv2014

Weak transport inequalities and applications to exponential inequalities and oracle inequalities

We extend the dimension free Talagrand inequalities for convex distance \cite{talagrand:1995} using an extension of Marton's weak transport \cite{marton:1996a} to other metrics than the Hamming distance. We study the dual form of these weak transport inequalities for the euclidian norm and prove that it implies sub-gaussianity and convex Poincaré inequality \cite{bobkov:gotze:1999a}. We obtain new weak transport inequalities for non products measures extending the results of Samson in \cite{samson:2000}. Many examples are provided to show that the euclidian norm is an appropriate metric for classical time series. Our approach, based on trajectories coupling, is more efficient to obtain dimension free concentration than existing contractive assumptions \cite{djellout:guillin:wu:2004,marton:2004}. Expressing the concentration properties of the ordinary least square estimator as a conditional weak transport problem, we derive new oracle inequalities with fast rates of convergence in dependent settings.

math.PR↗