arXiv subjects

Christian Brownlees

Publications and source records attributed to Christian Brownlees.

6 recordsLinked to original sources

arXiv2025

Concentration Inequalities for Suprema of Empirical Processes with Dependent Data via Generic Chaining with Applications to Statistical Learning

This paper develops a general concentration inequality for the suprema of empirical processes with dependent data. The concentration inequality is obtained by combining generic chaining with a coupling-based strategy. Our framework accommodates high-dimensional and heavy-tailed (sub-Weibull) data. We demonstrate the usefulness of our result by deriving non-asymptotic predictive performance guarantees for empirical risk minimization in regression problems with dependent data. In particular, we establish an oracle inequality for a broad class of nonlinear regression models and, as a special case, a single-layer neural network model. Our results show that empirical risk minimzaton with dependent data attains a prediction accuracy comparable to that in the i.i.d. setting for a wide range of nonlinear regression models.

econ.EM↗

arXiv2024

Performance of Empirical Risk Minimization For Principal Component Regression

This paper studies the predictive performance of empirical risk minimization for principal component regression. Our analysis accommodates the leading eigenvalues of the predictor covariance matrix growing either linearly or sublinearly with the number of predictors. Additionally, we allow for both light-tailed and heavy-tailed data. Our main result establishes that, under appropriate conditions, empirical risk minimization for principal component regression is consistent for prediction and achieves near-optimal performance.

econ.EM↗

arXiv2022

Unit Averaging for Heterogeneous Panels

In this work we introduce a unit averaging procedure to efficiently recover unit-specific parameters in a heterogeneous panel model. The procedure consists in estimating the parameter of a given unit using a weighted average of all the unit-specific parameter estimators in the panel. The weights of the average are determined by minimizing an MSE criterion we derive. We analyze the properties of the resulting minimum MSE unit averaging estimator in a local heterogeneity framework inspired by the literature on frequentist model averaging, and we derive the local asymptotic distribution of the estimator and the corresponding weights. The benefits of the procedure are showcased with an application to forecasting unemployment rates for a panel of German regions.

econ.EM↗

arXiv2021

Empirical Risk Minimization for Time Series: Nonparametric Performance Bounds for Prediction

Empirical risk minimization is a standard principle for choosing algorithms in learning theory. In this paper we study the properties of empirical risk minimization for time series. The analysis is carried out in a general framework that covers different types of forecasting applications encountered in the literature. We are concerned with 1-step-ahead prediction of a univariate time series generated by a parameter-driven process. A class of recursive algorithms is available to forecast the time series. The algorithms are recursive in the sense that the forecast produced in a given period is a function of the lagged values of the forecast and of the time series. The relationship between the generating mechanism of the time series and the class of algorithms is unspecified. Our main result establishes that the algorithm chosen by empirical risk minimization achieves asymptotically the optimal predictive performance that is attainable within the class of algorithms.

stat.ML↗

arXiv2021

Performance of Empirical Risk Minimization for Linear Regression with Dependent Data

This paper establishes bounds on the performance of empirical risk minimization for large-dimensional linear regression. We generalize existing results by allowing the data to be dependent and heavy-tailed. The analysis covers both the cases of identically and heterogeneously distributed observations. Our analysis is nonparametric in the sense that the relationship between the regressand and the regressors is not specified. The main results of this paper show that the empirical risk minimizer achieves the optimal performance (up to a logarithmic factor) in a dependent data setting.

econ.EM↗

arXiv2015

Empirical risk minimization for heavy-tailed losses

The purpose of this paper is to discuss empirical risk minimization when the losses are not necessarily bounded and may have a distribution with heavy tails. In such situations, usual empirical averages may fail to provide reliable estimates and empirical risk minimization may provide large excess risk. However, some robust mean estimators proposed in the literature may be used to replace empirical means. In this paper, we investigate empirical risk minimization based on a robust estimate proposed by Catoni. We develop performance bounds based on chaining arguments tailored to Catoni's mean estimator.

stat.ME↗