arXiv subjects

Dimitris N. Politis

Publications and source records attributed to Dimitris N. Politis.

At least 19 recordsLinked to original sources

arXiv2026

Distributional Conformal Prediction for Markov Processes

We introduce the Markov Distributional Conformal Prediction (MDCP) method that extends the distributional conformal prediction (previously developed for regression) to the setting of a strictly stationary Markov process. Instead of relying on a specific model structure to do prediction, the idea of distributional conformal prediction interval aligns with the Model-Free (MF) Prediction Principle. In analogy to MF prediction of Markov processes, our method exploits the probability integral transform based on estimated transition distribution functions to transform the Markov data to an i.i.d.~dataset. We show a non-asymptotic error bound of MDCPs unconditional coverage rate under a $\beta$-mixing condition and other standard assumptions on the kernel estimators. The asymptotic validity of the conditional prediction interval is also verified. In addition, we show that our conditional prediction interval is still asymptotically valid with Markov processes being $L^p$-$m$-approximable instead of satisfying the mixing property. Numerical simulations and real data experiments are deployed to empirically illustrate the finite-sample performance of MDCP, and compare it with the MF bootstrap prediction method.

stat.ME↗

arXiv2025

Calibration Prediction Interval for Non-parametric Regression and Neural Networks

Accurate conditional prediction in the regression setting plays an important role in many real-world problems. Typically, a point prediction often falls short since no attempt is made to quantify the prediction accuracy. Classically, under the normality and linearity assumptions, the Prediction Interval (PI) for the response variable can be determined routinely based on the $t$ distribution. Unfortunately, these two assumptions are rarely met in practice. To fully avoid these two conditions, we develop a so-called calibration PI (cPI) which leverages estimations by Deep Neural Networks (DNN) or kernel methods. Moreover, the cPI can be easily adjusted to capture the estimation variability within the prediction procedure, which is a crucial error source often ignored in practice. Under regular assumptions, we verify that our cPI has an asymptotically valid coverage rate. We also demonstrate that cPI based on the kernel method ensures a coverage rate with a high probability when the sample size is large. Besides, with several conditions, the cPI based on DNN works even with finite samples. A comprehensive simulation study supports the usefulness of cPI, and the convincing performance of cPI with a short sample is confirmed with two empirical datasets.

stat.ME↗

arXiv2024

Deep Limit Model-free Prediction in Regression

In this paper, we provide a novel Model-free approach based on Deep Neural Network (DNN) to accomplish point prediction and prediction interval under a general regression setting. Usually, people rely on parametric or non-parametric models to bridge dependent and independent variables (Y and X). However, this classical method relies heavily on the correct model specification. Even for the non-parametric approach, some additive form is often assumed. A newly proposed Model-free prediction principle sheds light on a prediction procedure without any model assumption. Previous work regarding this principle has shown better performance than other standard alternatives. Recently, DNN, one of the machine learning methods, has received increasing attention due to its great performance in practice. Guided by the Model-free prediction idea, we attempt to apply a fully connected forward DNN to map X and some appropriate reference random variable Z to Y. The targeted DNN is trained by minimizing a specially designed loss function so that the randomness of Y conditional on X is outsourced to Z through the trained DNN. Our method is more stable and accurate compared to other DNN-based counterparts, especially for optimal point predictions. With a specific prediction procedure, our prediction interval can capture the estimation variability so that it can render a better coverage rate for finite sample cases. The superior performance of our method is verified by simulation and empirical studies.

stat.ML↗

arXiv2024

Scalable Subsampling Inference for Deep Neural Networks

Deep neural networks (DNN) has received increasing attention in machine learning applications in the last several years. Recently, a non-asymptotic error bound has been developed to measure the performance of the fully connected DNN estimator with ReLU activation functions for estimating regression models. The paper at hand gives a small improvement on the current error bound based on the latest results on the approximation ability of DNN. More importantly, however, a non-random subsampling technique--scalable subsampling--is applied to construct a `subagged' DNN estimator. Under regularity conditions, it is shown that the subagged DNN estimator is computationally efficient without sacrificing accuracy for either estimation or prediction tasks. Beyond point estimation/prediction, we propose different approaches to build confidence and prediction intervals based on the subagged DNN estimator. In addition to being asymptotically valid, the proposed confidence/prediction intervals appear to work well in finite samples. All in all, the scalable subsampling DNN estimator offers the complete package in terms of statistical inference, i.e., (a) computational efficiency; (b) point estimation/prediction accuracy; and (c) allowing for the construction of practically useful confidence and prediction intervals.

stat.ML↗

arXiv2023

Multi-step ahead prediction intervals for non-parametric autoregressions via bootstrap: consistency, debiasing and pertinence

To address the difficult problem of multi-step ahead prediction of non-parametric autoregressions, we consider a forward bootstrap approach. Employing a local constant estimator, we can analyze a general type of non-parametric time series model, and show that the proposed point predictions are consistent with the true optimal predictor. We construct a quantile prediction interval that is asymptotically valid. Moreover, using a debiasing technique, we can asymptotically approximate the distribution of multi-step ahead non-parametric estimation by bootstrap. As a result, we can build bootstrap prediction intervals that are pertinent, i.e., can capture the model estimation variability, thus improving upon the standard quantile prediction intervals. Simulation studies are given to illustrate the performance of our point predictions and pertinent prediction intervals for finite samples.

stat.ME↗

arXiv2023

Bootstrap Prediction Inference of Non-linear Autoregressive Models

The non-linear autoregressive (NLAR) model plays an important role in modeling and predicting time series. One-step ahead prediction is straightforward using the NLAR model, but the multi-step ahead prediction is cumbersome. For instance, iterating the one-step ahead predictor is a convenient strategy for linear autoregressive (LAR) models, but it is suboptimal under NLAR. In this paper, we first propose a simulation and/or bootstrap algorithm to construct optimal point predictors under an $L_1$ or $L_2$ loss criterion. In addition, we construct bootstrap prediction intervals in the multi-step ahead prediction problem; in particular, we develop an asymptotically valid quantile prediction interval as well as a pertinent prediction interval for future values. In order to correct the undercoverage of prediction intervals with finite samples, we further employ predictive -- as opposed to fitted -- residuals in the bootstrap process. Simulation studies are also given to substantiate the finite sample performance of our methods.

stat.ME↗

arXiv2023

Simultaneous Statistical Inference for Second Order Parameters of Time Series under Weak Conditions

Strict stationarity is a common assumption used in the time series literature in order to derive asymptotic distributional results for second-order statistics, like sample autocovariances and sample autocorrelations. Focusing on weak stationarity, this paper derives the asymptotic distribution of the maximum of sample autocovariances and sample autocorrelations under weak conditions by using Gaussian approximation techniques. The asymptotic theory for parameter estimation obtained by fitting a (linear) autoregressive model to a general weakly stationary time series is revisited and a Gaussian approximation theorem for the maximum of the estimators of the autoregressive coefficients is derived. To perform statistical inference for the second order parameters considered, a bootstrap algorithm, the so-called second-order wild bootstrap, is applied. Consistency of this bootstrap procedure is proven. In contrast to existing bootstrap alternatives, validity of the second-order wild bootstrap does not require the imposition of strict stationary conditions or structural process assumptions, like linearity. The good finite sample performance of the second-order wild bootstrap is demonstrated by means of simulations.

math.ST↗

arXiv2022

Model-Based and Model-Free point prediction algorithms for locally stationary random fields

The Model-free Prediction Principle has been successfully applied to general regression problems, as well as problems involving stationary and locally stationary time series. In this paper we demonstrate how Model-Free Prediction can be applied to handle random fields that are only locally stationary, i.e., they can be assumed to be stationary only across a limited part over their entire region of definition. We construct one-step-ahead point predictors and compare the performance of Model-free to Model-based prediction using models that incorporate a trend and/or heteroscedasticity. Both aspects of the paper, Model-free and Model-based, are novel in the context of random fields that are locally (but not globally) stationary. We demonstrate the application of our Model-based and Model-free point prediction methods to synthetic data as well as images from the CIFAR-10 dataset and in the latter case show that our best Model-free point prediction results outperform those obtained using Model-based prediction.

stat.ME↗

arXiv2022

Local Quadratic Spectral and Covariance Matrix Estimation

The problem of estimating the spectral density matrix $f(w)$ of a multivariate time series is revisited with special focus on the frequencies $w=0$ and $w=π$. Recognizing that the entries of the spectral density matrix at these two boundary points are real-valued, we propose a new estimator constructed from a local polynomial regression of the real portion of the multivariate periodogram. The case $w=0$ is of particular importance, since $f(0)$ is associated with the large-sample covariance matrix of the sample mean; hence, estimating $f(0)$ is crucial in order to conduct any sort of statistical inference on the mean. We explore the properties of the local polynomial estimator through theory and simulations, and discuss an application to inflation and unemployment.

stat.ME↗

arXiv2021

Model-free Bootstrap Prediction Regions for Multivariate Time Series

In Das and Politis(2020), a model-free bootstrap(MFB) paradigm was proposed for generating prediction intervals of univariate, (locally) stationary time series. Theoretical guarantees for this algorithm was resolved in Wang and Politis(2019) under stationarity and weak dependence condition. Following this line of work, here we extend MFB for predictive inference under a multivariate time series setup. We describe two algorithms, the first one works for a particular class of time series under any fixed dimension d; the second one works for a more generalized class of time series under low-dimensional setting. We justify our procedure through theoretical validity and simulation performance.

stat.ME↗

arXiv2021

Scalable subsampling: computation, aggregation and inference

Subsampling is a general statistical method developed in the 1990s aimed at estimating the sampling distribution of a statistic $\hat θ_n$ in order to conduct nonparametric inference such as the construction of confidence intervals and hypothesis tests. Subsampling has seen a resurgence in the Big Data era where the standard, full-resample size bootstrap can be infeasible to compute. Nevertheless, even choosing a single random subsample of size $b$ can be computationally challenging with both $b$ and the sample size $n$ being very large. In the paper at hand, we show how a set of appropriately chosen, non-random subsamples can be used to conduct effective -- and computationally feasible -- distribution estimation via subsampling. Further, we show how the same set of subsamples can be used to yield a procedure for subsampling aggregation -- also known as subagging -- that is scalable with big data. Interestingly, the scalable subagging estimator can be tuned to have the same (or better) rate of convergence as compared to $\hat θ_n$. The paper is concluded by showing how to conduct inference, e.g., confidence intervals, based on the scalable subagging estimator instead of the original $\hat θ_n$.

math.ST↗

arXiv2021

Debiased and threshold ridge regression for linear model with heteroskedastic and dependent error

Focusing on a high dimensional linear model $y = Xβ+ ε$ with dependent, non-stationary, and heteroskedastic errors, this paper applies the debiased and threshold ridge regression method that gives a consistent estimator for linear combinations of $β$; and derives a Gaussian approximation theorem for the estimator. Besides, it proposes a dependent wild bootstrap algorithm to construct the estimator's confidence intervals and perform hypothesis testing. Numerical experiments on the proposed estimator and the bootstrap algorithm show that they have favorable finite sample performance. Research on a high dimensional linear model with dependent(non-stationary) errors is sparse, and our work should bring some new insights to this field.

math.ST↗

arXiv2021

Model-free Bootstrap and Conformal Prediction in Regression: Conditionality, Conjecture Testing, and Pertinent Prediction Intervals

Predictive inference under a general regression setting is gaining more interest in the big-data era. In terms of going beyond point prediction to develop prediction intervals, two main threads of development are conformal prediction and Model-free prediction. Recently, a new conformal prediction approach was proposed that exploits the same uniformization procedure as in the well-known Model-free Bootstrap. Hence, it is of interest to compare and further investigate the performance of the two methods. In the paper at hand, we contrast the two approaches via theoretical analysis and numerical experiments with a focus on conditional coverage of prediction intervals. We discuss suitable scenarios for applying each algorithm, underscore the importance of conditional vs. unconditional coverage, and show that, under mild conditions, the Model-free bootstrap yields prediction intervals with guaranteed better conditional coverage compared to quantile estimation. We also extend the concept of 'pertinence' of prediction intervals to the nonparametric regression setting, and give concrete examples where its importance emerges under finite sample scenarios. Finally, we define the new notion of 'conjecture testing' that is the analog of hypothesis testing as applied to the prediction problem; we also devise a modified conformal score to allow conformal prediction to handle one-sided 'conjecture tests', and compare to the Model-free bootstrap.

math.ST↗

arXiv2021

Ridge Regression Revisited: Debiasing, Thresholding and Bootstrap

The success of the Lasso in the era of high-dimensional data can be attributed to its conducting an implicit model selection, i.e., zeroing out regression coefficients that are not significant. By contrast, classical ridge regression can not reveal a potential sparsity of parameters, and may also introduce a large bias under the high-dimensional setting. Nevertheless, recent work on the Lasso involves debiasing and thresholding, the latter in order to further enhance the model selection. As a consequence, ridge regression may be worth another look since -- after debiasing and thresholding -- it may offer some advantages over the Lasso, e.g., it can be easily computed using a closed-form expression. % and it has similar performance to threshold Lasso. In this paper, we define a debiased and thresholded ridge regression method, and prove a consistency result and a Gaussian approximation theorem. We further introduce a wild bootstrap algorithm to construct confidence regions and perform hypothesis testing for a linear combination of parameters. In addition to estimation, we consider the problem of prediction, and present a novel, hybrid bootstrap algorithm tailored for prediction intervals. Extensive numerical simulations further show that the debiased and thresholded ridge regression has favorable finite sample performance and may be preferable in some settings.

math.ST↗

arXiv2021

Bootstrap prediction intervals with asymptotic conditional validity and unconditional guarantees

It can be argued that optimal prediction should take into account all available data. Therefore, to evaluate a prediction interval's performance one should employ conditional coverage probability, conditioning on all available observations. Focusing on a linear model, we derive the asymptotic distribution of the difference between the conditional coverage probability of a nominal prediction interval and the conditional coverage probability of a prediction interval obtained via a residual-based bootstrap. Applying this result, we show that a prediction interval generated by the residual-based bootstrap has approximately 50% probability to yield conditional under-coverage. We then develop a new bootstrap algorithm that generates a prediction interval that asymptotically controls both the conditional coverage probability as well as the possibility of conditional under-coverage. We complement the asymptotic results with several finite-sample simulations.

math.ST↗

arXiv2020

Semi-parametric estimation and prediction intervals in state space models

Literatures in state space models focus on parametric inference and prediction, which fail if the state space model is not fully specified and the maximum likelihood estimation does not work. In this paper, we assume the state transition matrix and the distribution of state noises are unknown. Under this assumption, we provide methods to consistently estimate these terms. In addition, we introduce an algorithm to construct consistent prediction intervals for state vectors and future observations. We complement the asymptotic results with several numerical experiments.

math.ST↗

arXiv2019

Model-free Bootstrap for a General Class of Stationary Time Series

A model-free bootstrap procedure for a general class of stationary time series is introduced. The theoretical framework is established, showing asymptotic validity of bootstrap confidence intervals for many statistics of interest. In addition, asymptotic validity of one-step ahead bootstrap prediction intervals is also demonstrated. Finite-sample experiments are conducted to empirically confirm the performance of the new method, and to compare with popular methods such as the block bootstrap and the autoregressive (AR)-sieve bootstrap.

math.ST↗

arXiv2019

Bootstrap Seasonal Unit Root Test under Periodic Variation

Both seasonal unit roots and periodic variation can be prevalent in seasonal data. When testing seasonal unit roots under periodic variation, the validity of the existing methods, such as the HEGY test, remains unknown. This paper analyzes the behavior of the augmented HEGY test and the unaugmented HEGY test under periodic variation. It turns out that the asymptotic null distributions of the HEGY statistics testing the single roots at $1$ or $-1$ when there is periodic variation are identical to the asymptotic null distributions when there is no periodic variation. On the other hand, the asymptotic null distributions of the statistics testing any coexistence of roots at $1$, $-1$, $i$, or $-i$ when there is periodic variation are non-standard and are different from the asymptotic null distributions when there is no periodic variation. Therefore, when periodic variation exists, HEGY tests are not directly applicable to the joint tests for any concurrence of seasonal unit roots. As a remedy, bootstrap is proposed; in particular, the augmented HEGY test with seasonal independent and identically distributed (iid) bootstrap and the unaugmented HEGY test with seasonal block bootstrap are implemented. The consistency of these bootstrap procedures is established. The finite-sample behavior of these bootstrap tests is illustrated via simulation and prevails over their competitors'. Finally, these bootstrap tests are applied to detect the seasonal unit roots in various economic time series.

stat.ME↗