arXiv subjects

Ioannis Kalogridis

Publications and source records attributed to Ioannis Kalogridis.

12 recordsLinked to original sources

arXiv2026

Generalized nonparametric regression in reproducing kernel Hilbert spaces: Consistency and rates of convergence

We develop a comprehensive theory for regularized M-estimation in reproducing kernel Hilbert spaces. Under mild conditions on the loss we establish existence and measurability of the estimator, covering a wide range of convex and non-convex losses, including bounded robust losses. We further prove sharp rates of convergence with an explicit bias-variance decomposition governed by a novel complexity measure. We show that the variance is independent of misspecification, while the bias depends on a source condition parameter known in the learning literature. For tensor product Sobolev spaces we obtain new rates that connect to spaces of functions with dominating mixed smoothness, substantially extending existing results and explaining why these estimators circumvent the curse of dimensionality. Our methodology, combining elements from both functional analysis and empirical process theory, allows for an asymptotic linearisation of the objective function that avoids both closed-form solutions and global Lipschitz assumptions, and may be of independent interest. The estimators are implemented in C++ and theory is supported by numerical experiments.

math.ST↗

arXiv2025

Penalized Spline M-Estimators for Discretely Sampled Functional Data: Existence and Asymptotics

Location estimation is a central problem in functional data analysis. In this paper, we investigate penalized spline estimators of location for discretely sampled functional data under a broad class of convex loss functions. Our framework generalizes and extends previously derived results for non-robust estimators to a broad penalized M-estimation framework. The analysis is built on two general-purpose non-asymptotic theoretical tools: (i) a non-asymptotic existence result for penalized spline estimators under minimal design conditions, and (ii) a localization lemma that captures both stochastic variability and approximation error. Under mild assumptions, we establish optimal convergence rates and identify the small- and large-knot regimes, along with the critical breakpoint known from penalized spline theory in nonparametric regression. Our results imply that parametric rates are attainable even with discretely sampled data and numerical experiments demonstrate that our estimators match or exceed the robustness of competing estimators while being considerably more computationally attractive.

stat.ME↗

arXiv2025

Asymptotics for M-type smoothing splines with non-smooth objective functions

M-type smoothing splines are a broad class of spline estimators that include the popular least-squares smoothing spline but also spline estimators that are less susceptible to outlying observations and model-misspecification. However, available asymptotic theory only covers smoothing spline estimators based on smooth objective functions and consequently leaves out frequently used resistant estimators such as quantile and Huber-type smoothing splines. We provide a general treatment in this paper and, assuming only the convexity of the objective function, show that the least-squares (super-)convergence rates can be extended to M-type estimators whose asymptotic properties have not been hitherto described. We further show that auxiliary scale estimates may be handled under significantly weaker assumptions than those found in the literature and we establish optimal rates of convergence for the derivatives, which have not been obtained outside the least-squares framework. A simulation study and a real-data example illustrate the competitive performance of non-smooth M-type splines in relation to the least-squares spline on regular data and their superior performance on data that contain anomalies.

math.ST↗

arXiv2023

Robust Functional Regression with Discretely Sampled Predictors

The functional linear model is an important extension of the classical regression model allowing for scalar responses to be modeled as functions of stochastic processes. Yet, despite the usefulness and popularity of the functional linear model in recent years, most treatments, theoretical and practical alike, suffer either from (i) lack of resistance towards the many types of anomalies one may encounter with functional data or (ii) biases resulting from the use of discretely sampled functional data instead of completely observed data. To address these deficiencies, this paper introduces and studies the first class of robust functional regression estimators for partially observed functional data. The proposed broad class of estimators is based on thin-plate splines with a novel computationally efficient quadratic penalty, is easily implementable and enjoys good theoretical properties under weak assumptions. We show that, in the incomplete data setting, both the sample size and discretization error of the processes determine the asymptotic rate of convergence of functional regression estimators and the latter cannot be ignored. These theoretical properties remain valid even with multi-dimensional random fields acting as predictors and random smoothing parameters. The effectiveness of the proposed class of estimators in practice is demonstrated by means of a simulation study and a real-data example.

stat.ME↗

arXiv2023

Robust thin-plate splines for multivariate spatial smoothing

We propose a novel family of multivariate robust smoothers based on the thin-plate (Sobolev) penalty that is particularly suitable for the analysis of spatial data. The proposed family of estimators can be expediently computed even in high dimensions, is invariant with respect to rigid transformations of the coordinate axes and can be shown to possess optimal theoretical properties under mild assumptions. The competitive performance of the proposed thin-plate spline estimators relative to its non-robust counterpart is illustrated in a simulation study and a real data example involving two-dimensional geographical data on ozone concentration.

stat.ME↗

arXiv2023

Robust and Adaptive Functional Logistic Regression

We introduce and study a family of robust estimators for the functional logistic regression model whose robustness automatically adapts to the data thereby leading to estimators with high efficiency in clean data and a high degree of resistance towards atypical observations. The estimators are based on the concept of density power divergence between densities and may be formed with any combination of lower rank approximations and penalties, as the need arises. For these estimators we prove uniform convergence and high rates of convergence with respect to the commonly used prediction error under fairly general assumptions. The highly competitive practical performance of our proposal is illustrated on a simulation study and a real data example which includes atypical observations.

stat.ME↗

arXiv2023

Robust and efficient estimation of nonparametric generalized linear models

Generalized linear models are flexible tools for the analysis of diverse datasets, but the classical formulation requires that the parametric component is correctly specified and the data contain no atypical observations. To address these shortcomings we introduce and study a family of nonparametric full rank and lower rank spline estimators that result from the minimization of a penalized power divergence. The proposed class of estimators is easily implementable, offers high protection against outlying observations and can be tuned for arbitrarily high efficiency in the case of clean data. We show that under weak assumptions these estimators converge at a fast rate and illustrate their highly competitive performance on a simulation study and two real-data examples.

stat.ME↗

arXiv2021

Robust penalized estimators for functional linear regression

Functional data analysis is a fast evolving branch of statistics. Estimation procedures for the popular functional linear model either suffer from lack of robustness or are computationally burdensome. To address these shortcomings, a flexible family of penalized lower-rank estimators based on a bounded loss function is proposed. The proposed class of estimators is shown to be consistent and can attain high rates of convergence with respect to prediction error under weak regularity conditions. These results can be generalized to higher dimensions under similar assumptions. The finite-sample performance of the proposed family of estimators is investigated by a Monte-Carlo study which shows that these estimators reach high efficiency while offering protection against outliers. The proposed estimators compare favorably to existing approaches robust as well as non-robust alternatives. The good performance of the method is also illustrated on a complex real dataset.

stat.ME↗

arXiv2021

Robust penalized spline estimation with difference penalties

Penalized spline estimation with discrete difference penalties (P-splines) is a popular estimation method for semiparametric models, but the classical least-squares estimator is highly sensitive to deviations from its ideal model assumptions. To remedy this deficiency, a broad class of P-spline estimators based on general loss functions is introduced and studied. Robust estimators are obtained by well-chosen loss functions, such as the Huber or Tukey loss function. A preliminary scale estimator can also be included in the loss function. It is shown that this class of P-spline estimators enjoys the same optimal asymptotic properties as least-squares P-splines, thereby providing strong theoretical motivation for its use. The proposed estimators may be computed very efficiently through a simple adaptation of well-established iterative least squares algorithms and exhibit excellent performance even in finite samples, as evidenced by a numerical study and a real-data example.

stat.ME↗

arXiv2021

Robust optimal estimation of location from discretely sampled functional data

Estimating location is a central problem in functional data analysis, yet most current estimation procedures either unrealistically assume completely observed trajectories or lack robustness with respect to the many kinds of anomalies one can encounter in the functional setting. To remedy these deficiencies we introduce the first class of optimal robust location estimators based on discretely sampled functional data. The proposed method is based on M-type smoothing spline estimation with repeated measurements and is suitable for both commonly and independently observed trajectories that are subject to measurement error. We show that under suitable assumptions the proposed family of estimators is minimax rate optimal both for commonly and independently observed trajectories and we illustrate its highly competitive performance and practical usefulness in a Monte-Carlo study and a real-data example involving recent Covid-19 data.

stat.ME↗

arXiv2020

M-type penalized splines with auxiliary scale estimation

Penalized spline smoothing is a popular and flexible method of obtaining estimates in nonparametric regression but the classical least-squares criterion is highly susceptible to model deviations and atypical observations. Penalized spline estimation with a resistant loss function is a natural remedy, yet to this day the asymptotic properties of M-type penalized spline estimators have not been studied. We show in this paper that M-type penalized spline estimators achieve the same rates of convergence as their least-squares counterparts, even with auxiliary scale estimation. We further find theoretical justification for the use of a small number of knots relative to the sample size. We illustrate the benefits of M-type penalized splines in a Monte-Carlo study and two real-data examples, which contain atypical observations.

stat.ME↗

arXiv2018

Robust functional regression based on principal components

Functional data analysis is a fast evolving branch of modern statistics and the functional linear model has become popular in recent years. However, most estimation methods for this model rely on generalized least squares procedures and therefore are sensitive to atypical observations. To remedy this, we propose a two-step estimation procedure that combines robust functional principal components and robust linear regression. Moreover, we propose a transformation that reduces the curvature of the estimators and can be advantageous in many settings. For these estimators we prove Fisher-consistency at elliptical distributions and consistency under mild regularity conditions. The influence function of the estimators is investigated as well. Simulation experiments show that the proposed estimators have reasonable efficiency, protect against outlying observations, produce smooth estimates and perform well in comparison to existing approaches.

stat.ME↗