arXiv subjects

Abderrazek Karoui

Publications and source records attributed to Abderrazek Karoui.

17 recordsLinked to original sources

arXiv2023

A Distribution Free Truncated Kernel Ridge Regression Estimator and Related Spectral Analyses

It is well known that kernel ridge regression (KRR) is a popular nonparametric regression estimator. Nonetheless, in the presence of a large data set with size $n\gg 1,$ the KRR estimator has the drawback to require an intensive computational load. Recently, scalable KRR approaches have been proposed with the aims to reduce the computational complexity of the KRR, while maintaining its superb convergence rate. In this work, we study a new scalable KRR based approach for nonparametric regression. Our truncated kernel ridge regression (TKRR) approach is simple. It is based on substituting the full $n\times n$ random kernel or Gram matrix $B_n,$ associated with a Mercer's kernel $\mathbb K,$ by its main $n\times N$ sub-matrix $A_N,$ where usually $N \ll n.$ Also, we show that the TKRR works with $d-$dimensional random sampling data following an unknown probability law. To do so, we give a spectral analysis for the compact kernel integral operator, associated with a probability measure, different from its usual probability measure. This decay estimate is then extended to the decay of the tail of the trace of the associated random Gram matrix. A special interest is devoted to develop rules for the optimal choices of the involved truncation order $N$ and the value for regularization parameter $λ>0.$ The proposed rules are based on the behavior and the decay rate of the spectrum of the positive integral operator, associated with the kernel $\mathbb K.$ These optimal values of the parameters ensure that in terms of the empirical risk error, the TKRR and the full KRR estimators have the same optimal convergence rate. Finally, we provide the reader with some numerical simulations that illustrate the performance of our proposed TKRR estimator.

math.ST↗

arXiv2022

A Stable Jacobi polynomials based least squares regression estimator associated with an ANOVA decomposition model

In this work, we construct a stable and fairly fast estimator for solving non-parametric multidimensional regression problems. The proposed estimator is based on the use of multivariate Jacobi polynomials that generate a basis for a reduced size of $d-$variate finite dimensional polynomial space. An ANOVA decomposition trick has been used for building this later polynomial space. Also, by using some results from the theory of positive definite random matrices, we show that the proposed estimator is stable under the condition that the i.i.d. random sampling points for the different covariates of the regression problem, follow a $d-$dimensional Beta distribution. Also, we provide the reader with an estimate for the $L^2-$risk error of the estimator. Moreover, a more precise estimate of the quality of the approximation is provided under the condition that the regression function belongs to some weighted Sobolev space. Finally, the various theoretical results of this work are supported by numerical simulations.

math.ST↗

arXiv2022

Multivariate nonparametric regression by least squares Jacobi polynomials approximations

In this work, we study a random orthogonal projection based least squares estimator for the stable solution of a multivariate nonparametric regression (MNPR) problem. More precisely, given an integer $d\geq 1$ corresponding to the dimension of the MNPR problem, a positive integer $N\geq 1$ and a real parameter $α\geq -\frac{1}{2},$ we show that a fairly large class of $d-$variate regression functions are well and stably approximated by its random projection over the orthonormal set of tensor product $d-$variate Jacobi polynomials with parameters $(α,α).$ The associated uni-variate Jacobi polynomials have degree at most $N$ and their tensor products are orthonormal over $\mathcal U=[0,1]^d,$ with respect to the associated multivariate Jacobi weights. In particular, if we consider $n$ random sampling points $\mathbf X_i$ following the $d-$variate Beta distribution, with parameters $(α+1,α+1),$ then we give a relation involving $n, N, α$ to ensure that the resulting $(N+1)^d\times (N+1)^d$ random projection matrix is well conditioned. Moreover, we provide squared integrated as well as $L^2-$risk errors of this estimator. Precise estimates of these errors are given in the case where the regression function belongs to an isotropic Sobolev space $H^s(I^d),$ with $s> \frac{d}{2}.$ Also, to handle the general and practical case of an unknown distribution of the $\mathbf X_i,$ we use Shepard's scattered interpolation scheme in order to generate fairly precise approximations of the observed data at $n$ i.i.d. sampling points $\mathbf X_i$ following a $d-$variate Beta distribution. Finally, we illustrate the performance of our proposed multivariate nonparametric estimator by some numerical simulations with synthetic as well as real data.

math.ST↗

arXiv2021

Random matrices based schemes for stable and robust nonparametric and functional regression estimators

In the first part of this work, we develop a novel scheme for solving nonparametric regression problems. That is the approximation of possibly low regular and noised functions from the knowledge of their approximate values given at some random points. Our proposed scheme is based on the use of the pseudo-inverse of a random projection matrix, combined with some specific properties of the Jacobi polynomials system, as well as some properties of positive definite random matrices. This scheme has the advantages to be stable, robust, accurate and fairly fast in terms of execution time. In particular, we provide an $L_2$ as well as an $L_2-$risk errors of our proposed nonparametric regression estimator. Moreover and unlike most of the existing nonparametric regression estimators, no extra regularization step is required by our proposed estimator. Although, this estimator is initially designed to work with random sampling set of uni-variate i.i.d. random variables following a Beta distribution, we show that it is still works for a wide range of sampling distribution laws. Moreover, we briefly describe how our estimator can be adapted in order to handle the multivariate case of random sampling sets. In the second part of this work, we extend the random pseudo-inverse scheme technique to build a stable and accurate estimator for solving linear functional regression (LFR) problems. A dyadic decomposition approach is used to construct this last stable estimator for the LFR problem. Alaso, we give an $L_2-$risk error of our proposed LFR estimator. Finally, the performance of the two proposed estimators are illustrated by various numerical simulations. In particular, a real dataset is used to illustrate the performance of our nonparametric regression estimator.

math.ST↗

arXiv2020

Reproducing kernels based schemes for nonparametric regression

In this work, we develop and study an empirical projection operator scheme for solving nonparametric regression problems. This scheme is based on an approximate projection of the regression function over a suitable reproducing kernel Hilbert space (RKHS). The RKHS considered in this paper are generated by the Mercer kernels given by the Legendre Christoffel-Darboux and convolution Sinc kernels. We provide error and convergence analysis of the proposed scheme under the assumption that the regression function belongs to some suitable functional spaces. We also consider the popular RKHS regularized least square minimization for nonparametric regression. In particular, we check the numerical stability of this second scheme and we provide its convergence rate in the special case of the Sinc kernel. Finally, we illustrate the proposed methods by various numerical simulation.

math.ST↗

arXiv2019

Random Discretization of the Finite Fourier Transform and Related Kernel Random Matrices

This paper is centred on the spectral study of a Random Fourier matrix, that is an $n\times n$ matrix $A$ whose $(j, k)$ entries are $\exp(2iπm X_jY_k)$, with $X_j$ and $Y_k$ two i.i.d sequences of random variables and $1\leq m\leq n$ is a real number. When they are uniformly distributed on a symmetric interval, this may be seen as a random discretization of the Finite Fourier transform, whose spectrum has been extensively studied in relation with band-limited functions. Our study is two-fold. Firstly, by pushing forward concentration inequalities, we find an accurate comparison in $\ell^2$- norm between the spectrum of $A^*A$ and the one of an integral operator that can be defined in terms of the two probability laws chosen for the rows and the columns. Our study includes the one of stationary Hermitian kernel matrices and can be generalized to non stationary ones, for which the same kind of comparison with an integral operator is possible. Because of possible applications in the data science area, these last matrices have been largely studied in the literature and our results are compared with previous ones. Secondly we concentrate on uniform distributions for the laws of $X_j$'s and $Y_k$'s, for which the integral operator is the well-known Sinc-kernel operator with parameter $m.$ Our previous study allows to translate to random Fourier matrices the knowledge that we have on the spectrum of this operator. We have for them asymptotic results for $m, n$ and $n/m$ tending to $\infty$, as well as non asymptotic bounds in the spirit of recent work on the integral operators. As an application, we give fairly good approximations of the number of degrees of freedom and the capacity of a MIMO wireless communication network approximation model. Finally, we provide the reader with some numerical examples that illustrate the theoretical results of this paper.

math.CA↗

arXiv2018

Weighted Finite Laplace Transform Operator: Spectral Analysis and Quality of Approximation by its Eigenfunctions

For two real numbers $c>0, α> -1,$ we study some spectral properties of the weighted finite bilateral Laplace transform operator, defined over the space $E=L^2(I,ω_α),$ $I=[-1,1],$ $ω_α(x)=(1-x^2)^α,$ by ${\displaystyle \mathcal L_c^α f(x)= \int_I e^{cxy} f(y) ω_α(y)\, dy}.$ In particular, we use a technique based on the Min-Max theorem to prove that the sequence of the eigenvalues of this operator has a super-exponential decay rate to zero. Moreover, we give a lower bound with a magnitude of order $e^c,$ for the largest eigenvalue of the operator $\mathcal L_c^α.$ Also, we give some local estimates and bounds of the eigenfunctions $φ_{n,c}^α$ of $\mathcal L_c^α.$ Moreover, we show that these eigenfunctions are good candidates for the spectral approximation of a function that can be written as a weighted finite Laplace transform of an other $L^2(I,ω_α)-$function. Finally, we give some numerical examples that illustrate the different results of this work. In particular, we provide an example that illustrate the Laplace based spectral method, for the inversion of the finite Laplace transform.

math.CA↗

arXiv2018

Non-Asymptotic behaviour of the spectrum of the Sinc Kernel Operator and Related Applications

Prolate spheroidal wave functions have recently attracted a lot of attention in applied harmonic analysis, signal processing and mathematical physics. They are eigenvectors of the Sinc-kernel operator Qc : the time-and band-limiting operator. The corresponding eigenvalues play a key role and it is the aim of this paper to obtain precise non-asymptotic estimates for these eigenvalues, within the three main regions of the spectrum of Qc. This issue is rarely studied in the literature, while the asymptotic behaviour of the spectrum of Qc has been well established from the sixties. As applications of our non-asymptotic estimates, we first provide estimates for the constants appearing in Remez and Tur{à}n-Nazarov type concentration inequalities. Then, we give an estimate for the hole probability, associated with a random matrix from the Gaussian Unitary Ensemble (GUE).

math.CA↗

arXiv2017

The finite Hankel transform operator: Some explicit and local estimates of the eigenfunctions and eigenvalues decay rates

For fixed real numbers $c>0,$ $α>-\frac{1}{2},$ the finite Hankel transform operator, denoted by $\mathcal{H}_c^α$ is given by the integral operator defined on $L^2(0,1)$ with kernel $K_α(x,y)= \sqrt{c xy} J_α(cxy).$ To the operator $\mathcal{H}_c^α,$ we associate a positive, self-adjoint compact integral operator $\mathcal Q_c^α=c\, \mathcal{H}_c^α\, \mathcal{H}_c^α.$ Note that the integral operators $\mathcal{H}_c^α$ and $\mathcal Q_c^α$ commute with a Sturm-Liouville differential operator $\mathcal D_c^α.$ In this paper, we first give some useful estimates and bounds of the eigenfunctions $\vp$ of $\mathcal H_c^α$ or $\mathcal Q_c^α.$ These estimates and bounds are obtained by using some special techniques from the theory of Sturm-Liouville operators, that we apply to the differential operator $\mathcal D_c^α.$ If $(μ_{n,α}(c))_n$ and $λ_{n,α}(c)=c\, |μ_{n,α}(c)|^2$ denote the infinite and countable sequence of the eigenvalues of the operators $\mathcal{H}_c^{(α)}$ and $\mathcal Q_c^α,$ arranged in the decreasing order of their magnitude, then we show an unexpected result that for a given integer $n\geq 0,$ $λ_{n,α}(c)$ is decreasing with respect to the parameter $α.$ As a consequence, we show that for $α\geq \frac{1}{2},$ the $λ_{n,α}(c)$ and the $μ_{n,α}(c)$ have a super-exponential decay rate. Also, we give a lower decay rate of these eigenvalues. As it will be seen, the previous results are essential tools for the analysis of a spectral approximation scheme based on the eigenfunctions of the finite Hankel transform operator. Some numerical examples will be provided to illustrate the results of this work.

math.CA↗

arXiv2015

Weighted finite Fourier transform operator: Uniform approximations of the eigenfunctions, eigenvalues decay and behaviour

In this paper, we first give two uniform asymptotic approximations of the eigenfunctions of the weighted finite Fourier transform operator, defined by ${\displaystyle \mathcal F_c^{(α)} f(x)=\int_{-1}^1 e^{icxy} f(y)\,(1-y^2)^α\, dy,\,}$ where $ c >0, α> -1$ are two fixed real numbers. The first uniform approximation is given in terms of a Bessel function, whereas the second one is given in terms of a normalized Jacobi polynomial. These eigenfunctions are called generalized prolate spheroidal wave functions (GPSWFs). By using the uniform asymptotic approximations of the GPSWFs, we prove the super-exponential decay rate of the eigenvalues of the operator $\mathcal F_c^{(α)}$ in the case where $0<α< 3/2.$ Finally, by computing the trace and an estimate of the norm of the operator ${\displaystyle \mathcal Q_c^α=\frac{c}{2π} \mathcal F_c^{α^*} \mathcal F_c^α,}$ we give a lower and an upper bound for the counting number of the eigenvalues of $Q_c^α,$ when $c>>1.$

math.CA↗

arXiv2015

Spectral Decay of Time and Frequency Limiting Operator

For fixed $c,$ the Prolate Spheroidal Wave Functions (PSWFs) $ψ_{n, c}$ form a basis with remarkable properties for the space of band-limited functions with bandwidth $c$. They have been largely studied and used after the seminal work of D. Slepian, H. Landau and H. Pollack. Many of the PSWFs applications rely heavily of the behavior and the decay rate of the eigenvalues $(λ_n(c))_{n\geq 0}$ of the time and frequency limiting operator, which we denote by $\mathcal Q_c.$ Hence, the issue of the accurate estimation of the spectrum of this operator has attracted a considerable interest, both in numerical and theoretical studies. In this work, we give an explicit integral approximation formula for these eigenvalues. This approximation holds true starting from the plunge region where the spectrum of $\mathcal Q_c$ starts to have a fast decay. As a consequence of our explicit approximation formula, we give a precise description of the super-exponential decay rate of the $λ_n(c).$ Also, we mention that the described approximation scheme provides us with fairly accurate approximations of the $λ_n(c)$ with low computational load, even for very large values of the parameters $c$ and $n.$ Finally, we provide the reader with some numerical examples that illustrate the different results of this work.

math.CA↗

arXiv2015

Approximations in Sobolev Spaces by Prolate Spheroidal Wave Functions

Recently, there is a growing interest in the spectral approximation by the Prolate Spheroidal Wave Functions (PSWFs) $ψ_{n, c},\, c>0.$ This is due to the promising new contributions of these functions in various classical as well as emerging applications from Signal Processing, Geophysics, Numerical Analysis, etc. The PSWFs form a basis with remarkable properties not only for the space of band-limited functions with bandwidth $c,$ but also for the Sobolev space $H^s([-1,1])$. The quality of the spectral approximation and the choice of the parameter $c$ when approximating a function in $H^s([-1,1])$ by its truncated PSWFs series expansion, are the main issues. By considering a function $f\in H^s([-1,1])$ as the restriction to $[-1,1]$ of an almost time-limited and band-limited function, we try to give satisfactory answers to these two issues. Also, we illustrate the different results of this work by some numerical examples.

math.CA↗

arXiv2015

Generalized Prolate Spheroidal Wave Functions: Spectral Analysis and Approximation of Almost Band-limited Functions

In this work, we first give various explicit and local estimates of the eigenfunctions of a perturbed Jacobi differential operator. These eigenfunctions generalize the famous classical prolate spheroidal wave functions (PSWFs), founded in 1960's by D. Slepian and his co-authors and corresponding to the case $α=β=0.$ They also generalize the new PSWFs introduced and studied recently in \cite{Wang2}, denoted by GPSWFs and corresponding to the case $α=β> -1.$ The main content of this work is devoted to the previous interesting special case $α=β.$ In particular, we give further computational improvements, as well as some useful explicit and local estimates of the GPSWFs. More importantly, by using the concept of a restricted Paley-Wiener space, we relate the GPSWFs to the solutions of a generalized energy maximisation problem. As a consequence, many desirable spectral properties of the self-adjoint compact integral operator associated with the GPSWFs are deduced from the rich literature of the PSWFs. In particular, we show that the GPSWFs are well adapted for the spectral approximation of the classical $c-$band-limited as well as almost $c-$band-limited functions. Finally, we provide the reader with some numerical examples that illustrate the different results of this work.

math.CA↗

arXiv2015

The approximation of almost time and band limited functions by their expansion in some orthogonal polynomials bases

The aim of this paper is to investigate the quality of approximation of almost time and almost band-limited functions by its expansion in three classical orthogonal polynomials bases: the Hermite, Legendre and Chebyshev bases. As a corollary, this allows us to obtain the quality of approximation in the L 2 --Sobolev space by these orthogonal polynomials bases. Also, we obtain the rate of the Legendre series expansion of the prolate spheroidal wave functions. Some numerical examples are given to illustrate the different results of this work.

math.CA↗

arXiv2014

Uniform approximation and explicit estimates for the prolate spheroidal wave functions

For fixed $c,$ Prolate Spheroidal Wave Functions (PSWFs), denoted by $ψ_{n, c},$ form an orthogonal basis with remarkable properties for the space of band-limited functions with bandwith $c$. They have been largely studied and used after the seminal work of D. Slepian and his co-authors. In several applications, uniform estimates of the $ψ_{n,c}$ in $n$ and $c,$ are needed. To progress in this direction, we push forward the uniform approximation error bounds and give an explicit approximation of their values at $1$ in terms of the Legendre complete elliptic integral of the first kind. Also, we give an explicit formula for the accurate approximation the eigenvalues of the Sturm-Liouville operator associated with the PSWFs.

math.CA↗

arXiv2014

Approximation of almost time and band limited functions I: Hermite expansions

The aim of this paper is to investigate the quality of approximation of almost time and band limited functions by its expansion in the Hermite and scaled Hermite basis. As a corollary, this allows us to obtain the rate of convergence of the Hermite expansion of function in the $L^2$-Sobolev space with fixed compact support.

math.CA↗

arXiv2014

Spectral decay of the sinc kernel operator and approximations by Prolate Spheroidal Wave Functions

For fixed $c,$ the Prolate Spheroidal Wave Functions (PSWFs) $ψ_{n, c}$ form a basis with remarkable properties for the space of band-limited functions with bandwidth $c$. They have been largely studied and used after the seminal work of D. Slepian, H. Landau and H. Pollack. Recently, they have been used for the approximation of functions in the Sobolev space $H^s([-1,1])$. In view of this, we give new estimates on the decay rate of eigenvalues of the Sinc kernel integral operators. This is one of the main issues of this work. A second one is the choice of the parameter $c$ when approximating a function in $H^s([-1,1])$ by its truncated PSWFs series expansion. Such functions may be seen as the restriction to $[-1,1]$ of almost time-limited and band-limited functions, for which PSWFs expansions are still well adapted. Finally, we provide the reader with some numerical examples that illustrate the different results of this work.

math.CA↗