arXiv subjects

H. N. Mhaskar

Publications and source records attributed to H. N. Mhaskar.

At least 19 recordsLinked to original sources

arXiv2026

Spectral Bounds for Kernel Quadrature

A bottleneck in the theory of kernel methods in machine learning is the storage requirement. To ameliorate this, a standard trick is to replace the kernel with an explicit feature map. Perhaps, the most well known example is the Gaussian kernel which can be expressed in terms of the Fourier features. Analytically, the kernel $K$ can be expressed in terms of an integral expression that involves a possibly asymmetric kernel $G$ representing the feature map. Numerically, one needs to approximate this integral by a suitable numerical integration scheme, typically Monte Carlo. In this paper, we demonstrate that the eigenvalues of $K$ are approximated much better by the eigenvalues of the kernel obtained by discretizing the integral using suitable quadrature formulas instead. We illustrate this fact in the case of the Gaussian kernel and neural tangent kernels on the unit sphere of a four-dimensional Euclidean space corresponding to the sigmoid and ReLU activation functions.

math.NA↗

arXiv2026

A signal separation view of classification

The problem of classification in machine learning has often been approached in terms of function approximation. In this paper, we propose an alternative approach for classification in arbitrary compact metric spaces which, in theory, yields both the number of classes, and a perfect classification using a minimal number of queried labels. Our approach uses localized trigonometric polynomial kernels initially developed for the point source signal separation problem in signal processing. Rather than point sources, we argue that the various classes come from different probability measures. The localized kernel technique developed for separating point sources is then shown to separate the supports of these distributions. This is done in a hierarchical manner in our MASC algorithm to accommodate touching/overlapping class boundaries. We illustrate our theory on several simulated and real life datasets, including the Salinas and Indian Pines hyperspectral datasets and a document dataset.

cs.LG↗

arXiv2025

Robust and tractable multidimensional exponential analysis

Motivated by a number of applications in signal processing, we study the following question. Given samples of a multidimensional signal of the form $$ f(\boldsymbol\ell)=\sum_{k=1}^K a_k\exp(-i\langle \boldsymbol\ell, \mathbf{w}_k\rangle), \quad \mathbf{w}_1,\cdots,\mathbf{w}_k\in\mathbb{R}^q, \ \boldsymbol\ell\in \mathbb{Z}^q, \ |\boldsymbol\ell| <n, $$ determine the values of the number $K$ of components, and the parameters $a_k$ and $\mathbf{w}_k$'s. We note that the the number of samples of $f$ in the above equation is $(2n-1)^q$. We develop an algorithm to recuperate these quantities accurately using only a subsample of size $\mathcal{O}(qn)$ of this data. For this purpose, we use a novel localized kernel method to identify the parameters, including the number $K$ of signals. Our method is easy to implement, and is shown to be stable under a very low SNR range. We demonstrate the effectiveness of our resulting algorithm using 2 and 3 dimensional examples from the literature, and show substantial improvements over state-of-the-art techniques including Prony based, MUSIC and ESPRIT approaches.

eess.SP↗

arXiv2024

Learning on manifolds without manifold learning

Function approximation based on data drawn randomly from an unknown distribution is an important problem in machine learning. The manifold hypothesis assumes that the data is sampled from an unknown submanifold of a high dimensional Euclidean space. A great deal of research deals with obtaining information about this manifold, such as the eigendecomposition of the Laplace-Beltrami operator or coordinate charts, and using this information for function approximation. This two-step approach implies some extra errors in the approximation stemming from estimating the basic quantities of the data manifold in addition to the errors inherent in function approximation. In this paper, we project the unknown manifold as a submanifold of an ambient hypersphere and study the question of constructing a one-shot approximation using a specially designed sequence of localized spherical polynomial kernels on the hypersphere. Our approach does not require preprocessing of the data to obtain information about the manifold other than its dimension. We give optimal rates of approximation for relatively ``rough'' functions.

cs.LG↗

arXiv2023

Local transfer learning from one data space to another

A fundamental problem in manifold learning is to approximate a functional relationship in a data chosen randomly from a probability distribution supported on a low dimensional sub-manifold of a high dimensional ambient Euclidean space. The manifold is essentially defined by the data set itself and, typically, designed so that the data is dense on the manifold in some sense. The notion of a data space is an abstraction of a manifold encapsulating the essential properties that allow for function approximation. The problem of transfer learning (meta-learning) is to use the learning of a function on one data set to learn a similar function on a new data set. In terms of function approximation, this means lifting a function on one data space (the base data space) to another (the target data space). This viewpoint enables us to connect some inverse problems in applied mathematics (such as inverse Radon transform) with transfer learning. In this paper we examine the question of such lifting when the data is assumed to be known only on a part of the base data space. We are interested in determining subsets of the target data space on which the lifting can be defined, and how the local smoothness of the function and its lifting are related.

cs.LG↗

arXiv2023

Numerical solutions to an inverse problem for a non-linear Helmholtz equation

In this work, we construct numerical solutions to an inverse problem of a nonlinear Helmholtz equation defined in a spherical shell between two concentric spheres centered at the origin.Assuming that the values of the forward problem are known at sufficiently many points, we would like to determine the form of the non-linear term on the right-hand side of the equation via its Chebyshev coefficients.

math.NA↗

arXiv2021

A function approximation approach to the prediction of blood glucose levels

The problem of real time prediction of blood glucose (BG) levels based on the readings from a continuous glucose monitoring (CGM) device is a problem of great importance in diabetes care, and therefore, has attracted a lot of research in recent years, especially based on machine learning. An accurate prediction with a 30, 60, or 90 minute prediction horizon has the potential of saving millions of dollars in emergency care costs. In this paper, we treat the problem as one of function approximation, where the value of the BG level at time $t+h$ (where $h$ the prediction horizon) is considered to be an unknown function of $d$ readings prior to the time $t$. This unknown function may be supported in particular on some unknown submanifold of the $d$-dimensional Euclidean space. While manifold learning is classically done in a semi-supervised setting, where the entire data has to be known in advance, we use recent ideas to achieve an accurate function approximation in a supervised setting; i.e., construct a model for the target function. We use the state-of-the-art clinically relevant PRED-EGA grid to evaluate our results, and demonstrate that for a real life dataset, our method performs better than a standard deep network, especially in hypoglycemic and hyperglycemic regimes. One noteworthy aspect of this work is that the training data and test data may come from different distributions.

cs.LG↗

arXiv2021

A low discrepancy sequence on graphs

Many applications such as election forecasting, environmental monitoring, health policy, and graph based machine learning require taking expectation of functions defined on the vertices of a graph. We describe a construction of a sampling scheme analogous to the so called Leja points in complex potential theory that can be proved to give low discrepancy estimates for the approximation of the expected value by the impirical expected value based on these points. In contrast to classical potential theory where the kernel is fixed and the equilibrium distribution depends upon the kernel, we fix a probability distribution and construct a kernel (which represents the graph structure) for which the equilibrium distribution is the given probability distribution. Our estimates do not depend upon the size of the graph.

cs.LG↗

arXiv2019

Function approximation by deep networks

We show that deep networks are better than shallow networks at approximating functions that can be expressed as a composition of functions described by a directed acyclic graph, because the deep networks can be designed to have the same compositional structure, while a shallow network cannot exploit this knowledge. Thus, the blessing of compositionality mitigates the curse of dimensionality. On the other hand, a theorem called good propagation of errors allows to `lift' theorems about shallow networks to those about deep networks with an appropriate choice of norms, smoothness, etc. We illustrate this in three contexts where each channel in the deep network calculates a spherical polynomial, a non-smooth ReLU network, or another zonal function network related closely with the ReLU network.

cs.LG↗

arXiv2019

Super-resolution meets machine learning: approximation of measures

The problem of super-resolution in general terms is to recuperate a finitely supported measure $μ$ given finitely many of its coefficients $\hatμ(k)$ with respect to some orthonormal system. The interesting case concerns situations, where the number of coefficients required is substantially smaller than a power of the reciprocal of the minimal separation among the points in the support of $μ$. In this paper, we consider the more severe problem of recuperating $μ$ approximately without any assumption on $μ$ beyond having a finite total variation. In particular, $μ$ may be supported on a continuum, so that the minimal separation among the points in the support of $μ$ is $0$. A variant of this problem is also of interest in machine learning as well as the inverse problem of de-convolution. We define an appropriate notion of a distance between the target measure and its recuperated version, give an explicit expression for the recuperation operator, and estimate the distance between $μ$ and its approximation. We show that these estimates are the best possible in many different ways. We also explain why for a finitely supported measure the approximation quality of its recuperation is bounded from below if the amount of information is smaller than what is demanded in the super-resolution problem.

math.FA↗

arXiv2019

A witness function based construction of discriminative models using Hermite polynomials

In machine learning, we are given a dataset of the form $\{(\mathbf{x}_j,y_j)\}_{j=1}^M$, drawn as i.i.d. samples from an unknown probability distribution $μ$; the marginal distribution for the $\mathbf{x}_j$'s being $μ^*$. We propose that rather than using a positive kernel such as the Gaussian for estimation of these measures, using a non-positive kernel that preserves a large number of moments of these measures yields an optimal approximation. We use multi-variate Hermite polynomials for this purpose, and prove optimal and local approximation results in a supremum norm in a probabilistic sense. Together with a permutation test developed with the same kernel, we prove that the kernel estimator serves as a `witness function' in classification problems. Thus, if the value of this estimator at a point $\mathbf{x}$ exceeds a certain threshold, then the point is reliably in a certain class. This approach can be used to modify pretrained algorithms, such as neural networks or nonlinear dimension reduction techniques, to identify in-class vs out-of-class regions for the purposes of generative models, classification uncertainty, or finding robust centroids. This fact is demonstrated in a number of real world data sets including MNIST, CIFAR10, Science News documents, and LaLonde data sets.

cs.LG↗

arXiv2017

Minimum Sobolev norm interpolation of derivative data

We study the problem of reconstructing a function on a manifold satisfying some mild conditions, given data on the values and some derivatives of the function at arbitrary points on the manifold. While the problem of finding a polynomial of two variables with total degree $\le n$ given the values of the polynomial and some of its derivatives at exactly the same number of points as the dimension of the polynomial space is sometimes impossible, we show that such a problem always has a solution in a very general situation if the degree of the polynomials is sufficiently large. We give estimates on how large the degree should be, and give explicit constructions for such a polynomial even in a far more general case. As the number of sampling points at which the data is available increases, our polynomials converge to the target function on the set where the sampling points are dense. Numerical examples in single and double precision show that this method is stable and of high-order.

math.NA↗

arXiv2017

A deep learning approach to diabetic blood glucose prediction

We consider the question of 30-minute prediction of blood glucose levels measured by continuous glucose monitoring devices, using clinical data. While most studies of this nature deal with one patient at a time, we take a certain percentage of patients in the data set as training data, and test on the remainder of the patients; i.e., the machine need not re-calibrate on the new patients in the data set. We demonstrate how deep learning can outperform shallow networks in this example. One novelty is to demonstrate how a parsimonious deep representation can be constructed using domain knowledge.

cs.LG↗

arXiv2016

Representation of functions on big data associated with directed graphs

This paper is an extension of the previous work of Chui, Filbir, and Mhaskar (Appl. Comput. Harm. Anal. 38 (3) 2015:489-509), not only from numeric data to include non-numeric data as in that paper, but also from undirected graphs to directed graphs (called digraphs, for simplicity). Besides theoretical development, this paper introduces effective mathematical tools in terms of certain data-dependent orthogonal systems for function representation and analysis directly on the digraphs. In addition, this paper also includes algorithmic development and discussion of various experimental results on such data-sets as CORA, Proposition, and Wiki-votes.

math.CA↗

arXiv2016

Local approximation using Hermite functions

We develop a wavelet like representation of functions in $L^p(\mathbb{R})$ based on their Fourier--Hermite coefficients; i.e., we describe an expansion of such functions where the local behavior of the terms characterize completely the local smoothness of the target function. In the case of continuous functions, a similar expansion is given based on the values of the functions at arbitrary points on the real line. In the process, we give new proofs for the localization of certain kernels, as well as some very classical estimates such as the Markov--Bernstein inequality.

math.CA↗

arXiv2016

Deep nets for local manifold learning

The problem of extending a function $f$ defined on a training data $\mathcal{C}$ on an unknown manifold $\mathbb{X}$ to the entire manifold and a tubular neighborhood of this manifold is considered in this paper. For $\mathbb{X}$ embedded in a high dimensional ambient Euclidean space $\mathbb{R}^D$, a deep learning algorithm is developed for finding a local coordinate system for the manifold {\bf without eigen--decomposition}, which reduces the problem to the classical problem of function approximation on a low dimensional cube. Deep nets (or multilayered neural networks) are proposed to accomplish this approximation scheme by using the training data. Our methods do not involve such optimization techniques as back--propagation, while assuring optimal (a priori) error bounds on the output in terms of the number of derivatives of the target function. In addition, these methods are universal, in that they do not require a prior knowledge of the smoothness of the target function, but adjust the accuracy of approximation locally and automatically, depending only upon the local smoothness of the target function. Our ideas are easily extended to solve both the pre--image problem and the out--of--sample extension problem, with a priori bounds on the growth of the function thus extended.

cs.LG↗

arXiv2010

Marcinkiewicz--Zygmund measures on manifolds

Let ${\mathbb X}$ be a compact, connected, Riemannian manifold (without boundary), $ρ$ be the geodesic distance on ${\mathbb X}$, $μ$ be a probability measure on ${\mathbb X}$, and $\{ϕ_k\}$ be an orthonormal system of continuous functions, $ϕ_0(x)=1$ for all $x\in{\mathbb X}$, $\{\ell_k\}_{k=0}^\infty$ be an nondecreasing sequence of real numbers with $\ell_0=1$, $\ell_k\uparrow\infty$ as $k\to\infty$, $Π_L:={\mathsf {span}}\{ϕ_j : \ell_j\le L\}$, $L\ge 0$. We describe conditions to ensure an equivalence between the $L^p$ norms of elements of $Π_L$ with their suitably discretized versions. We also give intrinsic criteria to determine if any system of weights and nodes allows such inequalities. The results are stated in a very general form, applicable for example, when the discretization of the integrals is based on weighted averages of the elements of $Π_L$ on geodesic balls rather than point evaluations.

math.CA↗

arXiv2010

A construction of linear bounded interpolatory operators on the torus

Let $q\ge 1$ be an integer. Given $M$ samples of a smooth function of $q$ variables, $2π$--periodic in each variable, we consider the problem of constructing a $q$--variate trigonometric polynomial of spherical degree $Ø(M^{1/q})$ which interpolates the given data, remains bounded (independent of $M$) on $[-π,π]^q$, and converges to the function at an optimal rate on the set where the data becomes dense. We prove that the solution of an appropriate optimization problem leads to such an interpolant. Numerical examples are given to demonstrate that this procedure overcomes the Runge phenomenon when interpolation at equidistant nodes on $[-1,1]$ is constructed, and also provides a respectable approximation for bivariate grid data, which does not become dense on the whole domain.

math.NA↗