New duality in choices of feature spaces via kernel analysis

Abstract

We present a systematic study of the family of positive definite (p.d.) kernels with the use of their associated feature maps and feature spaces. For a fixed set $X$, generalizing Loewner, we make precise the corresponding partially ordered set $Pos\left(X\right)$ of all p.d. kernels on $X$, as well as a study of its global properties. This new analysis includes both results dealing with applications and concrete examples, including such general notions for $Pos\left(X\right)$ as the structure of its partial order, its products, sums, and limits; as well as their Hilbert space-theoretic counterparts. For this purpose, we introduce a new duality for feature spaces, feature selections, and feature mappings. For our analysis, we further introduce a general notion of dual pairs of p.d. kernels. Three special classes of kernels are studied in detail: (a) the case when the reproducing kernel Hilbert spaces (RKHSs) may be chosen as Hilbert spaces of analytic functions, (b) when they are realized in spaces of Schwartz-distributions, and (c) arise as fractal limits. We further prove inverse theorems in which we derive results for the analysis of $Pos\left(X\right)$ from the operator theory of specified counterpart-feature spaces. We present constructions of new p.d. kernels in two ways: (i) as limits of monotone families in $Pos\left(X\right)$, and (ii) as p.d. kernels which model fractal limits, i.e., are invariant with respect to certain iterated function systems (IFS)-transformations.