arXiv subjects

Myung-Sin Song

Publications and source records attributed to Myung-Sin Song.

At least 19 recordsLinked to original sources

arXiv2026

The role of parameter Jacobians in the stability of network outputs

In the framework of network dynamics, learning models, and neural tangent kernels (NTK), we show that the corresponding linearized dynamics leads naturally to a semigroup formulation. More precisely, in our analysis of input/output models, the time-dynamics is presented via special semigroups of linear operators on Hilbert spaces, together with an associated class of semigroup perturbations. In this context, we then present new and explicit a priori perturbation-bound results: for the fixed-kernel linearization constructions arising in the NTK setting, we prove norm-bounds on the corresponding semigroup perturbations, in the form of explicit finite-time perturbation estimates. We further present refinements on prescribed task spaces, Cesàro-averaged (ergodic) comparisons estimates, and versions in which the lower spectral edge assumption is replaced by a spectral-distribution condition. We also extend the comparison to nonautonomous NTK evolutions through piecewise-frozen approximations, record a corresponding discrete Euler specialization, and offer worked examples in order to illustrate our perturbation-bound estimates.

math.FA↗

arXiv2026

Wavelet-Packet Content for Positive Operators

We study positive operator decompositions associated with rooted trees of orthogonal projections. In this sense, the refinement tree induces an ``MRA in $B\left(H\right)_{+}$''. To each node we assign a positive content operator, and these contents split along the tree and yield a positive decomposition at each fixed depth. The resulting decomposition gives a multiresolution description of positive operators adapted to the tree. In the trace class setting, the scalar contents determine a canonical boundary measure on the path space, and for each vector the corresponding quadratic data admit a nonnegative integrable density with respect to that measure. At fixed depth, we study greedy extraction rules based on trace and Hilbert-Schmidt norm. The trace rule gives a sharp geometric decay estimate for the trace of the positive remainder. In the Hilbert-Schmidt setting, a depth dependent coherence parameter measures departure from block diagonal form and yields geometric decay bounds. We also study adaptive partitions up to a terminal depth. In that setting, the change in total squared content under local refinement is determined by off-diagonal interaction among the child contents. This leads to an additive refinement calculus for adaptive decompositions and recursive criteria for optimal adaptive partitions.

math.FA↗

arXiv2026

Use of operator defect identities in multi-channel signal plus residual-analysis via iterated products and telescoping energy-residuals: Applications to kernels in machine learning

We present a new operator theoretic framework for analysis of complex systems with intrinsic subdivisions into components, taking the form of "residuals" in general, and "telescoping energy residuals" in particular. We prove new results which yield admissibility/effectiveness, and new a priori bounds on energy residuals. Applications include infinite-dimensional Kaczmarz theory for $λ_{n}$-relaxed variants, and $λ_{n}$-effectiveness. And we give applications of our framework to generalized machine learning algorithms, greedy Kernel Principal Component Analysis (KPCA), proving explicit convergence results, residual energy decomposition, and criteria for stability under noise.

math.FA↗

arXiv2025

Infinite-Dimensional Operator/Block Kaczmarz Algorithms: Regret Bounds and $λ$-Effectiveness

We present a variety of projection-based linear regression algorithms with a focus on modern machine-learning models and their algorithmic performance. We study the role of the relaxation parameter in generalized Kaczmarz algorithms and establish a priori regret bounds with explicit $λ$-dependence to quantify how much an algorithm's performance deviates from its optimal performance. A detailed analysis of relaxation parameter is also provided. Applications include: explicit regret bounds for the framework of Kaczmarz algorithm models, non-orthogonal Fourier expansions, and the use of regret estimates in modern machine learning models, including for noisy data, i.e., regret bounds for the noisy Kaczmarz algorithms. Motivated by machine-learning practice, our wider framework treats bounded operators (on infinite-dimensional Hilbert spaces), with updates realized as (block) Kaczmarz algorithms, leading to new and versatile results.

stat.ML↗

arXiv2023

Conditional mean embeddings and optimal feature selection via positive definite kernels

Motivated by applications, we consider here new operator theoretic approaches to Conditional mean embeddings (CME). Our present results combine a spectral analysis-based optimization scheme with the use of kernels, stochastic processes, and constructive learning algorithms. For initially given non-linear data, we consider optimization-based feature selections. This entails the use of convex sets of positive definite (p.d.) kernels in a construction of optimal feature selection via regression algorithms from learning models. Thus, with initial inputs of training data (for a suitable learning algorithm,) each choice of p.d. kernel $K$ in turn yields a variety of Hilbert spaces and realizations of features. A novel idea here is that we shall allow an optimization over selected sets of kernels $K$ from a convex set $C$ of positive definite kernels $K$. Hence our \textquotedblleft optimal\textquotedblright{} choices of feature representations will depend on a secondary optimization over p.d. kernels $K$ within a specified convex set $C$.

cs.LG↗

arXiv2023

Infinite-Dimensional Stochastic Transforms and Reproducing Kernel Hilbert space

By way of concrete presentations, we construct two infinite-dimensional transforms at the crossroads of Gaussian fields and reproducing kernel Hilbert spaces (RKHS), thus leading to a new infinite-dimensional Fourier transform in a general setting of Gaussian processes. Our results serve to unify existing tools from infinite-dimensional analysis.

math.FA↗

arXiv2023

Operator theory, kernels, and Feedforward Neural Networks

In this paper we show how specific families of positive definite kernels serve as powerful tools in analyses of iteration algorithms for multiple layer feedforward Neural Network models. Our focus is on particular kernels that adapt well to learning algorithms for data-sets/features which display intrinsic self-similarities at feedforward iterations of scaling.

cs.LG↗

arXiv2022

An Infinite Dimensional Analysis of Kernel Principal Components

We study non-linear data-dimension reduction. We are motivated by the classical linear framework of Principal Component Analysis. In nonlinear case, we introduce instead a new kernel-Principal Component Analysis, manifold and feature space transforms. Our results extend earlier work for probabilistic Karhunen-Loève transforms on compression of wavelet images. Our object is algorithms for optimization, selection of efficient bases, or components, which serve to minimize entropy and error; and hence to improve digital representation of images, and hence of optimal storage, and transmission. We prove several new theorems for data-dimension reduction. Moreover, with the use of frames in Hilbert space, and a new Hilbert-Schmidt analysis, we identify when a choice of Gaussian kernel is optimal.

math.FA↗

arXiv2021

Positive Definite Kernels, Algorithms, Frames, and Approximations

The main purpose of our paper is a new approach to design of algorithms of Kaczmarz type in the framework of operators in Hilbert space. Our applications include a diverse list of optimization problems, new Karhunen-Loève transforms, and Principal Component Analysis (PCA) for digital images. A key feature of our algorithms is our use of recursive systems of projection operators. Specifically, we apply our recursive projection algorithms for new computations of PCA probabilities and of variance data. For this we also make use of specific reproducing kernel Hilbert spaces, factorization for kernels, and finite-dimensional approximations. Our projection algorithms are designed with view to maximum likelihood solutions, minimization of "cost" problems, identification of principal components, and data-dimension reduction.

math.FA↗

arXiv2019

A Kaczmarz algorithm for sequences of projections, infinite products, and applications to frames in IFS $L^{2}$ spaces

We show that an idea, originating initially with a fundamental recursive iteration scheme (usually referred as "the" Kaczmarz algorithm), admits important applications in such infinite-dimensional, and non-commutative, settings as are central to spectral theory of operators in Hilbert space, to optimization, to large sparse systems, to iterated function systems (IFS), and to fractal harmonic analysis. We present a new recursive iteration scheme involving as input a prescribed sequence of selfadjoint projections. Applications include random Kaczmarz recursions, their limits, and their error-estimates.

math.FA↗

arXiv2016

Infinite-Dimensional Measure Spaces and Frame Analysis

We study certain infinite-dimensional probability measures in connection with frame analysis. Earlier work on frame-measures has so far focused on the case of finite-dimensional frames. We point out that there are good reasons for a sharp distinction between stochastic analysis involving frames in finite vs infinite dimensions. For the case of infinite-dimensional Hilbert space $\mathcal{H}$, we study three cases of measures. We first show that, for $\mathcal{H}$ infinite dimensional, 1 one must resort to infinite dimensional measure spaces which properly contain $\mathcal{H}$. The three cases we consider are: (i) Gaussian frame measures, (ii) Markov path-space measures, and (iii) determinantal measures.

math.FA↗

arXiv2016

Reproducing Kernel Hilbert Space vs. Frame Estimates

We consider conditions on a given system $\mathcal{F}$ of vectors in Hilbert space $\mathcal{H}$, forming a frame, which turn $\mathcal{H}$ into a reproducing kernel Hilbert space. It is assumed that the vectors in $\mathcal{F}$ are functions on some set $Ω$. We then identify conditions on these functions which automatically give $\mathcal{H}$ the structure of a reproducing kernel Hilbert space of functions on $Ω$. We further give an explicit formula for the kernel, and for the corresponding isometric isomorphism. Applications are given to Hilbert spaces associated to families of Gaussian processes.

math.FA↗

arXiv2014

Filters and Matrix Factorization

We give a number of explicit matrix-algorithms for analysis/synthesis in multi-phase filtering; i.e., the operation on discrete-time signals which allow a separation into frequency-band components, one for each of the ranges of bands, say $N$, starting with low-pass, and then corresponding filtering in the other band-ranges. If there are $N$ bands, the individual filters will be combined into a single matrix action; so a representation of the combined operation on all $N$ bands by an $N \times N$ matrix, where the corresponding matrix-entries are periodic functions; or their extensions to functions of a complex variable. Hence our setting entails a fixed $N \times N$ matrix over a prescribed algebra of functions of a complex variable. In the case of polynomial filters, the factorizations will always be finite. A novelty here is that we allow for a wide family of non-polynomial filter-banks. Working modulo $N$ in the time domain, our approach also allows for a natural matrix-representation of both down-sampling and up-sampling. The implementation encompasses the combined operation on input, filtering, down-sampling, transmission, up-sampling, an action by dual filters, and synthesis, merges into a single matrix operation. Hence our matrix-factorizations break down the global filtering-process into elementary steps. To accomplish this, we offer a number of adapted matrix factorization-algorithms, such that each factor in our product representation implements in a succession of steps the filtering across pairs of frequency-bands; and so it is of practical significance in implementing signal processing, including filtering of digitized images. Our matrix-factorizations are especially useful in the case of the processing a fixed, but large, number of bands.

math.FA↗

arXiv2012

Orthonormal bases generated by Cuntz algebras

We show how some orthonormal bases can be generated by representations of the Cuntz algebra; these include Fourier bases on fractal measures, generalized Walsh bases on the unit interval and piecewise exponential bases on the middle third Cantor set.

math.FA↗

arXiv2009

Spectral Theory of Discrete Processes

We offer a spectral analysis for a class of transfer operators. These transfer operators arise for a wide range of stochastic processes, ranging from random walks on infinite graphs to the processes that govern signals and recursive wavelet algorithms; even spectral theory for fractal measures. In each case, there is an associated class of harmonic functions which we study. And in addition, we study three questions in depth: In specific applications, and for a specific stochastic process, how do we realize the transfer operator $T$ as an operator in a suitable Hilbert space? And how to spectral analyze $T$ once the right Hilbert space $\mathcal{H}$ has been selected? Finally we characterize the stochastic processes that are governed by a single transfer operator. In our applications, the particular stochastic process will live on an infinite path-space which is realized in turn on a state space $S$. In the case of random walk on graphs $G$, $S$ will be the set of vertices of $G$. The Hilbert space $\mathcal{H}$ on which the transfer operator $T$ acts will then be an $L^{2}$ space on $S$, or a Hilbert space defined from an energy-quadratic form. This circle of problems is both interesting and non-trivial as it turns out that $T$ may often be an unbounded linear operator in $\mathcal{H}$; but even if it is bounded, it is a non-normal operator, so its spectral theory is not amenable to an analysis with the use of von Neumann's spectral theorem. While we offer a number of applications, we believe that our spectral analysis will have intrinsic interest for the theory of operators in Hilbert space.

math-ph↗

arXiv2009

An extension of Wiener integration with the use of operator theory

With the use of tensor product of Hilbert space, and a diagonalization procedure from operator theory, we derive an approximation formula for a general class of stochastic integrals. Further we establish a generalized Fourier expansion for these stochastic integrals. In our extension, we circumvent some of the limitations of the more widely used stochastic integral due to Wiener and Ito, i.e., stochastic integration with respect to Brownian motion. Finally we discuss the connection between the two approaches, as well as a priori estimates and applications.

math-ph↗

arXiv2007

Optimal Decompositions of Translations of $L^{2}$-functions

In this paper we offer a computational approach to the spectral function for a finite family of commuting operators, and give applications. Motivated by questions in wavelets and in signal processing, we study a problem about spectral concentration of integral translations of functions in the Hilbert space $L^{2}(\mathbb{R}^{n})$. Our approach applies more generally to families of $n$ arbitrary commuting unitary operators in a complex Hilbert space $\mathcal{H}$, or equivalent the spectral theory of a unitary representation $U$ of the rank-$n$ lattice $\mathbb{Z}^{n}$ in $\mathbb{R}^{n}$. Starting with a non-zero vector $ψ\in \mathcal{H}$, we look for relations among the vectors in the cyclic subspace in $\mathcal{H}$ generated by $ψ$. Since these vectors $\{U(k)ψ| k \in \mathbb{Z}^{n}\}$ involve infinite ``linear combinations," the problem arises of giving geometric characterizations of these non-trivial linear relations. A special case of the problem arose initially in work of Kolmogorov under the name $L^{2}$-independence. This refers to \textit{infinite} linear combinations of integral translates of a fixed function with $l^{2}$-coefficients. While we were motivated by the study of translation operators arising in wavelet and frame theory, we stress that our present results are general; our theorems are about spectral densities for general unitary operators, and for stochastic integrals.

math.FA↗

arXiv2007

Comparison of Discrete and Continuous Wavelet Transforms

In this paper we outline several points of view on the interplay between discrete and continuous wavelet transforms; stressing both pure and applied aspects of both. We outline some new links between the two transform technologies based on the theory of representations of generators and relations. By this we mean a finite system of generators which are represented by operators in Hilbert space. We further outline how these representations yield sub-band filter banks for signal and image processing algorithms.

cs.CE↗