arXiv subjects

Sai Tej Paruchuri

Publications and source records attributed to Sai Tej Paruchuri.

11 recordsLinked to original sources

arXiv2025

Tailoring Reproducing Kernels for Optimal Control via Policy Iteration

This paper presents a novel approach to formulating the actor-critic method for optimal control by casting policy iteration in reproducing kernel Hilbert spaces (RKHSs -- also known as native spaces). By tailoring the reproducing kernel and RKHS to the dynamics of the nonlinear optimal control problem, we leverage recent advancements in characterizing error bounds from statistical and machine learning theory. These approximations define a general strategy to select the bases of the actor-critic networks, and we formally guarantee for the first time that this basis selection procedure leads to closed-form error bounds for the individual steps of policy iteration. These bounds often have a geometric and computable form, making them potentially useful for a priori or a posteriori evaluation of candidate collections of scattered bases. Numerical studies subsequently provide qualitative evidence of the practical performance achieved for the full recursion using the algorithms and theory developed for the single-step error bounds.

math.OC↗

arXiv2024

Convergence Rates of Online Critic Value Function Approximation in Native Spaces

In this paper, the evolution equation that defines the online critic for the approximation of the optimal value function is cast in a general class of reproducing kernel Hilbert spaces (RKHSs). Exploiting some core tools of RKHS theory, this formulation allows deriving explicit bounds on the performance of the critic in terms of the kernel and definition of the RKHS, the number of basis functions, and the location of centers used to define scattered bases. The performance of the critic is precisely measured in terms of the power function of the scattered basis used in approximations, and it can be used either in an a priori evaluation of potential bases or in an a posteriori assessments of value function error for basis enrichment or pruning. The most concise bounds in the paper describe explicitly how the critic performance depends on the placement of centers, as measured by their fill distance in a subset that contains the trajectory of the critic.

math.OC↗

arXiv2023

Rates of Convergence in Certain Native Spaces of Approximations used in Reinforcement Learning

This paper studies convergence rates for some value function approximations that arise in a collection of reproducing kernel Hilbert spaces (RKHS) $H(Ω)$. By casting an optimal control problem in a specific class of native spaces, strong rates of convergence are derived for the operator equation that enables offline approximations that appear in policy iteration. Explicit upper bounds on error in value function and controller approximations are derived in terms of power function $\mathcal{P}_{H,N}$ for the space of finite dimensional approximants $H_N$ in the native space $H(Ω)$. These bounds are geometric in nature and refine some well-known, now classical results concerning convergence of approximations of value functions.

eess.SY↗

arXiv2021

Strictly Decentralized Adaptive Estimation of External Fields using Reproducing Kernels

This paper describes an adaptive method in continuous time for the estimation of external fields by a team of $N$ agents. The agents $i$ each explore subdomains $Ω^i$ of a bounded subset of interest $Ω\subset X := \mathbb{R}^d$. Ideal adaptive estimates $\hat{g}^i_t$ are derived for each agent from a distributed parameter system (DPS) that takes values in the scalar-valued reproducing kernel Hilbert space $H_X$ of functions over $X$. Approximations of the evolution of the ideal local estimate $\hat{g}^i_t$ of agent $i$ is constructed solely using observations made by agent $i$ on a fine time scale. Since the local estimates on the fine time scale are constructed independently for each agent, we say that the method is strictly decentralized. On a coarse time scale, the individual local estimates $\hat{g}^i_t$ are fused via the expression $\hat{g}_t:=\sum_{i=1}^NΨ^i \hat{g}^i_t$ that uses a partition of unity $\{Ψ^i\}_{1\leq i\leq N}$ subordinate to the cover $\{Ω^i\}_{i=1,\ldots,N}$ of $Ω$. Realizable algorithms are obtained by constructing finite dimensional approximations of the DPS in terms of scattered bases defined by each agent from samples along their trajectories. Rates of convergence of the error in the finite dimensional approximations are derived in terms of the fill distance of the samples that define the scattered centers in each subdomain. The qualitative performance of the convergence rates for the decentralized estimation method is illustrated via numerical simulations.

eess.SY↗

arXiv2021

Partial Persistence of Excitation in RKHS Embedded Adaptive Estimation

In this paper, an adaptive non-parametric method is proposed to estimate the scalar-valued nonlinear function that appears in uncertain systems governed by ordinary differential equations (ODEs). By employing an infinite-dimensional reproducing kernel Hilbert space (RKHS) as the hypothesis space, the nonlinear estimation problem in finite-dimensional Euclidean space is recast into that of constructing a linear observer in the infinite-dimensional RKHS. The analysis of convergence is facilitated by the introduction of a novel condition of partial persistent excitation (partial PE), which is defined for a subspace of the RKHS. Using this condition, we prove that the projection of the function estimation error onto the PE subspace converges in norm asymptotically to zero. While this is an abstract notion of convergence that depends implicitly on the kernel used to define the RKHS, we derive conditions that ensure the pointwise convergence of the function estimates over the PE subset. This paper additionally introduces a weaker but geometrically intuitive notion of a partial PE condition, one that resembles PE conditions as they have been formulated historically in Euclidean spaces. Sufficient conditions are derived that describe when the two conditions are equivalent. Finally, qualitative properties of the convergence proofs derived in the paper are illustrated with numerical simulations.

math.OC↗

arXiv2020

Sufficient Conditions for Parameter Convergence over Embedded Manifolds using Kernel Techniques

The persistence of excitation (PE) condition is sufficient to ensure parameter convergence in adaptive estimation problems. Recent results on adaptive estimation in reproducing kernel Hilbert spaces (RKHS) introduce PE conditions for RKHS. This paper presents sufficient conditions for PE for the particular class of uniformly embedded reproducing kernel Hilbert spaces (RKHS) defined over smooth Riemannian manifolds. This paper also studies the implications of the sufficient condition in the case when the RKHS is finite or infinite-dimensional. When the RKHS is finite-dimensional, the sufficient condition implies parameter convergence as in the conventional analysis. On the other hand, when the RKHS is infinite-dimensional, the same condition implies that the function estimate error is ultimately bounded by a constant that depends on the approximation error in the infinite-dimensional RKHS. We illustrate the effectiveness of the sufficient condition in a practical example.

eess.SY↗

arXiv2020

Kernel Center Adaptation in the Reproducing Kernel Hilbert Space Embedding Method

The performance of adaptive estimators that employ embedding in reproducing kernel Hilbert spaces (RKHS) depends on the choice of the location of basis kernel centers. Parameter convergence and error approximation rates depend on where and how the kernel centers are distributed in the state-space. In this paper, we develop the theory that relates parameter convergence and approximation rates to the position of kernel centers. We develop criteria for choosing kernel centers in a specific class of systems - ones in which the state trajectory regularly visits the neighborhood of the positive limit set. Two algorithms, based on centroidal Voronoi tessellations and Kohonen self-organizing maps, are derived to choose kernel centers in the RKHS embedding method. Finally, we implement these methods on two practical examples and test their effectiveness.

eess.SY↗

arXiv2020

Approximations of the Reproducing Kernel Hilbert Space (RKHS) Embedding Method over Manifolds

The reproducing kernel Hilbert space (RKHS) embedding method is a recently introduced estimation approach that seeks to identify the unknown or uncertain function in the governing equations of a nonlinear set of ordinary differential equations (ODEs). While the original state estimate evolves in Euclidean space, the function estimate is constructed in an infinite-dimensional RKHS that must be approximated in practice. When a finite-dimensional approximation is constructed using a basis defined in terms of shifted kernel functions centered at the observations along a trajectory, the RKHS embedding method can be understood as a data-driven approach. This paper derives sufficient conditions that ensure that approximations of the unknown function converge in a Sobolev norm over a submanifold that supports the dynamics. Moreover, the rate of convergence for the finite-dimensional approximations is derived in terms of the fill distance of the samples in the embedded manifold. Numerical simulation of an example problem is carried out to illustrate the qualitative nature of convergence results derived in the paper.

math.OC↗

arXiv2020

Intrinsic and Extrinsic Approximation of Koopman Operators over Manifolds

This paper derives rates of convergence of certain approximations of the Koopman operators that are associated with discrete, deterministic, continuous semiflows on a complete metric space $(X,d_X)$. Approximations are constructed in terms of reproducing kernel bases that are centered at samples taken along the system trajectory. It is proven that when the samples are dense in a certain type of smooth manifold $M\subseteq X$, the derived rates of convergence depend on the fill distance of samples along the trajectory in that manifold. Error bounds for projection-based and data-dependent approximations of the Koopman operator are derived in the paper. A discussion of how these bounds are realized in intrinsic and extrinsic approximation methods is given. Finally, a numerical example that illustrates qualitatively the convergence guarantees derived in the paper is given.

math.DS↗

arXiv2020

RKHS Embedding for Estimating Nonlinear Piezoelectric Systems

Nonlinearities in piezoelectric systems can arise from internal factors such as nonlinear constitutive laws or external factors like realizations of boundary conditions. It can be difficult or even impossible to derive detailed models from the first principles of all the sources of nonlinearity in a system. As a specific example, in traditional modeling techniques that use electric enthalpy density with higher-order terms, it can be problematic to choose which polynomial nonlinearities are essential. This paper introduces adaptive estimator techniques to estimate the nonlinearities that can arise in certain piezoelectric systems. Here an underlying assumption is that the nonlinearities can be modeled as functions in a reproducing kernel Hilbert space (RKHS). Unlike traditional modeling approaches, the approach discussed in this paper allows the development of models without knowledge of the precise form or structure of the nonlinearity. This approach can be viewed as a data-driven method to approximate the unknown nonlinear system. This paper introduces the theory behind the adaptive estimator and studies the effectiveness of this approach numerically for a class of nonlinear piezoelectric composite beams.

math.OC↗

arXiv2019

Persistence of Excitation in Reproducing Kernel Hilbert Spaces, Positive Limit Sets, and Smooth Manifolds

This paper studies the relationship between the positive limit sets of continuous semiflows and the newly introduced definition of persistently excited (PE) sets and associated subspaces of reproducing kernel Hilbert (RKH) spaces. It is shown that if the RKH space contains a rich collection of cut-off functions, persistently excited sets are contained as subsets of the positive limit set of the semiflow. The paper demonstrates how the new PE condition can be used to guarantee convergence of function estimates in the RKH space embedding method for adaptive estimation. In particular, the paper is applied to uncertain ODE systems with positive limit sets given by certain types of smooth manifolds, and it establishes convergence of adaptive function estimates over the manifolds.

math.OC↗