arXiv subjects

Amirhossein Taghvaei

Publications and source records attributed to Amirhossein Taghvaei.

At least 37 records · Page 2Linked to original sources

arXiv2024

Time-Reversal of Stochastic Maximum Principle

Stochastic maximum principle (SMP) specifies a necessary condition for the solution of a stochastic optimal control problem. The condition involves a coupled system of forward and backward stochastic differential equations (FBSDE) for the state and the adjoint processes. Numerical solution of the FBSDE is challenging because the boundary condition of the adjoint process is specified at the terminal time, while the solution should be adaptable to the forward in time filtration of a Wiener process. In this paper, a "time-reversal" of the FBSDE system is proposed that involves integration with respect to a backward in time Wiener process. The time-reversal is used to propose an iterative Monte-Carlo procedure to solves the FBSDE system and its time-reversal simultaneously. The procedure involves approximating the {Föllmer's drift} and solving a regression problem between the state and its adjoint at each time. The procedure is illustrated for the linear quadratic (LQ) optimal control problem with a numerical example.

eess.SY↗

arXiv2024

Conditional Optimal Transport on Function Spaces

We present a systematic study of conditional triangular transport maps in function spaces from the perspective of optimal transportation and with a view towards amortized Bayesian inference. More specifically, we develop a theory of constrained optimal transport problems that describe block-triangular Monge maps that characterize conditional measures along with their Kantorovich relaxations. This generalizes the theory of optimal triangular transport to separable infinite-dimensional function spaces with general cost functions. We further tailor our results to the case of Bayesian inference problems and obtain regularity estimates on the conditioning maps from the prior to the posterior. Finally, we present numerical experiments that demonstrate the computational applicability of our theoretical results for amortized and likelihood-free inference of functional parameters.

math.OC↗

arXiv2023

Computational Optimal Transport and Filtering on Riemannian manifolds

In this paper we extend recent developments in computational optimal transport to the setting of Riemannian manifolds. In particular, we show how to learn optimal transport maps from samples that relate probability distributions defined on manifolds. Specializing these maps for sampling conditional probability distributions provides an ensemble approach for solving nonlinear filtering problems defined on such geometries. The proposed computational methodology is illustrated with examples of transport and nonlinear filtering on Lie groups, including the circle $S^1$, the special Euclidean group $SE(2)$, and the special orthogonal group $SO(3)$.

math.OC↗

arXiv2023

Distributed Nonlinear Filtering using Triangular Transport Maps

The distributed filtering problem sequentially estimates a global state variable using observations from a network of local sensors with different measurement models. In this work, we introduce a novel methodology for distributed nonlinear filtering by combining techniques from transportation of measures, dimensionality reduction, and consensus algorithms. We illustrate our methodology on a satellite pose estimation problem from a network of direct and indirect observers. The numerical results serve as a proof of concept, offering new venues for theoretical and applied research in the domain of distributed filtering.

cs.MA↗

arXiv2023

Data-driven Optimal Filtering for Linear Systems with Unknown Noise Covariances

This paper examines learning the optimal filtering policy, known as the Kalman gain, for a linear system with unknown noise covariance matrices using noisy output data. The learning problem is formulated as a stochastic policy optimization problem, aiming to minimize the output prediction error. This formulation provides a direct bridge between data-driven optimal control and, its dual, optimal filtering. Our contributions are twofold. Firstly, we conduct a thorough convergence analysis of the stochastic gradient descent algorithm, adopted for the filtering problem, accounting for biased gradients and stability constraints. Secondly, we carefully leverage a combination of tools from linear system theory and high-dimensional statistics to derive bias-variance error bounds that scale logarithmically with problem dimension, and, in contrast to subspace methods, the length of output trajectories only affects the bias term.

eess.SY↗

arXiv2023

A matching principle for power transfer in Stochastic Thermodynamics

Gradients in temperature and particle concentration fuel many processes in the physical and biological world. In the present work we study a thermodynamic engine powered by anisotropic thermal excitation (that may be due to e.g., a temperature gradient), and draw parallels with the well-known principle of impedance matching in circuit theory, where for maximal power transfer, the load voltage needs to be half of that of the supplying power source. We maximize power output of the thermodynamic engine at steady-state and show that the optimal reactive force is precise half of that supplied by the anisotropy.

eess.SY↗

arXiv2023

Optimal Transport Particle Filters

This paper is concerned with the theoretical and computational development of a new class of nonlinear filtering algorithms called the optimal transport particle filters (OTPF). The algorithm is based on a recently introduced variational formulation of the Bayes' rule, which aims to find the Brenier optimal transport map between the prior and the posterior distributions as the solution to a stochastic optimization problem. On the theoretical side, the existing methods for the error analysis of particle filters and stability results for optimal transport map estimation are combined to obtain uniform error bounds for the filter's performance in terms of the optimization gap in solving the variational problem. The error analysis reveals a bias-variance trade-off that can ultimately be used to understand if/when the curse of dimensionality can be avoided in these filters. On the computational side, the proposed algorithm is evaluated on a nonlinear filtering example in comparison with the ensemble Kalman filter (EnKF) and the sequential importance resampling (SIR) particle filter.

math.OC↗

arXiv2023

A Survey of Feedback Particle Filter and related Controlled Interacting Particle Systems (CIPS)

In this survey, we describe controlled interacting particle systems (CIPS) to approximate the solution of the optimal filtering and the optimal control problems. Part I of the survey is focussed on the feedback particle filter (FPF) algorithm, its derivation based on optimal transportation theory, and its relationship to the ensemble Kalman filter (EnKF) and the conventional sequential importance sampling-resampling (SIR) particle filters. The central numerical problem of FPF -- to approximate the solution of the Poisson equation -- is described together with the main solution approaches. An analytical and numerical comparison with the SIR particle filter is given to illustrate the advantages of the CIPS approach. Part II of the survey is focussed on adapting these algorithms for the problem of reinforcement learning. The survey includes several remarks that describe extensions as well as open problems in this subject.

eess.SY↗

arXiv2023

Duality-Based Stochastic Policy Optimization for Estimation with Unknown Noise Covariances

Duality of control and estimation allows mapping recent advances in data-guided control to the estimation setup. This paper formalizes and utilizes such a mapping to consider learning the optimal (steady-state) Kalman gain when process and measurement noise statistics are unknown. Specifically, building on the duality between synthesizing optimal control and estimation gains, the filter design problem is formalized as direct policy learning. In this direction, the duality is used to extend existing theoretical guarantees of direct policy updates for Linear Quadratic Regulator (LQR) to establish global convergence of the Gradient Descent (GD) algorithm for the estimation problem--while addressing subtle differences between the two synthesis problems. Subsequently, a Stochastic Gradient Descent (SGD) approach is adopted to learn the optimal Kalman gain without the knowledge of noise covariances. The results are illustrated via several numerical examples.

eess.SY↗

arXiv2023

Minimal entropy production in the presence of anisotropic fluctuations

Anisotropy in temperature, chemical potential, or ion concentration, provides the fuel that feeds dynamical processes that sustain life. At the same time, anisotropy is a root cause of incurred losses manifested as entropy production. In this work we consider a rudimentary model of an overdamped stochastic thermodynamic system in an anisotropic temperature heat bath, and study minimum entropy production when driving the system between thermodynamic states in finite time. While entropy production in isotropic temperature environments can be expressed in terms of the length (in the Wasserstein-2 metric) traversed by the thermodynamic state of the system, anisotropy complicates substantially the mechanism of entropy production since, besides dissipation, seepage of energy between ambient anisotropic heat sources by way of the system dynamics is often a major contributing factor. A key result of the paper is to show that in the presence of anisotropy, minimization of entropy production can once again be expressed via a modified Optimal Mass Transport (OMT) problem. However, in contrast to the isotropic situation that leads to a classical OMT problem and a Wasserstein length, entropy production may not be identically zero when the thermodynamic state remains unchanged (unless one has control over non-conservative forces); this is due to the fact that maintaining a Non-Equilibrium Steady-State (NESS) incurs an intrinsic entropic cost that can be traced back to a seepage of heat between heat baths. As alluded to, NESSs represent hallmarks of life, since living matter by necessity operates far from equilibrium. Therefore, the question studied herein, to characterize minimal entropy production in anisotropic environments, appears of central importance in biological processes and on how such processes may have evolved to optimize for available usage of resources.

cond-mat.stat-mech↗

arXiv2022

Geometry of finite-time thermodynamic cycles with anisotropic thermal fluctuations

In contrast to the classical concept of a Carnot engine that alternates contact between heat baths of different temperatures, naturally occurring processes usually harvest energy from anisotropy, being exposed simultaneously to chemical and thermal fluctuations of different intensities. In these cases, the enabling mechanism responsible for transduction of energy is typically the presence of a non-equilibrium steady state (NESS). A suitable stochastic model for such a phenomenon is the Brownian gyrator -- a two-degree of freedom stochastically driven system that exchanges energy and heat with the environment. In the context of such a model we present, from a stochastic control perspective, a geometric view of the energy harvesting mechanism that entails a forced periodic trajectory of the system state on the thermodynamic manifold. Dissipation and work output are expressed accordingly as path integrals of a controlled process, and fundamental limitations on power and efficiency are expressed in geometric terms via a relationship to an isoperimetric problem. The theory is presented for high-order systems far from equilibrium and beyond the linear response regime.

eess.SY↗

arXiv2022

Minimal entropy production in anisotropic temperature fields

Anisotropy of temperature fields, chemical potentials and ion concentration gradients provide the fuel that feeds dynamical processes that sustain life. Dynamical flows in respective environments incur losses manifested as entropy production. In this work we consider a rudimentary model of an overdamped stochastic thermodynamic system in an anisotropic temperature heat bath, and analyze the problem to minimize entropy production while driving the system between thermodynamic states in finite time. It is noted that entropy production in a fully isotropic temperature field, can be expressed as the Wasserstein-2 length of the path traversed by the thermodynamic state of the system. In the presence of an anisotropic temperature field, the mechanism of entropy production is substantially more complicated as, besides dissipation, it entails seepage of energy between the ambient heat sources by way of the system dynamics. We show that, in this case, the entropy production can be expressed as the solution of a suitably constrained and generalized Optimal Mass Transport (OMT) problem. In contrast to the situation in standard OMT, entropy production may not be identically zero, even when the thermodynamic state remains unchanged. Physically, this is due to the fact that maintaining a Non-Equilibrium Steady State (NESS), incurs an intrinsic entropic cost. As already noted, NESSs are the hallmark of life and living systems by necessity operate away from equilibrium. Thus our problem of minimizing entropy production appears of central importance in understanding biological processes, such as molecular motors and motor proteins, and on how such processes may have evolved to optimize for available usage of resources.

cond-mat.stat-mech↗

arXiv2022

An Optimal Transport Formulation of Bayes' Law for Nonlinear Filtering Algorithms

This paper presents a variational representation of the Bayes' law using optimal transportation theory. The variational representation is in terms of the optimal transportation between the joint distribution of the (state, observation) and their independent coupling. By imposing certain structure on the transport map, the solution to the variational problem is used to construct a Brenier-type map that transports the prior distribution to the posterior distribution for any value of the observation signal. The new formulation is used to derive the optimal transport form of the Ensemble Kalman filter (EnKF) for the discrete-time filtering problem and propose a novel extension of EnKF to the non-Gaussian setting utilizing input convex neural networks. Finally, the proposed methodology is used to derive the optimal transport form of the feedback particle filler (FPF) in the continuous-time limit, which constitutes its first variational construction without explicitly using the nonlinear filtering equation or Bayes' law.

math.OC↗

arXiv2022

Inertialess Gyrating Engines

A typical model for a gyrating engine consists of an inertial wheel powered by an energy source that generates an angle-dependent torque. Examples of such engines include a pendulum with an externally applied torque, Stirling engines, and the Brownian gyrating engine. Variations in the torque are averaged out by the inertia of the system to produce limit cycle oscillations. While torque generating mechanisms are also ubiquitous in the biological world, where they typically feed on chemical gradients, inertia is not a property that one naturally associates with such processes. In the present work, seeking ways to dispense of the need for inertial effects, we study an inertia-less concept where the combined effect of coupled torque-producing components averages out variations in the ambient potential and helps overcome dissipative forces to allow sustained operation for vanishingly small inertia. We exemplify this inertia-less concept through analysis of two of the aforementioned engines, the Stirling engine and the Brownian gyrating engine. An analogous principle may be sought in biomolecular processes as well as in modern-day technological engines, where for the latter, the coupled torque-producing components reduce vibrations that stem from the variability of the generated torque.

cond-mat.stat-mech↗

arXiv2022

Variational Wasserstein gradient flow

Wasserstein gradient flow has emerged as a promising approach to solve optimization problems over the space of probability distributions. A recent trend is to use the well-known JKO scheme in combination with input convex neural networks to numerically implement the proximal step. The most challenging step, in this setup, is to evaluate functions involving density explicitly, such as entropy, in terms of samples. This paper builds on the recent works with a slight but crucial difference: we propose to utilize a variational formulation of the objective function formulated as maximization over a parametric class of functions. Theoretically, the proposed variational formulation allows the construction of gradient flows directly for empirical distributions with a well-defined and meaningful objective function. Computationally, this approach replaces the computationally expensive step in existing methods, to handle objective functions involving density, with inner loop updates that only require a small batch of samples and scale well with the dimension. The performance and scalability of the proposed method are illustrated with the aid of several numerical experiments involving high-dimensional synthetic and real datasets.

cs.LG↗

arXiv2022

Stochastic thermodynamic engines under time-varying temperature profile

In the present paper, we study the power output and efficiency of overdamped stochastic thermodynamic engines that are in contact with a heat bath having a temperature that varies periodically with time. This is in contrast to most of the existing literature that considers the Carnot paradigm of alternating contact with heat baths having different fixed temperatures, hot and cold. Specifically, we consider a periodic and bounded but otherwise arbitrary temperature profile and derive explicit bounds on the power and efficiency achievable by a suitable controlling potential that couples the thermodynamic engine to the external world. Standing assumptions in our analysis are bounds on the norm of the gradient of effective potentials -- in the absence of any such constraint, the physically questionable conclusion of arbitrarily large power can be drawn.

cond-mat.stat-mech↗

arXiv2022

Controlled Interacting Particle Algorithms for Simulation-based Reinforcement Learning

This paper is concerned with optimal control problems for control systems in continuous time, and interacting particle system methods designed to construct approximate control solutions. Particular attention is given to the linear quadratic (LQ) control problem. There is a growing interest in re-visiting this classical problem, in part due to the successes of reinforcement learning (RL). The main question of this body of research (and also of our paper) is to approximate the optimal control law {\em without} explicitly solving the Riccati equation. A novel simulation-based algorithm, namely a dual ensemble Kalman filter (EnKF), is introduced. The algorithm is used to obtain formulae for optimal control, expressed entirely in terms of the EnKF particles. An extension to the nonlinear case is also presented. The theoretical results and algorithms are illustrated with numerical experiments.

eess.SY↗

arXiv2022

Underdamped stochastic thermodynamic engines in contact with a heat bath with arbitrary temperature profile

We study thermodynamic processes in contact with a heat bath that may have an arbitrary time-varying periodic temperature profile. Within the framework of stochastic thermodynamics, and for models of thermo-dynamic engines in the idealized case of underdamped particles in the low-friction regime, we derive explicit bounds as well as optimal control protocols that draw maximum power and achieve maximum efficiency at any specified level of power.

math.OC↗