Searcharxiv⌕ Search

arXiv subjects

Michele Pavon

Publications and source records attributed to Michele Pavon.

At least 37 records · Page 2Linked to original sources
arXiv2017

Steering the distribution of agents in mean-field and cooperative games

The purpose of this work is to pose and solve the problem to guide a collection of weakly interacting dynamical systems (agents, particles, etc.) to a specified terminal distribution. The framework is that of mean-field and of cooperative games. A terminal cost is used to accomplish the task; we establish that the map between terminal costs and terminal probability distributions is onto. Our approach relies on and extends the theory of optimal mass transport and its generalizations.

eess.SY↗
arXiv2017

Extremal flows on Wasserstein space

We develop an intrinsic geometric approach to calculus of variations on Wasserstein space. We show that the flows associated to the Schroedinger bridge with general prior, to Optimal Mass Transport and to the Madelung fluid can all be characterized as annihilating the first variation of a suitable action. We then discuss the implications of this unified framework for stochastic mechanics: It entails, in particular, a sort of fluid-dynamic reconciliation between Bohm's and Nelson's stochastic mechanics.

math-ph↗
arXiv2017

Efficient-robust routing for single commodity network flows

We study single commodity network flows with suitable robustness and efficiency specs. An original use of a maximum entropy problem for distributions on the paths of the graph turns this problem into a steering problem for Markov chains with prescribed initial and final marginals. From a computational standpoint, viewing scheduling this way is especially attractive in light of the existence of an iterative algorithm to compute the solution. The present paper builds on [13] by introducing an index of efficiency of a transportation plan and points, accordingly, to efficient-robust transport policies. In developing the theory, we establish two new invariance properties of the solution (called bridge) - an iterated bridge invariance property and the invariance of the most probable paths. These properties, which were tangentially mentioned in our previous work, are fully developed here. We also show that the distribution on paths of the optimal transport policy, which depends on a "temperature" parameter, tends to the solution of the "most economical" but possibly less robust optimal mass transport problem as the temperature goes to zero. The relevance of all of these properties for transport over networks is illustrated in an example.

math-ph↗
arXiv2016

Optimal steering of a linear stochastic system to a final probability distribution, Part III

The subject of this work has its roots in the so called Schroedginer Bridge Problem (SBP) which asks for the most likely distribution of Brownian particles in their passage between observed empirical marginal distributions at two distinct points in time. Renewed interest in this problem was sparked by a reformulation in the language of stochastic control. In earlier works, presented as Part I and Part II, we explored a generalization of the original SBP that amounts to optimal steering of linear stochastic dynamical systems between state-distributions, at two points in time, under full state feedback. In these works the cost was quadratic in the control input. The purpose of the present work is to detail the technical steps in extending the framework to the case where a quadratic cost in the state is also present. In the zero-noise limit, we obtain the solution of a (deterministic) mass transport problem with general quadratic cost.

eess.SY↗
arXiv2016

Robust transport over networks

We consider transport over a strongly connected, directed graph. The scheduling amounts to selecting transition probabilities for a discrete-time Markov evolution which is designed to be consistent with certain initial and final marginals. The random evolution is selected to be closest to a prior measure on paths in the relative entropy sense, i.e., a Schroedinger bridge between the two marginals. This is an atypical stochastic control problem where the control consists in suitably modifying the transition mechanism. The prior can incorporate cost of traversing edges or allocate equal probability to all paths of equal length connecting any two given nodes, i.e., a uniform measure on paths. This latter choice relies on the so-called Ruelle-Bowen random walk and gives rise to a scheduling that tends to utilize all paths as uniformly as the topology allows. Thus, when the Ruelle-Bowen law is taken as prior, the transportation plan tends to lessen congestion and ensure a level of robustness. We show that the Ruelle-Bowen law is itself a Schroedinger bridge albeit with a prior that is not a probability measure. The paradigm of Schroedinger bridges as a mechanism for scheduling transport on networks can be adapted to graphs that are not strongly connected as well as to weighted graphs. The latter leads to transportation plans that effect a compromise between robustness and transportation cost.

eess.SY↗
arXiv2016

Stochastic control, entropic interpolation and gradient flows on Wasserstein product spaces

Since the early nineties, it has been observed that the Schroedinger bridge problem can be formulated as a stochastic control problem with atypical boundary constraints. This in turn has a fluid dynamic counterpart where the flow of probability densities represents an entropic interpolation between the given initial and final marginals. In the zero noise limit, such entropic interpolation converges in a suitable sense to the displacement interpolation of optimal mass transport (OMT). We consider two absolutely continuous curves in Wasserstein space ${\cal W}_2$ and study the evolution of the relative entropy on ${\cal W}_2\times {\cal W}_2$ on a finite time interval. Thus, this study differs from previous work in OMT theory concerning relative entropy from a fixed (often equilibrium) distribution (density). We derive a gradient flow on Wasserstein product space. We find the remarkable property that fluxes in the two components are opposite. Plugging in the "steepest descent" into the evolution of the relative entropy we get what appears to be a new formula: The two flows approach each other at a faster rate than that of two solutions of the same Fokker-Planck. We then study the evolution of relative entropy in the case of uncontrolled-controlled diffusions. In two special cases of the Schroedinger bridge problem, we show that such relative entropy may be monotonically decreasing or monotonically increasing.

math.PR↗
arXiv2015

Fast cooling for a system of stochastic oscillators

We study feedback control of coupled nonlinear stochastic oscillators in a force field. We first consider the problem of asymptotically driving the system to a desired {\em steady state} corresponding to reduced thermal noise. Among the feedback controls achieving the desired asymptotic transfer, we find that the most efficient one {from an energy point of view} is characterized by {\em time-reversibility}. We also extend the theory of Schrödinger bridges to this model, thereby steering the system in {\em finite time} and with minimum effort to a target steady-state distribution. The system can then be maintained in this state through the optimal steady-state feedback control. The solution, in the finite-horizon case, involves a space-time harmonic function $φ$, and $-\logφ$ plays the role of an artificial, time-varying potential in which the desired evolution occurs. This framework appears extremely general and flexible and can be viewed as a considerable generalization of existing active control strategies such as macromolecular cooling. In the case of a quadratic potential, the results assume a form particularly attractive from the algorithmic viewpoint as the optimal control can be computed via deterministic matricial differential equations. An example involving inertial particles illustrates both transient and steady state optimal feedback control.

math-ph↗
arXiv2015

Entropic and displacement interpolation: a computational approach using the Hilbert metric

Monge-Kantorovich optimal mass transport (OMT) provides a blueprint for geometries in the space of positive densities -- it quantifies the cost of transporting a mass distribution into another. In particular, it provides natural options for interpolation of distributions (displacement interpolation) and for modeling flows. As such it has been the cornerstone of recent developments in physics, probability theory, image processing, time-series analysis, and several other fields. In spite of extensive work and theoretical developments, the computation of OMT for large scale problems has remained a challenging task. An alternative framework for interpolating distributions, rooted in statistical mechanics and large deviations, is that of Schroedinger bridges (entropic interpolation). This may be seen as a stochastic regularization of OMT and can be cast as the stochastic control problem of steering the probability density of the state-vector of a dynamical system between two marginals. In this approach, however, the actual computation of flows had hardly received any attention. In recent work on Schroedinger bridges for Markov chains and quantum evolutions, we noted that the solution can be efficiently obtained from the fixed-point of a map which is contractive in the Hilbert metric. Thus, the purpose of this paper is to show that a similar approach can be taken in the context of diffusion processes which i) leads to a new proof of a classical result on Schroedinger bridges and ii) provides an efficient computational scheme for both, Schroedinger bridges and OMT. We illustrate this new computational approach by obtaining interpolation of densities in representative examples such as interpolation of images.

math.OC↗
arXiv2015

Steering state statistics with output feedback

Consider a linear stochastic system whose initial state is a random vector with a specified Gaussian distribution. Such a distribution may represent a collection of particles abiding by the specified system dynamics. In recent publications, we have shown that, provided the system is controllable, it is always possible to steer the state covariance to any specified terminal Gaussian distribution using state feedback. The purpose of the present work is to show that, in the case where only partial state observation is available, a necessary and sufficient condition for being able to steer the system to a specified terminal Gaussian distribution for the state vector is that the terminal state covariance be greater (in the positive-definite sense) than the error covariance of a corresponding Kalman filter.

math.OC↗
arXiv2015

Optimal control of the state statistics for a linear stochastic system

We consider a variant of the classical linear quadratic Gaussian regulator (LQG) in which penalties on the endpoint state are replaced by the specification of the terminal state distribution. The resulting theory considerably differs from LQG as well as from formulations that bound the probability of violating state constraints. We develop results for optimal state-feedback control in the two cases where i) steering of the state distribution is to take place over a finite window of time with minimum energy, and ii) the goal is to maintain the state at a stationary distribution over an infinite horizon with minimum power. For both problems the distribution of noise and state are Gaussian. In the first case, we show that provided the system is controllable, the state can be steered to any terminal Gaussian distribution over any specified finite time-interval. In the second case, we characterize explicitly the covariance of admissible stationary state distributions that can be maintained with constant state-feedback control. The conditions for optimality are expressed in terms of a system of dynamically coupled Riccati equations in the finite horizon case and in terms of algebraic conditions for the stationary case. In the case where the noise and control share identical input channels, the Riccati equations for finite-horizon steering become homogeneous and can be solved in closed form. The present paper is largely based on our recent work in arxiv.org/abs/1408.2222, arxiv.org/abs/1410.3447 and presents an overview of certain key results.

math.OC↗
arXiv2015

Optimal mass transport over bridges

We present an overview of our recent work on implementable solutions to the Schroedinger bridge problem and their potential application to optimal transport and various generalizations.

math.OC↗
arXiv2015

Optimal transport over a linear dynamical system

We consider the problem of steering an initial probability density for the state vector of a linear system to a final one, in finite time, using minimum energy control. In the case where the dynamics correspond to an integrator ($\dot x(t) = u(t)$) this amounts to a Monge-Kantorovich Optimal Mass Transport (OMT) problem. In general, we show that the problem can again be reduced to solving an OMT problem and that it has a unique solution. In parallel, we study the optimal steering of the state-density of a linear stochastic system with white noise disturbance; this is known to correspond to a Schrödinger bridge. As the white noise intensity tends to zero, the flow of densities converges to that of the deterministic dynamics and can serve as a way to compute the solution of its deterministic counterpart. The solution can be expressed in closed-form for Gaussian initial and final state densities in both cases.

math.OC↗
arXiv2014

On the relation between optimal transport and Schrödinger bridges: A stochastic control viewpoint

We take a new look at the relation between the optimal transport problem and the Schrödinger bridge problem from the stochastic control perspective. We show that the connections are richer and deeper than described in existing literature. In particular: a) We give an elementary derivation of the Benamou-Brenier fluid dynamics version of the optimal transport problem; b) We provide a new fluid dynamics version of the Schrödinger bridge problem; c) We observe that the latter provides an important connection with optimal transport without zero noise limits; d) We propose and solve a fluid dynamic version of optimal transport with prior; e) We can then view optimal transport with prior as the zero noise limit of Schrödinger bridges when the prior is any Markovian evolution. In particular, we work out the Gaussian case. A numerical example of the latter convergence involving Brownian particles is also provided.

eess.SY↗
arXiv2014

Optimal steering of a linear stochastic system to a final probability distribution, part II

We consider the problem of minimum energy steering of a linear stochastic system to a final prescribed distribution over a finite horizon and to maintain a stationary distribution over an infinite horizon. We present sufficient conditions for optimality in terms of a system of dynamically coupled Riccati equations in the finite horizon case and algebraic in the stationary case. We then address the question of feasibility for both problems. For the finite-horizon case, provided the system is controllable, we prove that without any restriction on the directionality of the stochastic disturbance it is always possible to steer the state to any arbitrary Gaussian distribution over any specified finite time-interval. For the stationary infinite horizon case, it is not always possible to maintain the state at an arbitrary Gaussian distribution through constant state-feedback. It is shown that covariances of admissible stationary Gaussian distributions are characterized by a certain Lyapunov-like equation. We finally present an alternative to solving the system of coupled Riccati equations, by expressing the optimal controls in the form of solutions to (convex) semi-definite programs for both cases. We conclude with an example to steer the state covariance of the distribution of inertial particles to an admissible stationary Gaussian distribution over a finite interval, to be maintained at that stationary distribution thereafter by constant-gain state-feedback control.

eess.SY↗
arXiv2014

Optimal steering of inertial particles diffusing anisotropically with losses

Exploiting a fluid dynamic formulation for which a probabilistic counterpart might not be available, we extend the theory of Schroedinger bridges to the case of inertial particles with losses and general, possibly singular diffusion coefficient. We find that, as for the case of constant diffusion coefficient matrix, the optimal control law is obtained by solving a system of two p.d.e.'s involving adjoint operators and coupled through their boundary values. In the linear case with quadratic loss function, the system turns into two matrix Riccati equations with coupled split boundary conditions. An alternative formulation of the control problem as a semidefinite programming problem allows computation of suboptimal solutions. This is illustrated in one example of inertial particles subject to a constant rate killing.

math-ph↗
arXiv2014

Positive contraction mappings for classical and quantum Schrodinger systems

The classical Schrodinger bridge seeks the most likely probability law for a diffusion process, in path space, that matches marginals at two end points in time; the likelihood is quantified by the relative entropy between the sought law and a prior, and the law dictates a controlled path that abides by the specified marginals. Schrodinger proved that the optimal steering of the density between the two end points is effected by a multiplicative functional transformation of the prior; this transformation represents an automorphism on the space of probability measures and has since been studied by Fortet, Beurling and others. A similar question can be raised for processes evolving in a discrete time and space as well as for processes defined over non-commutative probability spaces. The present paper builds on earlier work by Pavon and Ticozzi and begins with the problem of steering a Markov chain between given marginals. Our approach is based on the Hilbert metric and leads to an alternative proof which, however, is constructive. More specifically, we show that the solution to the Schrodinger bridge is provided by the fixed point of a contractive map. We approach in a similar manner the steering of a quantum system across a quantum channel. We are able to establish existence of quantum transitions that are multiplicative functional transformations of a given Kraus map, but only for the case of uniform marginals. As in the Markov chain case, and for uniform density matrices, the solution of the quantum bridge can be constructed from the fixed point of a certain contractive map. For arbitrary marginal densities, extensive numerical simulations indicate that iteration of a similar map leads to fixed points from which we can construct a quantum bridge. For this general case, however, a proof of convergence remains elusive.

math-ph↗
arXiv2014

Optimal steering of a linear stochastic system to a final probability distribution

We consider the problem to steer a linear dynamical system with full state observation from an initial gaussian distribution in state-space to a final one with minimum energy control. The system is stochastically driven through the control channels; an example for such a system is that of an inertial particle experiencing random "white noise" forcing. We show that a target probability distribution can always be achieved in finite time. The optimal control is given in state-feedback form and is computed explicitely by solving a pair of differential Lyapunov equations that are coupled through their boundary values. This result, given its attractive algorithmic nature, appears to have several potential applications such as to active control of nanomechanical systems and molecular cooling. The problem to steer a diffusion process between end-point marginals has a long history (Schrödinger bridges) and therefore, the present case of steering a linear stochastic system constitutes a Schrödinger bridge for possibly degenerate diffusions. Our results, however, provide the first implementable form of the optimal control for a general Gauss-Markov process. Illustrative examples of the optimal evolution and control for inertial particles and a stochastic oscillator are provided. A final result establishes directly the property of Schrödinger bridges as the most likely random evolution between given marginals to the present context of linear stochastic systems.

eess.SY↗