Searcharxiv⌕ Search

arXiv subjects

Alain Bensoussan

Publications and source records attributed to Alain Bensoussan.

At least 37 records · Page 2Linked to original sources

Alternating minimization for simultaneous estimation of a latent variable and identification of a linear continuous-time dynamic system

We propose an optimization formulation for the simultaneous estimation of a latent variable and the identification of a linear continuous-time dynamic system, given a single input-output pair. We justify this approach based on Bayesian maximum a posteriori estimators. Our scheme takes the form of a convex alternating minimization, over the trajectories and the dynamic model respectively. We prove its convergence to a local minimum which verifies a two point-boundary problem for the (latent) state variable and a tensor product expression for the optimal dynamics.

math.OC↗

Control on Hilbert Spaces and Application to Some Mean Field Type Control Problems

We propose a new approach to studying classical solutions of the Bellman equation and Master equation for mean field type control problems, using a novel form of the "lifting" idea introduced by P.-L. Lions. Rather than studying the usual system of Hamilton-Jacobi/Fokker-Planck PDEs using analytic techniques, we instead study a stochastic control problem on a specially constructed Hilbert space, which is reminiscent of a tangent space on the Wasserstein space in optimal transport. On this Hilbert space we can use classical control theory techniques, despite the fact that it is infinite dimensional. A consequence of our construction is that the mean field type control problem appears as a special case. Thus we preserve the advantages of the lifiting procedure, while removing some of the difficulties. Our approach extends previous work by two of the coauthors, which dealt with a deterministic control problem for which the Hilbert space could be generic.

math.PR↗

Mean Field Type Control Problems, Some Hilbert-space-valued FBSDEs, and Related Equations

In this article, we provide an original systematic global-in-time analysis of mean field type control problems on $\mathbb{R}^n$ with generic cost functionals by the modified approach but not the same, firstly proposed in [7], as the ``lifting'' idea introduced by P. L. Lions. As an alternative to the recent popular analytical method by tackling the master equation, we resolve the control problem in a certain proper Hilbert subspace of the whole space of $L^2$ random variables, it can be regarded as tangent space attached at the initial probability measure. The present work also fills the gap of the global-in-time solvability and extends the previous works of [7,11] which only dealt with quadratic cost functionals in control; the problem is linked to the global solvability of the Hilbert-space-valued forward-backward stochastic differential equation (FBSDE), which is solved by variational techniques here. We also rely on the Jacobian flow of the solution to this FBSDE to establish the regularities of the value function, including its linearly functional differentiability, which leads to the classical well-posedness of the Bellman equation. Together with the linear functional derivatives and the gradient of the linear functional derivatives of the solution to the FBSDE, we also obtain the classical well-posedness of the master equation.

math.OC↗

The reproducing kernel Hilbert spaces underlying linear SDE Estimation, Kalman filtering and their relation to optimal control

It is often said that control and estimation problems are in duality. Recently, in (Aubin-Frankowski,2021), we found new reproducing kernels in Linear-Quadratic optimal control by focusing on the Hilbert space of controlled trajectories, allowing for a convenient handling of state constraints and meeting points. We now extend this viewpoint to estimation problems where it is known that kernels are the covariances of stochastic processes. Here, the Markovian Gaussian processes stem from the linear stochastic differential equations describing the continuous-time dynamics and observations. Taking extensive care to require minimal invertibility requirements on the operators, we give novel explicit formulas for these covariances. We also determine their reproducing kernel Hilbert spaces, stressing the symmetries between a space of forward-time trajectories and a space of backward-time information vectors. The two spaces play an analogue role for filtering to Sobolev spaces in variational analysis, and allow to recover the Kalman estimate through a direct variational argument. For comparison, we then recover the Kalman filter and smoother formulas through more classical arguments based on the innovation process. Extension to discrete-time observations or infinite-dimensional state, tough technical, would be straightforward.

math.OC↗

Operator-valued Kernels and Control of Infinite dimensional Dynamic Systems

The Linear Quadratic Regulator (LQR), which is arguably the most classical problem in control theory, was recently related to kernel methods in (Aubin-Frankowski, SICON, 2021) for finite dimensional systems. We show that this result extends to infinite dimensional systems, i.e.\ control of linear partial differential equations. The quadratic objective paired with the linear dynamics encode the relevant kernel, defining a Hilbert space of controlled trajectories, for which we obtain a concise formula based on the solution of the differential Riccati equation. This paves the way to applying representer theorems from kernel methods to solve infinite dimensional optimal control problems.

math.OC↗

A Deep Learning Approximation of Non-Stationary Solutions to Wave Kinetic Equations

We present a deep learning approximation, stochastic optimization based, method for wave kinetic equations. To build confidence in our approach, we apply the method to a Smoluchowski coagulation equation with multiplicative kernel for which an analytic solution exists. Our deep learning approach is then used to approximate the non-stationary solution to a 3-wave kinetic equation corresponding to acoustic wave systems. To validate the neural network approximation, we compare the decay rate of the total energy with previously obtained theoretical results. A finite volume solution is presented and compared with the present method.

math.NA↗

Value-Gradient based Formulation of Optimal Control Problem and Machine Learning Algorithm

Optimal control problem is typically solved by first finding the value function through Hamilton-Jacobi equation (HJE) and then taking the minimizer of the Hamiltonian to obtain the control. In this work, instead of focusing on the value function, we propose a new formulation for the gradient of the value function (value-gradient) as a decoupled system of partial differential equations in the context of continuous-time deterministic discounted optimal control problem. We develop an efficient iterative scheme for this system of equations in parallel by utilizing the properties that they share the same characteristic curves as the HJE for the value function. For the theoretical part, we prove that this iterative scheme converges linearly in $L_α^2$ sense for some suitable exponent $α$ in a weight function. For the numerical method, we combine characteristic line method with machine learning techniques. Specifically, we generate multiple characteristic curves at each policy iteration from an ensemble of initial states, and compute both the value function and its gradient simultaneously on each curve as the labelled data. Then supervised machine learning is applied to minimize the weighted squared loss for both the value function and its gradients. Experimental results demonstrate that this new method not only significantly increases the accuracy but also improves the efficiency and robustness of the numerical estimates, particularly with less amount of characteristics data or fewer training steps.

math.OC↗

Control in Hilbert Space and First Order Mean Field Type Problem

We extend the work \cite{bensoussan2019control} by two of the coauthors, which dealt with a deterministic control problem for which the Hilbert space could be generic and investigated a novel form of the `lifting' technique proposed by P. L. Lions. In \cite{bensoussan2019control}, we only showed the local existence and uniqueness of solutions to the FBODEs in the Hilbert space which were associated to the control problems with drift function consisting of the control only. In this article, we establish the global existence and uniqueness of the solutions to the FBODEs in Hilbert space corresponding to control problems with separable drift function which is nonlinear in state and linear in control. We shall also prove the sufficiency of the Pontryagin Maximum Principle and derive the corresponding Bellman equation. Besides, we shall show an analogue in the stationary case. Finally, by using the `lifting' idea as in \cite{stochasticv2,stochasticv1}, we shall apply the result to solve the linear quadratic mean field type control problems, and to show the global existence of the corresponding Bellman equations.

math.OC↗

Optimal periodic replenishment policies for spectrally positive Lévy demand processes

We consider a version of the stochastic inventory control problem for a spectrally positive Lévy demand process, in which the inventory can only be replenished at independent exponential times. We show the optimality of a periodic barrier replenishment policy that restocks any shortage below a certain threshold at each replenishment opportunity. The optimal policies and value functions are concisely written in terms of the scale functions. Numerical results are also provided.

math.OC↗

Machine Learning and Control Theory

We survey in this article the connections between Machine Learning and Control Theory. Control Theory provide useful concepts and tools for Machine Learning. Conversely Machine Learning can be used to solve large control problems. In the first part of the paper, we develop the connections between reinforcement learning and Markov Decision Processes, which are discrete time control problems. In the second part, we review the concept of supervised learning and the relation with static optimization. Deep learning which extends supervised learning, can be viewed as a control problem. In the third part, we present the links between stochastic gradient descent and mean-field theory. Conversely, in the fourth and fifth parts, we review machine learning approaches to stochastic control problems, and focus on the deterministic case, to explain, more easily, the numerical algorithms.

cs.LG↗

Identification of linear dynamical systems and machine learning

The topic of identification of dynamic systems, has been at the core of modern control , following the fundamental works of Kalman. Realization Theory has been one of the major outcomes in this domain, with the possibility of identifying a dynamic system from an input-output relationship. The recent development of machine learning concepts has rejuvanated interest for identification. In this paper, we review briefly the results of realization theory, and develop some methods inspired by Machine Learning concepts.

math.OC↗

Mean Field approach to stochastic control with partial information

The classical stochastic control problem under partial information can be formulated as a control problem for Zakai equation, whose solution is the unnormalized conditional probability distribution of the state of the system. Zakai equation is a stochastic Fokker-Planck equation. Therefore, the problem to be solved is similar to that met in Mean Field Control theory. Since Mean Field Control theory is much posterior to the development of Stochastic Control with partial information, the tools, techniques, and concepts obtained in the last decade, for Mean Field Games and Mean field type Control theory, have not been used for the control of Zakai equation. Our objective is to connect the two theories. We get the power of new tools, and we get new insights for the problem of stochastic control with partial information. For mean field theory, we get new interesting applications, but also new problems. Indeed, Mean Field Control Theory leads to very complex equations, like the Master equation, which is a nonlinear infinite dimensional P.D.E., for which general theorems are hardly available, although active research in this direction is performed. Direct methods are useful to obtain regularity results. We will develop in detail the LQ regulator problem, but since we cannot just consider the Gaussian case, well-known results, such as the separation principle is not available. An important result is available in the literature, due to A. Makowsky. It describes the solution of Zakai equation for linear systems with general initial condition (non-gaussian). We show that the separation principle can be extended for quadratic pay-off functionals, but the Kalman filter is much more complex than in the gaussian case. Finally we compare our work to the work of E. Bandini et al. and we show that the example E. Bandini et al. provided does not cover ours. Our system remains nonlinear in their setting.

math.OC↗

Stochastic Control on Space of Random Variables

By extending \cite{bensoussan2015control}, we implement the proposal of Lions \cite{lions14} on studying mean field games and their master equations via certain control problems on the Hilbert space of square integrable random variables. In \cite{bensoussan2015control}, the Hilbert space could be quite general in the face of the "deterministic control problem" due to the absence of additional randomness; while the special case of $L^2$ space of square integrable random variables was brought in at the interpretation stage. The effectiveness of the approach was demonstrated by deriving Bellman equations and the first order master equations through control theory of dynamical systems valued in the Hilbert space. In our present problem for second order master equations, it connects with a stochastic control problem over the space of random variables, and it possesses an additional randomness generated by the Wiener process which cannot be detached from the randomness caused by the elements in the Hilbert space. Nevertheless, we demonstrate how to tackle this difficulty, while preserving most of the efficiency of the approach suggested by Lions \cite{lions14}.

math.OC↗

Mean Field Control and Mean Field Game Models with Several Populations

In this paper, we investigate the interaction of two populations with a large number of indistinguishable agents. The problem consists in two levels: the interaction between agents of a same population, and the interaction between the two populations. In the spirit of mean field type control (MFC) problems and mean field games (MFG), each population is approximated by a continuum of infinitesimal agents. We define four different problems in a general context and interpret them in the framework of MFC or MFG. By calculus of variations, we derive formally in each case the adjoint equations for the necessary conditions of optimality. Importantly, we find that in the case of a competition between two coalitions, one needs to rely on a system of Master equations in order to describe the equilibrium. Examples are provided, in particular linear-quadratic models for which we obtain systems of ODEs that can be related to Riccati equations.

math.OC↗

Bellman systems with mean field dependent dynamics

We deal with nonlinear elliptic and parabolic systems that are the Bellman like systems associated to stochastic differential games with mean field dependent dynamics. The key novelty of the paper is that we allow heavily mean field dependent dynamics. This in particular leads to a system of PDE's with critical growth, for which it is rare to have an existence and/or regularity result. In the paper, we introduce a structural assumptions that cover many cases in stochastic differential games with mean filed dependent dynamics for which we are able to establish the existence of a weak solution. In addition, we present here a completely new method for obtaining the maximum/minimum principles for systems with critical growths, which is a starting point for further existence and also qualitative analysis.

math.AP↗

Risk-Sensitive Mean-Field-Type Control

We study risk-sensitive optimal control of a stochastic differential equation (SDE) of mean-field type, where the coefficients are allowed to depend on some functional of the law as well as the state and control processes. Moreover the risk-sensitive cost functional is also of mean-field type. We derive optimality equations in infinite dimensions connecting dual functions associated with Bellman functional to the adjoint process of the Pontryagin maximum principle. The case of linear-exponentiated quadratic cost and its connection with the risk-neutral solution is discussed.

math.OC↗

Mean-field-game model for Botnet defense in Cyber-security

We initiate the analysis of the response of computer owners to various offers of defence systems against a cyber-hacker (for instance, a botnet attack), as a stochastic game of a large number of interacting agents. We introduce a simple mean-field game that models their behavior. It takes into account both the random process of the propagation of the infection (controlled by the botner herder) and the decision making process of customers. Its stationary version turns out to be exactly solvable (but not at all trivial) under an additional natural assumption that the execution time of the decisions of the customers (say, switch on or out the defence system) is much faster that the infection rates.

math.OC↗

Existence and uniqueness of solutions for Bertrand and Cournot mean field games

We study a system of partial differential equations used to describe Bertrand and Cournot competition among a continuum of producers of an exhaustible resource. By deriving new a priori estimates, we prove the existence of classical solutions under general assumptions on the data. Moreover, under an additional hypothesis we prove uniqueness. Keywords: mean field games, Hamilton-Jacobi, Fokker-Planck, coupled systems, optimal control, nonlinear partial differential equations

math.AP↗