arXiv subjects
Publications and source records attributed to Vladimir Gaitsgory.
We study a control system resembling a singularly perturbed system whose variables are decomposed into groups that change their values with rates of different orders of magnitude. We establish that the slow trajectories of this system are dense in the set of solutions of a certain differential inclusion and discuss an implication of this result for optimal control.
In this paper, we study asymptotic properties of problems of control of stochastic discrete time systems (also known as Markov decision processes) with time averaging and time discounting optimality criteria, and we establish that the Cesàro and Abel limits of the optimal values in such problems can be evaluated with the help of a certain infinite-dimensional linear programming problem and its dual.
We consider infinite horizon optimal control problems with time averaging and time discounting criteria and give estimates for the Cesaro and Abel limits of their optimal values in the case when they depend on the initial conditions. We establish that these limits are bounded from above by the optimal value of a certain infinite dimensional (ID) linear programming (LP) problem and that they are bounded from below by the optimal value of the corresponding dual problem. (These estimates imply, in particular, that the Cesaro and Abel limits exist and are equal to each other if there is no duality gap). In addition, we obtain IDLP-based optimality conditions for the long run average optimal control problem, and we illustrate these conditions by an example.
We formulate and study the infinite dimensional linear programming (LP) problem associated with the deterministic discrete time long-run average criterion optimal control problem. Along with its dual, this LP problem allows one to characterize the optimal value of the optimal control problem. The novelty of our approach is that we focus on the general case wherein the optimal value may depend on the initial condition of the system.
We introduce and study the infinite dimensional linear programming problem which along with its dual allows one to characterize the optimal value of the deterministic long-run average optimal control problem in the general case when the latter may depend on the initial conditions of the system.
It has been recently established that a deterministic infinite horizon discounted optimal control problem in discrete time is closely related to a certain infinite dimensional linear programming problem and its dual. In the present paper, we use these results to establish necessary and sufficient optimality conditions for this optimal control problem and apply them to construct a near optimal control.
This paper is devoted to a study of infinite horizon optimal control problems with time discounting and time averaging criteria in discrete time. We establish that these problems are related to certain infinite-dimensional linear programming (IDLP) problems. We also establish asymptotic relationships between the optimal values of problems with time discounting and long-run average criteria.
Linear programming formulations for the discounted and long-run average MDPs have evolved along separate trajectories. In 2006, E. Altman conjectured that the two linear programming formulations of discounted and long-run average MDPs are, most likely, a manifestation of general properties of singularly perturbed linear programs. In this note we demonstrate that this is, indeed, the case.
The paper deals with a problem of control of a system characterized by the fact that the influence of controls on the dynamics of certain functions of state variables (called observables) is relatively weak and the rates of change of these observables are much slower than the rates of change of the state variables themselves. The contributions of the paper are twofold. Firstly, the averaged system whose solutions approximate the trajectories of the slow observables is introduced, and it is shown that the optimal value of the problem of optimal control with time discounting criterion considered on the solutions of the system with slow observables (this problem is referred to as perturbed) converges to the optimal value of the corresponding problem of optimal control of the averaged system. Secondly, a way how an asymptotically optimal control of the perturbed problem can be constructed on the basis of an optimal solution of the averaged problem is indicated, sufficient and necessary optimality conditions for the averaged problem are stated, and a way how a near optimal solution of the latter can be constructed numerically is outlined (the construction being illustrated with an example).
The paper aims at the development of an apparatus for analysis and construction of near optimal solutions of singularly perturbed (SP) optimal controls problems (that is, problems of optimal control of SP systems) considered on the infinite time horizon. We mostly focus on problems with time discounting criteria but a possibility of the extension of results to periodic optimization problems is discussed as well. Our consideration is based on earlier results on averaging of SP control systems and on linear programming formulations of optimal control problems. The idea that we exploit is to first asymptotically approximate a given problem of optimal control of the SP system by a certain averaged optimal control problem, then reformulate this averaged problem as an infinite-dimensional (ID) linear programming (LP) problem, and then approximate the latter by semi-infinite LP problems. We show that the optimal solution of these semi-infinite LP problems and their duals (that can be found with the help of a modification of an available LP software) allow one to construct near optimal controls of the SP system. We demonstrate the construction with two numerical examples.
The paper aims at the development of tools for analysis and construction of near optimal solutions of singularly perturbed (SP) optimal controls problems with long run average optimality criteria. The idea that we exploit is to first asymptotically approximate a given problem of optimal control of the SP system by a certain averaged optimal control problem, then reformulate this averaged problem as an infinite-dimensional (ID) linear programming (LP) problem, and then approximate the latter by semi-infinite LP problems. We show that the optimal solution of these semi-infinite LP problems and their duals (that can be found with the help of a modification of an available LP software) allow one to construct near optimal controls of the SP system. We demonstrate the construction with a numerical example.
We show that necessary and sufficient conditions of optimality in periodic optimization problems can be stated in terms of a solution of the corresponding HJB inequality, the latter being equivalent to a max-min type variational problem considered on the space of continuously differentiable functions. We approximate the latter with a maximin problem on a finite dimensional subspace of the space of continuously differentiable functions and show that a solution of this problem (existing under natural controllability conditions) can be used for construction of near optimal controls. We illustrate the construction with a numerical example.