arXiv subjects

Elena Bandini

Publications and source records attributed to Elena Bandini.

23 records · Page 2Linked to original sources

arXiv2016

Randomized filtering and Bellman equation in Wasserstein space for partial observation control problem

We study a stochastic optimal control problem for a partially observed diffusion. By using the control randomization method in [4], we prove a corresponding randomized dynamic programming principle (DPP) for the value function, which is obtained from a flow property of an associated filter process. This DPP is the key step towards our main result: a characterization of the value function of the partial observation control problem as the unique viscosity solution to the corresponding dynamic programming Hamilton-Jacobi-Bellman (HJB) equation. The latter is formulated as a new, fully non linear partial differential equation on the Wasserstein space of probability measures. An important feature of our approach is that it does not require any non-degeneracy condition on the diffusion coefficient, and no condition is imposed to guarantee existence of a density for the filter process solution to the controlled Zakai equation, as usually done for the separated problem. Finally, we give an explicit solution to our HJB equation in the case of a partially observed non Gaussian linear quadratic model.

math.PR↗

arXiv2015

Optimal control of Piecewise Deterministic Markov Processes: a BSDE representation of the value function

We consider an infinite horizon discounted optimal control problem for piecewise deterministic Markov processes, where a piecewise open-loop control acts continuously on the jump dynamics and on the deterministic flow. For this class of control problems, the value function can in general be characterized as the unique viscosity solution to the corresponding Hamilton-Jacobi-Bellman equation. We prove that the value function can be represented by means of a backward stochastic differential equation (BSDE) on infinite horizon, driven by a random measure and with a sign constraint on its martingale part, for which we give existence and uniqueness results. This probabilistic representation is known as nonlinear Feynman-Kac formula. Finally we show that the constrained BSDE is related to an auxiliary dominated control problem, whose value function coincides with the value function of the original non-dominated control problem.

math.OC↗

arXiv2015

Existence and uniqueness for backward stochastic differential equations driven by a random measure

We study the following backward stochastic differential equation on finite time horizon driven by an integer-valued random measure $μ$ on $\mathbb R_+\times E$, where $E$ is a Lusin space, with compensator $ν(dt,dx)=dA_t\,ϕ_t(dx)$: \[ Y_t = ξ+ \int_{(t,T]} f(s,Y_{s-},Z_s(\cdot))\, d A_s - \int_{(t,T]} \int_E Z_s(x) \, (μ-ν)(ds,dx),\qquad 0\leq t\leq T. \] The generator $f$ satisfies, as usual, a uniform Lipschitz condition with respect to its last two arguments. In the literature, the existence and uniqueness for the above equation in the present general setting has only been established when $A$ is continuous or deterministic. The general case, i.e. $A$ is a right-continuous nondecreasing predictable process, is addressed in this paper. These results are relevant, for example, in the study of control problems related to Piecewise Deterministic Markov Processes (PDMPs). Indeed, when $μ$ is the jump measure of a PDMP, then $A$ is predictable (but not deterministic) and discontinuous, with jumps of size equal to 1.

math.PR↗

arXiv2015

Optimal control of semi-Markov processes with a backward stochastic differential equations approach

In the present work we employ, for the first time, backward stochastic differential equations (BSDEs) to study the optimal control of semi-Markov processes on finite horizon, with general state and action spaces. More precisely, we prove that the value function and the optimal control law can be represented by means of the solution of a class of BSDEs driven by a semi-Markov process or, equivalently, by the associated random measure. The peculiarity of the semi-Markov framework, with respect to the pure jump Markov case, consists in the proof of the relation between BSDE and optimal control problem. This is done, as usual, via the Hamilton-Jacobi-Bellman (HJB) equation, which however in the semi-Markov case is characterized by an additional differential term $\partial_a$. Taking into account the particular structure of semi-Markov processes we rewrite the HJB equation in a suitable integral form which involves a directional derivative operator $D$ related to $\partial_a$. Then, using a formula of Ito type tailor-made for semi-Markov processes and the operator $D$, we are able to prove that the BSDE provides the unique classical solution to the HJB equation, which is shown to be the value function of our control problem.

math.OC↗

arXiv2015

Constrained BSDEs representation of the value function in optimal control of pure jump Markov processes

We consider a classical finite horizon optimal control problem for continuous-time pure jump Markov processes described by means of a rate transition measure depending on a control parameter and controlled by a feedback law. For this class of problems the value function can often be described as the unique solution to the corresponding Hamilton-Jacobi-Bellman equation. We prove a probabilistic representation for the value function, known as nonlinear Feynman-Kac formula. It relates the value function with a backward stochastic differential equation (BSDE) driven by a random measure and with a sign constraint on its martingale part. We also prove existence and uniqueness results for this class of constrained BSDEs. The connection of the control problem with the constrained BSDE uses a control randomization method recently developed in the works of I. Kharroubi and H. Pham and their co-authors. This approach also allows to prove that the value function of the original non-dominated control problem coincides with the value function of an auxiliary dominated control problem, expressed in terms of equivalent changes of probability measures.

math.PR↗