Searcharxiv⌕ Search

arXiv subjects

Marco Fuhrman

Publications and source records attributed to Marco Fuhrman.

At least 19 recordsLinked to original sources
arXiv2026

Partially observed controlled Markov chains and optimal control of the Wonham filter

We consider a class of optimal control problems, with finite or infinite horizon, for a continuous-time Markov chain with finite state space. In this case, the control process affects the transition rates. We suppose that the controlled process can not be observed, and at any time the control actions are chosen based on the observation of a related stochastic process perturbed by an exogenous Brownian motion. We describe a construction of the controlled Markov chain, having stochastic transition rates adapted to the observation filtration. By a change of probability measure of Girsanov type, we introduce the so-called separated optimal control problem, where the state is the conditional (unnormalized) distribution of the controlled Markov chain and the observation process becomes a driving Brownian motion, and we prove the equivalence with the original control problem. The controlled equations for the separated problem are an instance of the Wonham filtering equations. Next we present an analysis of the separated problem: we characterize the value function as the unique viscosity solution to the dynamic programming equations (both in the parabolic and the elliptic case) we prove verifications theorems and a version of the stochastic maximum principle in the form of a necessary conditions for optimality.

math.OC↗
arXiv2026

Optimal control of McKean-Vlasov systems under partial observation and hidden Markov switching

We study a class of mean-field control problems under partial observation. The controlled dynamics are of McKean-Vlasov type and are subject to regime switching driven by a hidden Markov chain. The observation process depends on the control and on the joint distribution of the state and control, which prevents the direct application of standard filtering techniques. The main contribution of this paper is to show how this distribution dependence can be handled within a change-of-probability framework, leading to a well-posed separated control problem. We derive a Zakai equation with a specific structure for the unnormalized filter, and show that the corresponding value function satisfies a dynamic programming principle. This yields a Bellman equation posed on a convex subset of a Wasserstein space, characterizing the optimal control problem under partial observation.

math.OC↗
arXiv2025

Ergodic control of McKean-Vlasov systems on the Wasserstein space

We consider an optimal control problem with ergodic (long term average) reward for a McKean-Vlasov dynamics, where the coefficients of a controlled stochastic differential equation depend on the marginal law of the solution. Starting from the associated infinite time horizon expected discounted reward, we construct both the value $λ$ of the ergodic problem and an associated function $ϕ$, which provide a viscosity solution to an ergodic Hamilton-Jacobi-Bellman (HJB) equation of elliptic type. In contrast to previous results, we consider the function $ϕ$ and the HJB equation on the Wasserstein space, using concepts of derivatives with respect to probability measures. The pair $(λ,ϕ)$ also provides information on limit behavior of related optimization problems, for instance, results of Abelian-Tauberian type or limits of value functions of control problems for finite time horizon when the latter tends to infinity. Many arguments are simplified by the use of a functional relation for $ϕ$ in the form of a suitable dynamic programming principle.

math.OC↗
arXiv2025

The randomization method in stochastic optimal control

In this paper we make a survey on the so called randomization method, a recent methodology to study stochastic optimization problems. It allows to represent the value function of an optimal control problem by a suitable backward stochastic differential equation (BSDE), by means of an auxiliary optimization problem having the same value as the starting one. This method works for a large class of control problems and provides a BSDE representation to many related PDEs of Hamilton-Jacobi-Bellman type, even in the fully non linear case. After a general informal introduction we explain the method giving full details in a basic case. Then we try to give a complete picture of the existing applications and we present some related open problems.

math.OC↗
arXiv2024

Mean-field control of non exchangeable systems

We study the optimal control of mean-field systems with heterogeneous and asymmetric interactions. This leads to considering a family of controlled Brownian diffusion processes with dynamics depending on the whole collection of marginal probability laws. We prove the well-posedness of such systems and define the control problem together with its related value function. We next prove a law invariance property for the value function which allows us to work on the set of collections of probability laws. We show that the value function satisfies a dynamic programming principle (DPP) on the flow of collections of probability measures. We also derive a chain rule for a class of regular functions along the flows of collections of marginal laws of diffusion processes. Combining the DPP and the chain rule, we prove that the value function is a viscosity solution of a Bellman dynamic programming equation in a $L^2$-set of Wasserstein space-valued functions.

math.PR↗
arXiv2019

Optimal switching problems with an infinite set of modes: an approach by randomization and constrained backward SDEs

We address a general optimal switching problem over finite horizon for a stochastic system described by a differential equation driven by Brownian motion. The main novelty is the fact that we allow for infinitely many modes (or regimes, i.e. the possible values of the piecewise-constant control process). We allow all the given coefficients in the model to be path-dependent, that is, their value at any time depends on the past trajectory of the controlled system. The main aim is to introduce a suitable (scalar) backward stochastic differential equation (BSDE), with a constraint on the martingale part, that allows to give a probabilistic representation of the value function of the given problem. This is achieved by randomization of control, i.e. by introducing an auxiliary optimization problem which has the same value as the starting optimal switching problem and for which the desired BSDE representation is obtained. In comparison with the existing literature we do not rely on a system of reflected BSDE nor can we use the associated Hamilton-Jacobi-Bellman equation in our non-Markovian framework.

math.OC↗
arXiv2017

Stochastic Maximum Principle for Optimal Control ofPartial Differential Equations Driven by White Noise

We prove a stochastic maximum principle ofPontryagin's type for the optimal control of a stochastic partial differential equationdriven by white noise in the case when the set of control actions is convex. Particular attention is paid to well-posedness of the adjoint backward stochastic differential equation and the regularity properties of its solution with values in infinite-dimensional spaces.

math.PR↗
arXiv2016

Linear-quadratic optimal control under non-Markovian switching

We study a finite-dimensional continuous-time optimal control problem on finite horizon for a controlled diffusion driven by Brownian motion, in the linear-quadratic case. We admit stochastic coefficients, possibly depending on an underlying independent marked point process, so that our model is general enough to include controlled switching systems where the switching mechanism is not required to be Markovian. The problem is solved by means of a Riccati equation, which a backward stochastic differential equation driven by the Bronwian motion and by the random measure associated to the marked point process.

math.OC↗
arXiv2016

Randomization method and backward SDEs for optimal control of partially observed path-dependent stochastic systems

We consider a unifying framework for stochastic control problem including the following features: partial observation, path-dependence (both with respect to the state and the control), and without any non-degeneracy condition on the stochastic differential equation (SDE) for the controlled state process, driven by a Wiener process. In this context, we develop a general methodology, refereed to as the randomization method, studied in [23] for classical Markovian control under full observation, and consisting basically in replacing the control by an exogenous process independent of the driving noise of the SDE. Our first main result is to prove the equivalence between the primal control problem and the randomized control problem where optimization is performed over change of equivalent probability measures affecting the characteristics of the exogenous process. The randomized problem turns out to be associated by duality and separation argument to a backward SDE, which leads to the so-called randomized dynamic programming principle and randomized equation in terms of the path-dependent filter, and then characterizes the value function of the primal problem. In particular, classical optimal control problems with partial observation affected by non-degenerate Gaussian noise fall within the scope of our framework, and are treated by means of an associated backward SDE.

math.PR↗
arXiv2016

Randomized filtering and Bellman equation in Wasserstein space for partial observation control problem

We study a stochastic optimal control problem for a partially observed diffusion. By using the control randomization method in [4], we prove a corresponding randomized dynamic programming principle (DPP) for the value function, which is obtained from a flow property of an associated filter process. This DPP is the key step towards our main result: a characterization of the value function of the partial observation control problem as the unique viscosity solution to the corresponding dynamic programming Hamilton-Jacobi-Bellman (HJB) equation. The latter is formulated as a new, fully non linear partial differential equation on the Wasserstein space of probability measures. An important feature of our approach is that it does not require any non-degeneracy condition on the diffusion coefficient, and no condition is imposed to guarantee existence of a density for the filter process solution to the controlled Zakai equation, as usually done for the separated problem. Finally, we give an explicit solution to our HJB equation in the case of a partially observed non Gaussian linear quadratic model.

math.PR↗
arXiv2016

Backward stochastic differential equation driven by a marked point process: An elementary approach with an application to optimal control

We address a class of backward stochastic differential equations on a bounded interval, where the driving noise is a marked, or multivariate, point process. Assuming that the jump times are totally inaccessible and a technical condition holds (see Assumption (A) below), we prove existence and uniqueness results under Lipschitz conditions on the coefficients. Some counter-examples show that our assumptions are indeed needed. We use a novel approach that allows reduction to a (finite or infinite) system of deterministic differential equations, thus avoiding the use of martingale representation theorems and allowing potential use of standard numerical methods. Finally, we apply the main results to solve an optimal control problem for a marked point process, formulated in a classical way.

math.PR↗
arXiv2015

Stochastic maximum principle for optimal control of a class of nonlinear SPDEs with dissipative drift

We prove a version of the stochastic maximum principle, in the sense of Pontryagin, for the finite horizon optimal control of a stochastic partial differential equation driven by an infinite dimensional additive noise. In particular we treat the case in which the non-linear term is of Nemytskii type, dissipative and with polynomial growth. The performance functional to be optimized is fairly general and may depend on point evaluation of the controlled equation. The results can be applied to a large class of non-linear parabolic equations such as reaction-diffusion equations.

math.PR↗
arXiv2015

Non-Markovian optimal stopping problems and constrained BSDEs with jump

We consider a non-Markovian optimal stopping problem on finite horizon. We prove that the value process can be represented by means of a backward stochastic differential equation (BSDE), defined on an enlarged probability space, containing a stochastic integral having a one-jump point process as integrator and an (unknown) process with a sign constraint as integrand. This provides an alternative representation with respect to the classical one given by a reflected BSDE. The connection between the two BSDEs is also clarified. Finally, we prove that the value of the optimal stopping problem is the same as the value of an auxiliary optimization problem where the intensity of the point process is controlled.

math.PR↗
arXiv2015

Constrained BSDEs representation of the value function in optimal control of pure jump Markov processes

We consider a classical finite horizon optimal control problem for continuous-time pure jump Markov processes described by means of a rate transition measure depending on a control parameter and controlled by a feedback law. For this class of problems the value function can often be described as the unique solution to the corresponding Hamilton-Jacobi-Bellman equation. We prove a probabilistic representation for the value function, known as nonlinear Feynman-Kac formula. It relates the value function with a backward stochastic differential equation (BSDE) driven by a random measure and with a sign constraint on its martingale part. We also prove existence and uniqueness results for this class of constrained BSDEs. The connection of the control problem with the constrained BSDE uses a control randomization method recently developed in the works of I. Kharroubi and H. Pham and their co-authors. This approach also allows to prove that the value function of the original non-dominated control problem coincides with the value function of an auxiliary dominated control problem, expressed in terms of equivalent changes of probability measures.

math.PR↗
arXiv2014

Long time asymptotics for fully nonlinear Bellman equations: a Backward SDE approach

We study the large time behavior of solutions to fully nonlinear parabolic equations of Hamilton-Jacobi-Bellman type arising typically in stochastic control theory with control both on drift and diffusion coefficients. We prove that, as time horizon goes to infinity, the long run average solution is characterized by a nonlinear ergodic equation. Our results hold under dissipativity conditions, and without any nondegeneracy assumption on the diffusion term. Our approach uses mainly probabilistic arguments relying on new backward SDE representation for nonlinear parabolic, elliptic and ergodic equations.

math.PR↗
arXiv2013

Dual and backward SDE representation for optimal control of non-Markovian SDEs

We study optimal stochastic control problem for non-Markovian stochastic differential equations (SDEs) where the drift, diffusion coefficients, and gain functionals are path-dependent, and importantly we do not make any ellipticity assumption on the SDE. We develop a controls randomization approach, and prove that the value function can be reformulated under a family of dominated measures on an enlarged filtered probability space. This value function is then characterized by a backward SDE with nonpositive jumps under a single probability measure, which can be viewed as a path-dependent version of the Hamilton-Jacobi-Bellman equation, and an extension to $G$ expectation.

math.PR↗
arXiv2013

Backward stochastic differential equations associated to jump Markov processes and applications

In this paper we study backward stochastic differential equations (BSDEs) driven by the compensated random measure associated to a given pure jump Markov process X on a general state space K. We apply these results to prove well-posedness of a class of nonlinear parabolic differential equations on K, that generalize the Kolmogorov equation of X. Finally we formulate and solve optimal control problems for Markov jump processes, relating the value function and the optimal control law to an appropriate BSDE that also allows to construct probabilistically the unique solution to the Hamilton-Jacobi-Bellman equation and to identify it with the value function.

math.PR↗