arXiv subjects

Benoîte de Saporta

Publications and source records attributed to Benoîte de Saporta.

At least 19 recordsLinked to original sources

arXiv2026

Bridging Impulse Control of Piecewise Deterministic Markov Processes and Markov Decision Processes: Frameworks, Extensions, and Open Challenges

Control theory plays a pivotal role in understanding and optimizing the behavior of complex dynamical systems across various scientific and engineering disciplines. Two key frameworks that have emerged for modeling and solving control problems in stochastic systems are piecewise deterministic Markov processes (PDMPs) and Markov decision processes (MDPs). Each framework has its unique strengths, and their intersection offers promising opportunities for tackling a broad class of problems, particularly in the context of impulse controls and decision-making in complex systems. The relationship between PDMPs and MDPs is a natural subject of exploration, as embedding impulse control problems for PDMPs into the MDP framework could open new avenues for their analysis and resolution. Specifically, this integration would allow leveraging the computational and theoretical tools developed for MDPs to address the challenges inherent in PDMPs. On the other hand, PDMPs can offer a versatile and simple paradigm to model continuous time problems that are often described as discrete-time MDPs parametrized by complex transition kernels. This transformation has the potential to bridge the gap between the two frameworks, enabling solutions to previously intractable problems and expanding the scope of both fields. This paper presents a comprehensive review of two research domains, illustrated through a recurring medical example. The example is revisited and progressively formalized within the framework of thevarious concepts and objects introduced

stat.ME↗

arXiv2026

Estimating relapse time distribution from longitudinal biomarker trajectories using iterative regression and continuous time Markov processes

Biomarker measurements obtained by blood sampling are often used as a non-invasive means of monitoring tumour progression in cancer patients. Diseases evolve dynamically over time, and studying longitudinal observations of specific biomarkers can help to understand patients response to treatment and predict disease progression. We propose a novel iterative regression-based method to estimate changes in patients status within a cohort that includes censored patients, and illustrate it on clinical data from myeloma cases. We formulate the relapse time estimation problem in the framework of Piecewise Deterministic Markov processes (PDMP), where the Euclidean component is a surrogate biomarker for patient state. This approach enables continuous-time estimation of the status-change dates, which in turn allows for accurate inference of the relapse time distribution. A key challenge lies in the partial observability of the process, a complexity that has been rarely addressed in previous studies. . We evaluate the performance of our procedure through a simulation study and compare it with different approaches. This work is a proof of concept on biomarker trajectories with simple behaviour, but our method can easily be extended to more complex dynamics.

stat.AP↗

arXiv2025

Numerical method to solve impulse control problems for partially observed piecewise deterministic Markov processes

Designing efficient and rigorous numerical methods for sequential decision-making under uncertainty is a difficult problem that arises in many applications frameworks. In this paper we focus on the numerical solution of a subclass of impulse control problem for piecewise deterministic Markov process (PDMP) when the jump times are hidden. We first state the problem as a partially observed Markov decision process (POMDP) on a continuous state space and with controlled transition kernels corresponding to some specific skeleton chains of the PDMP. Then we proceed to build a numerically tractable approximation of the POMDP by tailor-made discretizations of the state spaces. The main difficulty in evaluating the discretization error comes from the possible random jumps of the PDMP between consecutive epochs of the POMDP and requires special care. Finally we discuss the practical construction of discretization grids and illustrate our method on simulations.

math.ST↗

arXiv2024

Estimating parameters of continuous-time multi-chain hidden Markov models for infectious diseases

This study aims to estimate the parameters of a stochastic exposed-infected epidemiological model for the transmission dynamics of notifiable infectious diseases, based on observations related to isolated cases counts only. We use the setting of hidden multi-chain Markov models and adapt the Baum-Welch algorithm to the special structure of the multi-chain. From the estimated transition matrix, we retrieve the parameters of interest (contamination rates, incubation rate, and isolation rate) from analytical expressions of the moments and Monte Carlo simulations. The performance of this approach is investigated on synthetic data, together with an analysis of the impact of using a model with one less compartment to fit the data in order to help for model selection.

stat.AP↗

arXiv2024

Parameter estimation for a hidden linear birth and death process with immigration

In this paper, we use a linear birth and death process with immigration to model infectious disease propagation when contamination stems from both person-to-person contact and contact with the environment. Our aim is to estimate the parameters of the process. The main originality and difficulty comes from the observation scheme. Counts of infected population are hidden. The only data available are periodic cumulated new retired counts. Although very common in epidemiology, this observation scheme is mathematically challenging even for such a standard stochastic process. We first derive an analytic expression of the unknown parameters as functions of well-chosen discrete time transition probabilities. Second, we extend and adapt the standard Baum-Welch algorithm in order to estimate the said discrete time transition probabilities in our hidden data framework. The performance of our estimators is illustrated both on synthetic data and real data of typhoid fever in Mayotte.

math.ST↗

arXiv2024

Medical follow-up optimization: A Monte-Carlo planning strategy

Designing patient-specific follow-up strategy is a crucial step towards personalized medicine in cancer. Tools to help doctors deciding on treatment allocation together with next visit date, based on patient preferences and medical observations, would be particularly beneficial. Such tools should be based on realistic models of disease progress under the impact of medical treatments, involve the design of (multi-)objective functions that a treatment strategy should optimize along the patient's medical journey, and include efficient resolution algorithms to optimize personalized follow-up by taking the patient's history and preferences into account. We propose to model cancer evolution with a Piecewise Deterministic Markov Process where patients alternate between remission and relapse phases with disease-specific tumor evolution. This model is controlled via the online optimization of a long-term cost function accounting for treatment side-effects, hospital visits burden and disease impact on the quality of life. Optimization is based on noisy measurements of blood markers at visit dates. We leverage the Partially-Observed Monte-Carlo Planning algorithm to solve this continuous-time, continuous-state problem, taking advantage of the nearly-deterministic nature of cancer evolution. We show that this approximate solution approach of the exact model performs better than the counterpart exact resolution of the discrete model, while allowing for more versatility in the cost function model.

math.OC↗

arXiv2020

Long-time behavior and darwinian optimality for an asymmetric size-structured branching process

We study the long time behavior of an asymmetric size-structured measure-valued growth-fragmentation branching process that models the dynamics of a population of cells taking into account physiological and morphological asymmetry at division. We show that the process exhibits a Malthusian behavior; that is that the global population size grows exponentially fast and that the trait distribution of individuals converges to some stable distribution. The proof is based on a generalization of Lyapunov function techniques for non-conservative semi-groups. We then investigate the fluctuations of the growth rate with respect to the parameters guiding asymmetry. In particular, we exhibit that, under some special assumptions, asymmetric division is optimal in a Darwinian sense.

math.PR↗

arXiv2018

Optimal stopping for measure-valued piecewise deterministic Markov processes

This paper investigates the random horizon optimal stopping problem for measure-valued piecewise deterministic Markov processes (PDMPs). This is motivated by population dynamics applications, when one wants to monitor some characteristics of the individuals in a small population. The population and its individual characteristics can be represented by a point measure. We first define a PDMP on a space of locally finite measures. Then we define a sequence of random horizon optimal stopping problems for such processes. We prove that the value function of the problems can be obtained by iterating some dynamic programming operator. Finally we prove on a simple counter-example that controlling the whole population is not equivalent to controlling a random lineage.

math.PR↗

arXiv2017

Change-point detection for Piecewise Deterministic Markov Processes

We consider a change-point detection problem for a simple class of Piecewise Deterministic Markov Processes (PDMPs). A continuous-time PDMP is observed in discrete time and through noise, and the aim is to propose a numerical method to accurately detect both the date of the change of dynamics and the new regime after the change. To do so, we state the problem as an optimal stopping problem for a partially observed discrete-time Markov decision process taking values in a continuous state space and provide a discretization of the state space based on quantization to approximate the value function and build a tractable stopping policy. We provide error bounds for the approximation of the value function and numerical simulations to assess the performance of our candidate policy.

math.OC↗

arXiv2017

Stochastic Control of Observer Trajectories in Bearings-only Tracking with Acoustic Signal Propagation Optimization

We present in this paper a numerical method which computes the optimal trajectory of a underwater vehicle subject to some mission objectives. The method is applied to a submarine whose goal is to best detect one or several targets, or/and to minimize its own detection range perceived by the other targets. The signal considered is acoustic propagation attenuation. Our approach is based on dynamic programming of a finite horizon Markov decision process. A quantization method is applied to fully discretize the problem and allows a numerically tractable solution. Different scenarios are considered. We suppose at first that the position and the velocity of the targets are known and in the second we suppose that they are unknown and estimated by a Kalman type filter in a context of bearings-only tracking.

math.OC↗

arXiv2016

Optimal strategies for impulse control of piecewise deterministic Markov processes

This paper deals with the general discounted impulse control problem of a piecewise deterministic Markov process. We investigate a new family of epsilon-optimal strategies. The construction of such strategies is explicit and only necessitates the previous knowledge of the cost of the no-impulse strategy. In particular, it does not require the resolution of auxiliary optimal stopping problem or the computation of the value function at each point of the state space. This approach is based on the iteration of a single-jump-or-intervention operator associated to the piecewise deterministic Markov process.

math.PR↗

arXiv2016

Partially observed optimal stopping problem for discrete-time Markov processes

This paper is dedicated to the investigation of a new numerical method to approximate the optimal stopping problem for a discrete-time continuous state space Markov chain under partial observations. It is based on a two-step discretization procedure based on optimal quantization. First,we discretize the state space of the unobserved variable by quantizing an underlying reference measure. Then we jointly discretize the resulting approximate filter and the observation process. We obtain a fully computable approximation of the value function with explicit error bounds for its convergence towards the true value fonction.

math.OC↗

arXiv2016

Optimization of a launcher integration process: a Markov decision process approach

This paper is dedicated to the numerical study of the optimization of an industrial launcher integration process. It is an original case of inventory-production system where a calendar plays a crucial role. The process is modeled using the Markov Decision Processes (MDPs) framework. Classical optimization procedures for MDPs cannot be used because of specificities of the transition law and cost function. Two simulation-based algorithms are tuned to fit this special case. We obtain a non trivial optimal policy that can be applied in practice and significantly outperforms reference policies.

math.OC↗

arXiv2016

Linear minimum mean square filters for Markov jump linear systems

New linear minimum mean square estimators are introduced in this paper by considering a cluster information structure in the filter design. The set of filters constructed in this way can be ordered in a lattice according to the refines of clusters of the Markov chain, including the linear Markovian estimator at one end (with only one cluster) and the Kalman filter at the other hand (with as many clusters as Markov states). The higher is the number of clusters, the heavier are pre-compuations and smaller is the estimation error, so that the cluster cardinality allows for a trade-off between performance and computational burden. In this paper we propose the estimator, give the formulas for pre-computation of gains, present some properties, and give an illustrative numerical example.

math.OC↗

arXiv2015

Investigation of asymmetry in E. coli growth rate

The data we analyze derives from the observation of numerous cells of the bacterium Escherichia coli (E. coli) growing and dividing. Single cells grow and divide to give birth to two daughter cells, that in turn grow and divide. Thus, a colony of cells from a single ancestor is structured as a binary genealogical tree. At each node the measured data is the growth rate of the bacterium. In this paper, we study two different data sets. One set corresponds to small complete trees, whereas the other one corresponds to long specific sub-trees. Our aim is to compare both sets. This paper is accessible to post graduate students and readers with advanced knowledge in statistics.

stat.AP↗

arXiv2014

Approximate Kalman-Bucy filter for continuous-time semi-Markov jump linear systems

The aim of this paper is to propose a new numerical approximation of the Kalman-Bucy filter for semi-Markov jump linear systems. This approximation is based on the selection of typical trajectories of the driving semi-Markov chain of the process by using an optimal quantization technique. The main advantage of this approach is that it makes pre-computations possible. We derive a Lipschitz property for the solution of the Riccati equation and a general result on the convergence of perturbed solutions of semi-Markov switching Riccati equations when the perturbation comes from the driving semi-Markov chain. Based on these results, we prove the convergence of our approximation scheme in a general infinite countable state space framework and derive an error bound in terms of the quantization error and time discretization step. We employ the proposed filter in a magnetic levitation example with markovian failures and compare its performance with both the Kalman-Bucy filter and the Markovian linear minimum mean squares estimator.

math.OC↗

arXiv2013

Random coefficients bifurcating autoregressive processes

This paper presents a model of asymmetric bifurcating autoregressive process with random coefficients. We couple this model with a Galton Watson tree to take into account possibly missing observations. We propose least-squares estimators for the various parameters of the model and prove their consistency with a convergence rate, and their asymptotic normality. We use both the bifurcating Markov chain and martingale approaches and derive new important general results in both these frameworks.

math.PR↗

arXiv2013

Statistical study of asymmetry in cell lineage data

A rigorous methodology is proposed to study cell division data consisting in several observed genealogical trees of possibly different shapes. The procedure takes into account missing observations, data from different trees, as well as the dependence structure within genealogical trees. Its main new feature is the joint use of all available information from several data sets instead of single data set estimation, to avoid the drawbacks of low accuracy for estimators or low power for tests on small single-trees. The data is modeled by an asymmetric bifurcating autoregressive process and possibly missing observations are taken into account by modeling the genealogies with a two-type Galton-Watson process. Least-squares estimators of the unknown parameters of the processes are given and symmetry tests are derived. Results are applied on real data of Escherichia coli division and an empirical study of the convergence rates of the estimators and power of the tests is conducted on simulated data.

stat.AP↗