Searcharxiv⌕ Search

arXiv subjects

Alexandre Genadot

Publications and source records attributed to Alexandre Genadot.

12 recordsLinked to original sources
arXiv2019

A convex programming approach for discrete-time Markov decision processes under the expected total reward criterion

In this work, we study discrete-time Markov decision processes (MDPs) under constraints with Borel state and action spaces and where all the performance functions have the same form of the expected total reward (ETR) criterion over the infinite time horizon. One of our objective is to propose a convex programming formulation for this type of MDPs. It will be shown that the values of the constrained control problem and the associated convex program coincide and that if there exists an optimal solution to the convex program then there exists a stationary randomized policy which is optimal for the MDP. It will be also shown that in the framework of constrained control problems, the supremum of the expected total rewards over the set of randomized policies is equal to the supremum of the expected total rewards over the set of stationary randomized policies. We consider standard hypotheses such as the so-called continuity-compactness conditions and a Slater-type condition. Our assumptions are quite weak to deal with cases that have not yet been addressed in the literature. An example is presented to illustrate our results with respect to those of the literature.

math.PR↗
arXiv2018

Self-similar behaviour of a non-local diffusion equation with time delay

We study the asymptotic behaviour of solutions of a class of linear non-local measure-valued differential equations with time delay. Our main result states that the solutions asymptotically exhibit a parabolic like behaviour in the large times, that is precisely expressed in term of heat kernel. Our proof relies on the study of a-self-similar-rescaled family of solutions. We first identify the asymptotic behaviour of the solutions by deriving a convergence result in the sense of the Young measures. Then we strengthen this convergence by deriving suitable fractional Sobolev compactness estimates. As a by-product, our main result allows to obtain asymptotic results for a class of piecewise constant stochastic processes with memory.

math.DS↗
arXiv2017

On the expected total reward with unbounded returns for Markov decision processes

We consider a discrete-time Markov decision process with Borel state and action spaces. The performance criterion is to maximize a total expected {utility determined by unbounded return function. It is shown the existence of optimal strategies under general conditions allowing the reward function to be unbounded both from above and below and the action sets available at each step to the decision maker to be not necessarily compact. To deal with unbounded reward functions, a new characterization for the weak convergence of probability measures is derived. Our results are illustrated by examples.

math.PR↗
arXiv2017

Probabilistic and Piecewise Deterministic models in Biology

We present recent results on Piecewise Deterministic Markov Processes (PDMPs), involved in biological modeling. PDMPs, first introduced in the probabilistic literature by Davis (1984), are a very general class of Markov processes and are being increasingly popular in biological applications. They also give new interesting challenges from the theoretical point of view. We give here different examples on the long time behavior of switching Markov models applied to population dynamics, on uniform sampling in general branching models applied to structured population dynamic, on time scale separation in integrate-and-fire models used in neuroscience, and, finally, on moment calculus in stochastic models of gene expression.

math.PR↗
arXiv2017

Estimation of the average number of continuous crossings for non-stationary non-diffusion processes

Assume that you observe trajectories of a non-diffusive non-stationary process and that you are interested in the average number of times where the process crosses some threshold (in dimension $d=1$) or hypersurface (in dimension $d\geq2$). Of course, you can actually estimate this quantity by its empirical version counting the number of observed crossings. But is there a better way? In this paper, for a wide class of piecewise smooth processes, we propose estimators of the average number of continuous crossings of an hypersurface based on Kac-Rice formulae. We revisit these formulae in the uni- and multivariate framework in order to be able to handle non-stationary processes. Our statistical method is tested on both simulated and real data.

stat.ME↗
arXiv2016

Inference for conditioned Galton-Watson trees from their Harris path

Tree-structured data naturally appear in various fields, particularly in biology where plants and blood vessels may be described by trees, but also in computer science because XML documents form a tree structure. This paper is devoted to the estimation of the relative scale parameter of conditioned Galton-Watson trees. New estimators are introduced and their consistency is stated. A comparison is made with an existing approach of the literature. A simulation study shows the good behavior of our procedure on finite-sample sizes and from missing or noisy data. An application to the analysis of revisions of Wikipedia articles is also considered through real data.

math.ST↗
arXiv2016

Averaging for some simple constrained Markov processes

In this paper, a class of piecewise deterministic Markov processes with underlying fast dynamic is studied. Using a "penalty method" , an averaging result is obtained when the underlying dynamic is infinitely accelerated. The features of the averaged process, which is still a piecewise deterministic Markov process, are fully described.

math.PR↗
arXiv2016

A new characterization of the jump rate for piecewise-deterministic Markov processes with discrete transitions

Piecewise-deterministic Markov processes form a general class of non-diffusion stochastic models that involve both deterministic trajectories and random jumps at random times. In this paper, we state a new characterization of the jump rate of such a process with discrete transitions. We deduce from this result a nonparametric technique for estimating this feature of interest. We state the uniform convergence in probability of the estimator. The methodology is illustrated on a numerical example.

stat.ME↗
arXiv2014

A multi-scale study of a class of hybrid predator-prey models

We address the question of an averaging principle for a general class of multi-scale hybrid predator-prey models. We consider prey-predator models where the kinetic of the prey population, described by a differential equation, is faster than the kinetic of the predator population, described by a jump process, the two dynamics being fully coupled. An averaged model is obtained and compared to the original slow-fast system in term of probability and absorption time.

math.PR↗
arXiv2013

Piecewise deterministic Markov process - recent results

We give a short overview of recent results on a specific class of Markov process: the Piecewise Deterministic Markov Processes (PDMPs). We first recall the definition of these processes and give some general results. On more specific cases such as the TCP model or a model of switched vector fields, better results can be proved, especially as regards long time behaviour. We continue our review with an infinite dimensional example of neuronal activity. From the statistical point of view, these models provide specific challenges: we illustrate this point with the example of the estimation of the distribution of the inter-jumping times. We conclude with a short overview on numerical methods used for simulating PDMPs.

math.ST↗
arXiv2011

Averaging for a Fully-Coupled Piecewise Deterministic Markov Process in Infinite Dimension

In this paper, we consider the generalized Hodgkin-Huxley model introduced by Austin in \cite{Austin}. This model describes the propagation of an action potential along the axon of a neuron at the scale of ion channels. Mathematically, this model is a fully-coupled Piecewise Deterministic Markov Process (PDMP) in infinite dimension. We introduce two time scales in this model in considering that some ion channels open and close at faster jump rates than others. We perform a slow-fast analysis of this model and prove that asymptotically this two time scales model reduces to the so called averaged model which is still a PDMP in infinite dimension for which we provide effective evolution equations and jump rates.

math.PR↗