Selection principles for quasi-stationary distributions and reinforcement processes

Abstract

Let $P$ be a sub-Markov matrix on a finite set $S$, representing the transition probabilities of a Markov chain on \(S\) absorbed at a cemetery point $\partial\notin S$. We consider a reinforced process \((X_n,\mu_n)\) defined as follows: $(X_n)$ behaves like a chain with kernel $P$ until it dies, and when it dies at time $n$, it is instantaneously ``resurrected'' at a point sampled according to its weighted past occupation measure $$ \mu_n = \frac{1}{W_n} \left( w_0\mu_0+\sum_{k=1}^n w_k\delta_{X_k} \right), \qquad W_n=\sum_{k=0}^n w_k, $$ where the positive weights $w_k$ satisfy certain technical assumptions, a typical example being given by $w_k = k^q$, with $q\geq -1$. When $P$ is irreducible, the behaviour of $(\mu_n)$ is well understood \cite{AFP}, \cite{bansaye2022non}: it converges almost surely toward the unique quasi-stationary distribution (QSD) of $P$. The purpose of this paper is to investigate the general situation where $P$ is not irreducible. Under generic assumptions on $P$, there are finitely many QSDs. We prove that the asymptotic selection depends on the summability of the inverse cumulative weights $1/W_n$. If $$ \sum_{n\geq 0}\frac1{W_n}=\infty, $$ then $(\mu_n)$ almost surely converges toward the QSD associated with the largest Perron value. If instead $$ \sum_{n\geq0}\frac1{W_n}<\infty, $$ then each QSD is selected with positive probability. In particular, for polynomial weights $w_0=1$ and $w_k=k^q$, $k\geq1$, this gives almost sure selection of the QSD with largest Perron value for $-1\leq q\leq 0$, whereas each quasi-stationary distribution is selected with positive probability for $q>0$.