Reinforced random walks with geometric inter-transition times

Abstract

We consider interacting vertex-reinforced random walks on a finite graph, each transitioning according to independent geometric holding times of parameter $p_i \in (0,1]$. Letting $x=X(n)$ be the vector of vertex-occupation proportions up to time $n$, the one-step transition probabilities of walk $i$ are governed by $Q^i(x,p_i)=p_i\Pi^i(x)+(1-p_i)I$, where $\Pi^i(x)$ has rows equal to a probability measure $\pi^i(x)$ on the vertex set and $I$ is the identity. Its unique invariant measure is thus $\pi^i(x)$, independent of $p_i$. Consequently, the limiting points of $X(n)$ coincide with those of the simultaneous-transition model ($p_i=1$): the solutions of $x=\pi(x)$. However, almost sure convergence is non-trivial: the standard stochastic-approximation approach requires the Clark-Kushner condition, which is not immediate since the stochastic input is biased by the walk current state. We overcome this via a decomposition of the input into a martingale and a geometrically decaying correction, establishing almost sure convergence.