Markovian restless bandits and index policies: A review

José Niño-Mora

doi:10.3390/math11071639

Home/Archive/arXiv

arXiv · 2601.13045

Markovian restless bandits and index policies: A review

José Niño-Mora

View original source ↗Read source document ↗Export metadata ↓

Abstract

The restless multi-armed bandit problem is a paradigmatic modeling framework for optimal dynamic priority allocation in stochastic models of wide-ranging applications that has been widely investigated and applied since its inception in a seminal paper by Whittle in the late 1980s. The problem has generated a vast and fast-growing literature from which a significant sample is thematically organized and reviewed in this paper. While the main focus is on priority-index policies due to their intuitive appeal, tractability, asymptotic optimality properties, and often strong empirical performance, other lines of work are also reviewed. Theoretical and algorithmic developments are discussed, along with diverse applications. The main goals are to highlight the remarkable breadth of work that has been carried out on the topic and to stimulate further research in the field.

Explore related subjects

math.GM

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

José Niño-Mora. 2026-01-19. Markovian restless bandits and index policies: A review. https://doi.org/10.3390/math11071639

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Average Chord Lengths in a Triangle

Let $P$ be a point inside a triangle $T$. We consider the average length of the chords of $T$ through $P$, where the direction of the chord is chosen uniformly. An elementary formula is obtained in terms of the distances from $P$ to the sides and vertices of the triangle. Several classical triangle centers give especially simple specializations. For example, if $I$ is the incenter, then \[ M_T(I)=\frac{2r}{\pi} \log\left(\cot\frac A4\cot\frac B4\cot\frac C4\right). \] Our main result is the sharp inequality \[ M_T(P)\le \frac{p}{\pi\sqrt3}\log(2+\sqrt3), \] valid simultaneously for every triangle of perimeter $p$ and every interior point $P$. Thus, among all such pairs $(T,P)$, the largest possible average chord length occurs only when $T$ is equilateral and $P$ is its center. The proof is an elementary symmetrization argument. We close with brief remarks relating the problem to the radial center of a convex body, the electrostatic potential center of a triangle, and dual quermassintegrals.

math.GM↗

arXiv2026

The decrease in sectional curvature of the Fisher--Rao manifold of beta distributions

In the paper, the author develops a general convexity lemma and applies it to demonstrate the monotonic decrease of a function involving polygamma functions, arising from the sectional curvature of the Fisher--Rao manifold of beta distributions.

math.GM↗

arXiv2026

A Proof of Liu's Conjecture on the Fundamental Triangle Inequality

Let $a,b,c$ be the side lengths of a triangle, and let $R$ and $r$ denote its circumradius and inradius, respectively. We prove a conjecture of Liu stating that \[\sum_{\mathrm{cyc}} \left(\frac{a(b+c-a)}{bc}\right)^k \geq 2+\left(\frac{2r}{R}\right)^k,~~k>1, \] with the reverse inequality for $0<k<1$. The proof reduces the problem to three positive variables with fixed sum and product. We also determine the equality cases.

math.GM↗