Searcharxiv⌕ Search
Home/Archive/arXiv

arXiv · math/0508319

Combinations and Mixtures of Optimal Policies in Unichain Markov Decision Processes are Optimal

Ronald Ortner
View original source ↗Read source document ↗Export metadata ↓

Abstract

We show that combinations of optimal (stationary) policies in unichain Markov decision processes are optimal. That is, let M be a unichain Markov decision process with state space S, action space A and policies π_j^*: S -> A (1\leq j\leq n) with optimal average infinite horizon reward. Then any combination πof these policies, where for each state i in S there is a j such that π(i)=π_j^*(i), is optimal as well. Furthermore, we prove that any mixture of optimal policies, where at each visit in a state i an arbitrary action π_j^*(i) of an optimal policy is chosen, yields optimal average reward, too.

Explore related subjects

math.COcs.DMcs.LGmath.OCmath.PR

Keep this discovery

BibTeXRIS

Ronald Ortner. 2005-08-17. Combinations and Mixtures of Optimal Policies in Unichain Markov Decision Processes are Optimal. https://arxiv.org/abs/math/0508319

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Balanced even cycles in signed graphs:Tur\'an bounds, double covers, and parity obstructions

We study Tur\'an problems for balanced even cycles in simple signed graphs, where signed subgraphs are considered up to switching. For every balanced bipartite signed graph, the signed and ordinary Tur\'an numbers differ by at most a factor of two. Our main structural results concern the underlying graphs that admit a signing in which every $2k$-cycle is unbalanced. We characterize these graphs by the absence of an odd dependence among their $2k$-cycle incidence vectors, give a cohomological formulation, and construct subgraph-minimal obstructions of arbitrarily large order. In particular, there is no finite forbidden-subgraph characterization. We also give an exact closed-walk criterion for cycles in double covers and derive a direct signed breadth-first-search upper bound. As applications, we prove \[ \hex(n,C_{+4})=\left(\frac{\sqrt2}{2}+o(1)\right)n^{3/2} \] and study the signed hexagon number $R_6(n)=\hex(n,\{C_{-3},C_{+6}\})$. We characterize the underlying graphs counted by $R_6$ and express it as an extremal problem for ordinary $C_6$-free graphs with a prescribed involution. For every sufficiently large $n$, we construct examples with $\Omega(n^{4/3})$ edges, and we give an equivariant construction attaining the coefficient obtained from the F\"uredi--Naor--Verstra\"ete lower bound by double-cover transfer. Finally, we give $n$-vertex $C_{+10}$-free signed graphs with $\Omega(n^{6/5})$ edges and use octagon examples to illustrate the limitations of theta-freeness as a signing criterion.

math.CO↗
arXiv2026

Fractional DP-colorings of $d$-degenerate locally sparse graphs

Bernshteyn, Kostochka, and Zhu (2020) introduced the notion of fractional DP-coloring, which generalizes both fractional coloring and fractional list coloring. Among several foundational results, they proved that every $d$-degenerate bipartite graph $G$ satisfies $\chi_f^{\mathrm{DP}} \le (1 + o(1))\frac{d}{\log d}$, and that this bound is optimal---a stark contrast to ordinary fractional coloring. In this paper, we extend this upper bound to all $d$-degenerate triangle-free graphs, proving that $\chi_f^{\mathrm{DP}} \le (4 + o(1))\frac{d}{\log d}$. This generalizes a recent result of Martinsson and Steiner (2025) for ordinary fractional coloring. We derive this result as a corollary of a more general upper bound concerning locally sparse graph orderings. Specifically, a $d$-degenerate graph $G$ is left $k$-locally-sparse if it admits a degeneracy ordering in which, for every vertex $v$, the subgraph induced by its back-neighbors contains at most $k$ edges. We show that if a $d$-degenerate graph $G$ is left $\frac{d^2}{f}$-locally-sparse, then \[ \chi_f^{\mathrm{DP}}(G) \le (8 + o(1))\frac{d}{\log f}. \] This immediately yields an identical upper bound on the ordinary fractional chromatic number $\chi_f(G)$, improving upon the leading constants of previously known bounds. Additionally, we establish the asymptotic sharpness of this result up to the leading constant. For any $1 \ll f \le d^2$, we construct $d$-degenerate graphs that are left $\frac{d^2}{f}$-locally-sparse and satisfy $\chi_f(G) \ge (1 - o(1))\frac{d}{\log f}$. Finally, as applications of our main theorem, we obtain improved upper bounds on the fractional DP-chromatic number of $d$-degenerate $K_{1,t,t}$-free graphs, as well as $K_{t,t,t}$-free graphs with maximum degree $\Delta$. Notably, these bounds improve upon existing results even in the setting of ordinary fractional coloring.

math.CO↗
arXiv2026

Erd\H{o}s-S\'{o}s for digraphs

It is shown that every Eulerian digraph on $n$ vertices with more than $(t-1)n$ arcs contains every oriented tree with $t$ edges. The digraphs have no loops or repeated arcs, but opposite arcs are permitted. The bound is sharp for each fixed oriented tree, as witnessed by disjoint unions of complete bidirected graphs. Previously, such tight bounds were not known, even just for directed paths. This can be considered as a directed analog of the recently proved Erd\H{o}s-S\'os conjecture. The result was proved by GPT-6 Astra.

math.CO↗