arXiv subjects

Aleksei Pakharev

Publications and source records attributed to Aleksei Pakharev.

3 recordsLinked to original sources

arXiv2025

Geometric Re-Analysis of Classical MDP Solving Algorithms

We build on a recently introduced geometric interpretation of Markov Decision Processes (MDPs) to analyze classical MDP-solving algorithms: Value Iteration (VI) and Policy Iteration (PI). First, we develop a geometry-based analytical apparatus, including a transformation that modifies the discount factor $γ$, to improve convergence guarantees for these algorithms in several settings. In particular, one of our results identifies a rotation component in the VI method, and as a consequence shows that when a Markov Reward Process (MRP) induced by the optimal policy is irreducible and aperiodic, the asymptotic convergence rate of value iteration is strictly smaller than $γ$.

cs.LG↗

arXiv2025

MDP Geometry, Normalization and Reward Balancing Solvers

We present a new geometric interpretation of Markov Decision Processes (MDPs) with a natural normalization procedure that allows us to adjust the value function at each state without altering the advantage of any action with respect to any policy. This advantage-preserving transformation of the MDP motivates a class of algorithms which we call Reward Balancing, which solve MDPs by iterating through these transformations, until an approximately optimal policy can be trivially found. We provide a convergence analysis of several algorithms in this class, in particular showing that for MDPs for unknown transition probabilities we can improve upon state-of-the-art sample complexity results.

cs.LG↗

arXiv2019

Weyl-Kac character formula for affine Lie algebra in Deligne's category

We study the characters of simple modules in the parabolic BGG category of the affine Lie algebra in Deligne's category. More specifically, we take the limit of Weyl-Kac formula to compute the character of the irreducible quotient $L(X,k)$ of the parabolic Verma module $M(X,k)$ of level $k$, where $X$ is an indecomposable object of Deligne's category $\underline{\mathrm{Rep}}(GL_t)$, $\underline{\mathrm{Rep}}(O_t)$, or $\underline{\mathrm{Rep}}(Sp_t)$, under conditions that the highest weight of $X$ plus the level gives a fundamental weight, $t$ is transcendental, and the base field $\Bbbk$ has characteristic $0$. We compare our result to the partial result of Etingof, and evaluate the characters to the categorical dimensions to get a categorical interpretation of the Nekrasov-Okounkov hook length formula.

math.RT↗