Chandradithya S Jonnalagadda

arXiv2026

Latent Actions from Factorized Transition Effects under Agent Ambiguity

Latent Action Models (LAMs) learn action-like proxies from observation. However, in multi-object or distractor-rich scenes, observations contain not only agent motion but also distractors, camera dynamics, and background changes, making recovery of the underlying action intrinsically ambiguous without supervision. We argue that the appropriate unsupervised target is therefore not the true action itself, but a state-conditioned compositional summary of the transition effects present in the scene, enabling better action alignment and more effective utilization of action supervision. To this end, we propose a two-stage framework. We first pretrain Observed Transition Factorization (OTF) to discover reusable local transition primitives using a compositional codebook. We then aggregate these primitives into compact state-conditioned latent actions, instantiated as OTF-LAM-Pixel under the standard inverse-forward dynamics framework and OTF-LAM-Dino, a decoder-free variant operating in a frozen DINOv2 representation space. Experiments show that the learned transition primitives transfer across visual appearance and morphology, while the resulting latent actions exhibit substantially stronger action alignment than existing LAMs, make more effective use of downstream action supervision, and achieve competitive or superior downstream policy performance.

cs.AI↗

arXiv2026

The Price of Paranoia: Robust Risk-Sensitive Cooperation in Non-Stationary Multi-Agent Reinforcement Learning

Cooperative equilibria are fragile. When agents learn alongside each other rather than in a fixed environment, the process of learning destabilizes the cooperation they are trying to sustain: every gradient step an agent takes shifts the distribution of actions its partner will play, turning a cooperative partner into a source of stochastic noise precisely where the cooperation decision is most sensitive. We study how this co-learning noise propagates through the structure of coordination games, and find that the cooperative equilibrium, even when strongly Pareto-dominant, is exponentially unstable under standard risk-neutral learning, collapsing irreversibly once partner noise crosses the game's critical cooperation threshold. The natural response to apply distributional robustness to hedge against partner uncertainty makes things strictly worse: risk-averse return objectives penalize the high-variance cooperative action relative to defection, widening the instability region rather than shrinking it, a paradox that reveals a fundamental mismatch between the domains where robustness is applied and instability originates. We resolve this by showing that robustness should target the policy gradient update variance induced by partner uncertainty, not the return distribution. This distinction yields an algorithm whose gradient updates are modulated by an online measure of partner unpredictability, provably expanding the cooperation basin in symmetric coordination games. To unify stability, sample complexity, and welfare consequences of this approach, we introduce the Price of Paranoia as the structural dual of the Price of Anarchy. Together with a novel Cooperation Window, it precisely characterizes how much welfare learning algorithms can recover under partner noise, pinning down the optimal degree of robustness as a closed-form balance between equilibrium stability and sample efficiency.

cs.GT↗