Djamel Rassem Lamouri

arXiv2026

The Bias of Nonlinear Two-Time-scale Stochastic Approximation under Constant Step-Sizes

Two-timescale stochastic approximation (TTSA) is a fundamental tool for analyzing coupled iterative algorithms in reinforcement learning, optimization, and stochastic control. However, finite-time guarantees for nonlinear two-timescale schemes remain difficult to obtain, especially under constant step-sizes. In this paper, we study nonlinear TTSA with step-sizes $α\ggβ$. Under standard stability, regularity, and Markovian noise assumptions, we upper bound the mean-squared error and the bias of both iterates around their limiting equilibria. Our bounds scale as $O(α+β^2/α^2)$, which we prove to be tight when $β\leα^{3/2}$. The analysis separates the contributions of initial conditions, fast-timescale tracking error, Markovian dependence, and timescale coupling, thereby clarifying the origin of the $β^2/α^2$ term. Our results reveal qualitative differences from the linear TTSA setting previously studied, showing that nonlinear dynamics introduce additional finite-time effects that are absent in the linear case.

cs.LG↗

arXiv2025

Pearl: Automatic Code Optimization Using Deep Reinforcement Learning

Compilers are crucial in optimizing programs and accelerating their execution. However, optimizing programs automatically using compilers is not trivial. Recent work has attempted to use reinforcement learning (RL) to solve this problem. It has limitations though. Current methods either do not support the optimization of general loop nests or can only be used to optimize loop nests seen during training. In this paper, we propose Pearl, a novel framework that uses deep reinforcement learning to automate compiler code optimization. It uses an RL agent to select the sequence of code optimizations a compiler should apply to make the input code run faster. This agent can optimize general loop nests and can generalize to programs unseen during training. To enable the optimization of general loop nests, we propose a novel representation of the action space that allows the RL agent to select on which part of the loop nest a given code optimization should be applied. Training RL agents for loop nest optimization is slow and data-intensive. We accelerate this process by caching results and pre-training the agent. Integrated with the Tiramisu compiler, our approach streamlines optimization and outperforms existing methods. To the best of our knowledge, Pearl is the first RL-based system to support general programs composed of loop nests manipulating tensors while still being able to generalize to programs unseen during training. It is also the first to support the class of polyhedral optimizations, a class of advanced loop nest optimizations. We evaluate Pearl on a set of benchmarks, and demonstrate competitive performance improvements over state-of-the-art compilers. Notably, Pearl achieves a geometric mean speedup of 2.02x compared to Tiramisu and 3.36x compared to Pluto.

cs.PL↗