Searcharxiv⌕ Search

arXiv subjects

Salah Chikhi

Publications and source records attributed to Salah Chikhi.

4 recordsLinked to original sources

Adaptive Scheduling Pipeline for Multi-Instance Asynchronous Reinforcement Learning

We present the Adaptive Scheduling Pipeline (ASP), an offline rollout scheduler for multi-instance asynchronous reinforcement learning (RL) of large language models (LLMs). ASP jointly reduces three sources of rollout underutilization - padding bubbles from intra-minibatch length heterogeneity, skewness bubbles from long-tail outputs, and sorting bubbles from fluctuations in training-queue occupancy induced by output-length-sorted dispatch - through a single offline preprocessing pass. This enables efficient multi-instance rollout that overcomes the early scaling plateau of single-instance inference while reducing fluctuations in the supply of generated samples to the trainer. We integrate the scheduler into an asynchronous architecture that decouples rollout from training while keeping policy lag observable and bounded. Within our asynchronous architecture, ASP's scheduling reduces mean iteration time by 10.9%. The complete system improves end-to-end throughput by up to 3.22 times and reduces iteration time by up to 76.8% relative to MindSpeed-RL, while reducing iteration time by up to 62.5% relative to verl. It also scales more robustly with additional generation resources than these baselines and shows no visible convergence degradation in the measured bounded-lag regime. ASP has since been used in an internal telecom-domain RL post-training workflow, reducing the reported training period from more than three months to two months and producing a domain-enhanced model for knowledge answering and tool calling.

cs.DC↗

Taylor Representations for Model-Free RL in Networked MDPs

In Networked Markov Decision Processes, transition dynamics are often unknown and the state--action space grows rapidly with the number of agents. In this setting, Taylor representations naturally approximate $Q$-functions, but a naive order-$n$ expansion over $N$ agents requires $Θ(N^n)$ coefficients. We justify these expansions under smooth expected future local rewards with controlled derivatives. Under this condition, finite-speed information propagation and discounting imply that local-critic Taylor coefficients decay exponentially with the graph distance to the farthest agent involved. Discarding distant-agent coefficients and marginalizing then yield scalable local Taylor representations with a bound controlled by graph locality. Building on these representations, we propose a scalable model-free actor--critic algorithm, establishing finite-sample critic and near-stationarity guarantees for a linear LSTD critic. We then introduce a more expressive neural TD parameterization. Unlike prior constructive spectral methods, our approach covers settings without access to a known local dynamics map, such as hidden switched linear--quadratic regulation. Across three control benchmarks, our method matches or outperforms spectral baselines while scaling efficiently to large graphs.

stat.ML↗

Last-Iterate Performance of Gradient Descent and Relaxed Proximal Point via $s$-Composability

The notion of s-composability was introduced to compose optimized stepsize schedules while preserving sharp guarantees. We show that the same joint potential has a second use: it can serve directly as a certificate of sharp last-iterate performance. We develop this viewpoint for constant and silver schedules. For the constant schedule, we prove the s-composability statement posed as an open question by Grimmer et al. (2025), at every finite horizon, through an explicit nonnegative smooth-convex interpolation certificate. The result identifies the unique constant stepsize minimizing the worst-case final gradient norm of smooth convex gradient descent under an initial-distance bound, thereby proving the optimal stepsize, value, and uniqueness predicted by Conjecture 3 of Taylor et al. (2017) without resolving its full worst-case curve. Through the Moreau envelope, the same constant is also the unique constant relaxation minimizing the worst-case final residual of relaxed proximal point. More generally, for every positive s-composable schedule, we determine the exact worst-case final proximal objective-gap constant and give a matching one-dimensional example. Applying this result to the already s-composable original silver schedule yields its exact last-iterate objective-gap constant, closing a question left open by Wang et al. (2025).

math.OC↗

An analytical framework for the Levine hats problem: new strategies, bounds and generalizations

We study the Levine hat problem, a cooperative puzzle introduced by Lionel Levine in 2010, in which $n \geq 2$ players must simultaneously identify a black hat on their own infinite stack, each seeing only their teammates' stacks. While the optimal winning probability $V_n$ remains unknown even for $n=2$, we make three key advances. First, we develop a geometric and integral framework representing strategies as Lebesgue-measurable functions, yielding a new integral expression for $V_n$ and a unified treatment of finite and infinite stacks. Second, we construct a recursive strategy $\mathscr{S}_5$ processing hats in blocks of five, which attains the conjectured optimal probability $7/20$ for two players. Although this bound was already achieved by the known strategy $\mathscr{S}_3$, the existence of $\mathscr{S}_5$ refutes the previously held expectation that recursive strategies with block size greater than three yield no improvement, and produces a strictly better geometric convergence rate for $V_{2,h}$ as well as a new lower bound for $V_2(p)$ which improves known results for $p < 0.312$. Building upon this, we improve the geometric convergence rate of $V_{2,h}$ up to the near-optimal $1/4^{1-\varepsilon}$ for any $\varepsilon > 0$. Third, we introduce and completely solve a generalization of the problem where players are given uncountably infinite stacks of hats, showing that the optimal winning probability in this setting equals exactly $1/2$ for all $n \geq 2$. This new formulation allows to study the original combinatorial problem using tools from analytic optimization, and provides a natural framework for computing optimal responses to fixed strategies.

math.CO↗