arXiv subjects

Nadir Farhi

Publications and source records attributed to Nadir Farhi.

At least 19 recordsLinked to original sources

arXiv2026

Reinforcement Learning-Based Control of CAV Platoon Joining Maneuvers in Mixed Traffic

Connected and automated vehicle (CAV) platooning offers a promising approach to improving road safety and traffic capacity. However, platoon control in real-world traffic is challenging due to uncertainty and heterogeneous driving behaviors. Reinforcement learning (RL) has strong potential for addressing such control problems, but its practical deployment raises challenges related to safety and learning efficiency. This paper proposes a generic modeling and simulation framework for investigating CAV platoon joining maneuvers and comparing deep reinforcement learning (DRL)-based control algorithms. The problem is particularly challenging in mixed-traffic environments, where CAVs coexist with human-driven vehicles exhibiting heterogeneous longitudinal and lateral behaviors. The objective is to achieve safe and efficient joining maneuvers by either incorporating penalties for risky behaviors into the learning process or using an external safety controller to constrain the learned policy. An agent-based modeling framework coupled with the Simulation of Urban MObility (SUMO) simulator is used to evaluate Deep Q-Network (DQN), Double Deep Q-Network (DDQN), and Proximal Policy Optimization (PPO). Results show that PPO outperforms DQN and DDQN, achieving a joining success rate of approximately 98 % and a collision rate below 1 %, largely due to risk-related penalties incorporated into the reward function. However, this improved performance requires more decision steps to complete the maneuver, revealing a trade-off between safety, joining effectiveness, and decision efficiency. An external safety controller effectively prevents collisions, although its interventions may reduce joining efficiency. The results highlight the importance of jointly considering safety and efficiency when designing RL-based controllers for CAV platoon joining in mixed traffic.

cs.LG↗

arXiv2026

Regime-Conditional Stabilisation of LLM-Augmented Cooperative Multi-Agent Reinforcement Learning

Large Language Models (LLMs) offer a natural interface for translating human objectives into reward signals for cooperative multi-agent reinforcement learning (MARL), yet the training-time dynamics of this integration remain poorly understood. We show that dynamically updating LLM-generated reward weights during off-policy MARL violates the stationarity assumption of Potential-Based Reward Shaping (PBRS) and contaminates the experience replay buffer, whose stored transitions carry reward labels computed under stale shaping weights. We characterise the result as a regime-dependent failure whose severity depends on how competent the unshaped baseline already is. To control it we propose two stabilisation strategies: a Phase-Based Freeze Schedule that enforces strict stationarity within training phases, and Exponential Moving Average (EMA) smoothing that bounds per-episode weight drift. We evaluate across three cooperative environments and five random seeds with QMIX, complemented by an exploratory VDN extension, yielding a three-regime taxonomy. In the augmentative regime (Simple Spread), where the baseline is functional (74.4 %), EMA significantly improves success to 86.7 % ($+12.3$ pp, $p<0.01$) while naive dynamic updates collapse it to 15.2 %. In the essential regime (Level-Based Foraging), where the baseline is broken (0.1 %), any shaping unlocks the task (95.9 % under EMA). In the supplementary regime (SMAC 3m), where the baseline is near-saturated (98.8 %), stabilised shaping preserves performance (99.9 %) while unstabilised shaping adds variance without gain. These findings establish reward-signal stationarity as a necessary design constraint and indicate that regime placement is a practical predictor of whether dynamic LLM shaping helps or harms.

cs.LG↗

arXiv2026

Hierarchical Reinforcement Learning for Neural Network Compression (HiReLC): Pruning and Quantization

We present HiReLC, a hierarchical ensemble-reinforcement learning framework for automated joint quantization and structured pruning of deep neural networks. The framework decomposes the compression search across two levels of abstraction: low-level agents (LLAs) operate independently per block, selecting per-kernel configurations over a multi-discrete action space spanning bitwidth, pruning keep-ratio, quantization type, and granularity, while high-level agents (HLAs) coordinate global budget allocation via ensemble voting guided by Fisher Information-based sensitivity estimates. To mitigate the computational cost of policy evaluation, an iterative active learning loop interleaves surrogate-guided RL optimization with post-compression fine-tuning, using a lightweight MLP surrogate to amortize expensive evaluations and a logit-MSE proxy during cold-start. The surrogate is used for reward shaping rather than as a replacement for final post-compression evaluation. The controller is architecture-agnostic by design, with a modular layer abstraction decoupling the RL environment from the underlying network topology. Experiments across Vision Transformer and CNN benchmarks demonstrate effective parameter-storage compression ratios of 5.99 - 6.72$\times$ with a 3.83 % gain in one setting and 0.55 - 5.62 % accuracy drops elsewhere, supporting hierarchical policy decomposition and sensitivity-aware guidance as practical design choices for joint neural network compression.

cs.LG↗

arXiv2026

Reward-Conditioned Attention: How Reward Design Shapes What Autonomous Driving Agents See

We investigate how reward design shapes the internal attention patterns of reinforcement learning agents trained for autonomous driving. Using three Perceiver-based agents that share identical architectures and training data but differ only in their reward configurations$\unicode{x2014}$ranging from basic violation penalties to continuous proximity penalties$\unicode{x2014}$we analyze cross-attention allocation across 50 real-world scenarios from the Waymo Open Motion Dataset. A central methodological finding is that naïve pooling of timesteps across episodes substantially underestimates the attention$\unicode{x2013}$risk relationship; within-episode correlation with Fisher z-transform aggregation is the appropriate statistic and reveals a robustly positive link between collision risk and agent-directed attention. Building on this validated methodology, we demonstrate two reward-conditioned effects: agents trained with navigation rewards allocate up to $2.0\times$ more attention to GPS-path tokens than those trained with additional proximity penalties$\unicode{x2014}$and $4.7\times$ more than agents with no navigation incentive$\unicode{x2014}$revealing that reward content directly determines which scene elements the encoder prioritizes, and continuous time-to-collision penalties create a $\textit{learned vigilance prior}$$\unicode{x2014}$elevated resting agent surveillance maintained throughout collision-free phases. In several scenarios, the complete-reward and minimal-reward models exhibit opposite attention$\unicode{x2013}$risk correlation directions, demonstrating that reward design can qualitatively reverse attentional strategy rather than merely modulating its magnitude. These results suggest that attention analysis is a practical diagnostic for verifying that a reward function produces the intended representational behaviour in safety-critical RL systems.

cs.LG↗

arXiv2026

NOVA: Symbolic Regression Discovery of Interpretable Car-Following and Lane-Change Models with Driver Heterogeneity

We present NOVA, an autonomous symbolic regression framework that identifies interpretable car-following and lane-change structures from raw trajectory data with minimal behavioral priors. Applied to 4,765,788 active driving observations from the NGSIM I-80 and US-101 datasets, NOVA's deterministic Rust-powered search engine evaluates over 10,000 candidate algebraic structures and identifies a compact two-term acceleration model under a forward-shifted rolling-mean prediction target. Evaluated under two complementary preprocessing pipelines, NOVA achieves $RMSE = 1.376 m/s^2$ ($R^2 = 15.57\%$) on the intent-forecasting benchmark, outperforming the best recalibrated symbolic-regression baseline (SR-LLM, PNAS~2025) by 0.135 m/s$^2$ in RMSE under an identical evaluation protocol. Across eight independent experiments, a single dominant nonlinear term emerges as a robust backbone of human car-following; a residual-guided extension further links the selected structure to an established psychophysical theory of collision avoidance. The discovered feature operators transfer zero-shot between freeway sites with under 3 pp $R^2$ loss. Extended to lane-change modelling within a multinomial logit framework, NOVA achieves 67.4\% balanced accuracy under strict vehicle-ID holdout on 502 unseen drivers, surpassing existing lane-changing baselines by +29.8 percentage points on a three-class problem.

cs.LG↗

arXiv2026

An LLM-Explainable DRL Framework for Passenger-Directed Autonomous Driving

Autonomous vehicles offer the potential for safer and more efficient mobility, yet public trust remains limited due to the lack of transparency in their decision-making. This work addresses this issue by combining deep reinforcement learning (DRL) for adaptive driving control with large language model (LLM)-based explainability modules designed to communicate agent behavior to passengers. DRL agents were trained in simulation using a Dueling Double Deep Q-Network to follow distinct driving requests: \textit{fast}, \textit{comfort}, and \textit{stop}. They demonstrated stable learning, safe compliance with traffic rules, and reliable switching between modes within a single trip. In parallel, LLM modules were introduced to interpret passenger requests, determine when explanations were needed, and generate concise, safety-oriented justifications. Results show that this framework, serving as a proof of concept for integrating RL decision-making and LLMs, balances safety, adaptability, and explainability, and is most effective when requests are delayed or overridden due to safety constraints.

cs.AI↗

arXiv2025

R2L: Reliable Reinforcement Learning: Guaranteed Return & Reliable Policies in Reinforcement Learning

In this work, we address the problem of determining reliable policies in reinforcement learning (RL), with a focus on optimization under uncertainty and the need for performance guarantees. While classical RL algorithms aim at maximizing the expected return, many real-world applications - such as routing, resource allocation, or sequential decision-making under risk - require strategies that ensure not only high average performance but also a guaranteed probability of success. To this end, we propose a novel formulation in which the objective is to maximize the probability that the cumulative return exceeds a prescribed threshold. We demonstrate that this reliable RL problem can be reformulated, via a state-augmented representation, into a standard RL problem, thereby allowing the use of existing RL and deep RL algorithms without the need for entirely new algorithmic frameworks. Theoretical results establish the equivalence of the two formulations and show that reliable strategies can be derived by appropriately adapting well-known methods such as Q-learning or Dueling Double DQN. To illustrate the practical relevance of the approach, we consider the problem of reliable routing, where the goal is not to minimize the expected travel time but rather to maximize the probability of reaching the destination within a given time budget. Numerical experiments confirm that the proposed formulation leads to policies that effectively balance efficiency and reliability, highlighting the potential of reliable RL for applications in stochastic and safety-critical environments.

cs.LG↗

arXiv2022

Estimation of road traffic state at a multi-lanes controlled junction

We present in this paper a method for the estimation of traffic state at road junctions controlled with traffic lights. We assume mixed traffic where a proportion of vehicles are equipped with communication resources. The estimation of road traffic state uses information given by communicating vehicles. The method we propose is built upon a previously published method which was applied to estimate the traffic in the case where roads are composed of two lanes. In this paper, we consider the case where roads are composed of three lanes and we show that this solution can address the general case, where roads are composed of any number of lanes. We assume the geometry of the road junction is known, as well as its connections between incoming and outgoing lanes and roads. Using the location data provided by the communicating vehicles, first, we estimate some primary parameters including the penetration ratio of the probe vehicles, as well as the arrival rates of vehicles (equipped and non-equipped) per lane by introducing the assignment onto the lanes. Second, we give estimations of the queue length of the 3-lanes road, without and with the additional information provided by the location of the communicating vehicles in the queue. We illustrate and discuss the proposed model with numerical simulations.

eess.SY↗

arXiv2021

Deep Reinforcement Q-Learning for Intelligent Traffic Signal Control with Partial Detection

Intelligent traffic signal controllers, applying DQN algorithms to traffic light policy optimization, efficiently reduce traffic congestion by adjusting traffic signals to real-time traffic. Most propositions in the literature however consider that all vehicles at the intersection are detected, an unrealistic scenario. Recently, new wireless communication technologies have enabled cost-efficient detection of connected vehicles by infrastructures. With only a small fraction of the total fleet currently equipped, methods able to perform under low detection rates are desirable. In this paper, we propose a deep reinforcement Q-learning model to optimize traffic signal control at an isolated intersection, in a partially observable environment with connected vehicles. First, we present the novel DQN model within the RL framework. We introduce a new state representation for partially observable environments and a new reward function for traffic signal control, and provide a network architecture and tuned hyper-parameters. Second, we evaluate the performances of the model in numerical simulations on multiple scenarios, in two steps. At first in full detection against existing actuated controllers, then in partial detection with loss estimates for proportions of connected vehicles. Finally, from the obtained results, we define thresholds for detection rates with acceptable and optimal performance levels.

cs.LG↗

arXiv2020

Estimation of urban traffic state with probe vehicles

We present in this paper a method to estimate urban traffic state with communicating vehicles. Vehicles moving on the links of the urban road network form queues at the traffic lights. We assume that a proportion of vehicles are equipped with localization and communication capabilities, and name them probe vehicles. First, we propose a method for the estimation of the penetration ratio of probe vehicles, as well as the vehicles arrival rate on a link. Moreover, we show that turn ratios at each junction can be estimated. Second, assuming that the turn ratios at each junction are given, we propose an estimation of the queue lengths on a 2-lanes link, by extending a 1-lane existing method. Our extension introduces vehicles assignment onto the lanes. Third, based on this approach, we propose optimal control laws for the traffic light and for the assignment of the arriving vehicles onto the lane queues. Finally, numerical simulations are conducted with Veins framework that bi-directionally couples microscopic road traffic and communication simulators. We illustrate and discuss our propositions with the simulation results.

eess.SY↗

arXiv2019

Dynamic programming systems for modeling and control of the traffic in transportation networks

This thesis is entitled Dynamic programming systems for modeling and control of the traffic in transportation networks. Two parts are distinguished in this dissertation: 1) methods and approaches based on min-plus or max-plus algebra, where the dynamics are deterministic dynamic programming systems; 2) methods and approaches whose dynamic systems are non-linear but are interpreted as stochastic dynamic programming systems. Each of the two parts includes a chapter of necessary reviews, two main chapters and a chapter summarizing other works related to the concerned part. Part 1 includes a first chapter containing an introduction and some necessary reviews; two main chapters, one on the max-plus algebra model for the train dynamics on a metro line, the other one on the network calculus approach for modeling and calculating performance bounds on road networks; and a final chapter summarizing my other contributions on the topic of this part. Part 2 includes a first chapter containing an introduction and some necessary reviews; two main chapters, one on the microscopic modeling of traffic taking into account anticipation in driving, the other one on the modeling of the train dynamics on a metro line taking into account the passenger travel demand; and a final chapter summarizing my other contributions on the topic of this part.

math.OC↗

arXiv2018

Comprehensive passenger demand-dependent traffic control on a metro line with a junction and a derivation of the traffic phases

This paper is an extended abstract on a traffic model for the train dynamics on a metro line with a junction. The dynamic model includes control laws on the dwell and on the run time, taking into account the passenger travel demand. Our model extends two existing traffic models: 1. a model that describes the train dynamics on a metro line with a junction but without taking into account the travel demand, i.e. dwell and run times are fixed, and 2. another model that controls the train dynamics on a linear metro line (without junction) taking into account the passenger travel demand. We present the phase diagram of the train dynamics in the metro line with a junction. Train time-headway and frequency are functions of 1. the number of trains on the line, 2. the difference between the number of trains on the two branches, and 3. the passenger demand level. The derived diagram can be used for the optimization of the traffic control on the metro line and at the junction.

math.OC↗

arXiv2018

Real-time control of metro train dynamics with minimization of train time-headway variance

We present here a real-time control model for the train dynamics in a linear metro line system. The model describes the train dynamics taking into account average passenger arrival rates on platforms, including control laws for train dwell and run times, based on the feedback of the train dynamics. The model extends a recently developed Max-plus linear traffic model with demand-dependent dwell times and a run time control. The extension permits the elimination of eventual irregularities on the train time-headway. The resulting train dynamics are interpreted as a dynamic programming system of a stochastic optimal control problem of a Markov chain. The train dynamics still admit a stable stationary regime with a unique average growth rate interpreted as the asymptotic average train time-headway. Moreover, beyond the transient regime of the train dynamics, our extension guarantees uniformity in time of the train time-headways at every platform.

math.OC↗

arXiv2018

A discrete event traffic model explaining the traffic phases of the train dynamics on a linear metro line with demand-dependent control

In this paper we present a mathematical model of the train dynamics in a linear metro line system with demand-dependent run and dwell times. On every segment of the line, we consider two main constraints. The first constraint is on the travel time, which is the sum of run and dwell time. The second one is on the safe separation time, modeling the signaling system, so that only one train can occupy a segment at a time. The dwell and the run times are modeled dynamically, with two control laws. The one on the dwell time makes sure that all the passengers can debark from and embark into the train. The one on the run time ensures train time-headway regularity in the case where perturbations do not exceed a run time margin. We use a Max-plus algebra approach which allows to derive analytic formulas for the train time-headway and frequency depending on the number of trains and on the passenger demand. The analytic formulas, illustrated by 3D figures, permit to understand the phases of the train dynamics of a linear metro line being operated as a transport on demand system.

math.OC↗

arXiv2018

Traffic Modeling and Real-time Control for Metro Lines

We present in this article traffic flow and control models for the train dynamics in metro lines. The first model, written in the max-plus algebra, takes into account minimum running, dwell and safety time constraints, without any control of the train dwell times at platforms, and without consideration of the passenger travel demand. We show that the dynamics are stable and converge to stationary regimes with a unique asymptotic average growth rate. Moreover, the asymptotic average train time-headway, dwell time, as well as close-in time, are derived analytically, as functions of the number of running trains on the metro line. We then introduce, in a second model, the effect of the passenger demand on the train dwell times at platforms. We review that, if this effect is not well controlled, then the traffic is unstable. Finally, we propose a traffic control model which deals with this instability, by well controlling the effect of passenger arrivals on the train dwell times at platforms. We show that the dynamics are stable and converge to stationary regimes with a unique asymptotic average growth rate. We then calculate by numerical simulations the asymptotic average time-headway as a function of the number of running trains, compare the results with those of the max-plus algebra model, and derive the effect of the passenger travel demand on the frequency of the metro line, under the proposed control model.

math.OC↗

arXiv2018

A discrete event traffic model explaining the traffic phases of the train dynamics in a metro line system with a junction

This paper presents a mathematical model for the train dynamics in a mass-transit metro line system with one symmetrically operated junction. We distinguish three parts: a central part and two branches. The tracks are spatially discretized into segments (or blocks) and the train dynamics are described by a discrete event system where the variables are the $k^{th}$ departure times from each segment. The train dynamics are based on two main constraints: a travel time constraint modeling theoretic run and dwell times, and a safe separation constraint modeling the signaling system in case where the traffic gets very dense. The Max-plus algebra model allows to analytically derive the asymptotic average train frequency as a function of many parameters, including train travel times, minimum safety intervals, the total number of trains on the line and the number of trains on each branch. This derivation permits to understand the physics of traffic. In a further step, the results will be used for traffic control.

math.OC↗

arXiv2017

A vehicle-to-infrastructure communication based algorithm for urban traffic control

We present in this paper a new algorithm for urban traffic light control with mixed traffic (communicating and non communicating vehicles) and mixed infrastructure (equipped and unequipped junctions). We call equipped junction here a junction with a traffic light signal (TLS) controlled by a road side unit (RSU). On such a junction, the RSU manifests its connectedness to equipped vehicles by broadcasting its communication address and geographical coordinates. The RSU builds a map of connected vehicles approaching and leaving the junction. The algorithm allows the RSU to select a traffic phase, based on the built map. The selected traffic phase is applied by the TLS; and both equipped and unequipped vehicles must respect it. The traffic management is in feedback on the traffic demand of communicating vehicles. We simulated the vehicular traffic as well as the communications. The two simulations are combined in a closed loop with visualization and monitoring interfaces. Several indicators on vehicular traffic (mean travel time, ended vehicles) and IEEE 802.11p communication performances (end-to-end delay, throughput) are derived and illustrated in three dimension maps. We then extended the traffic control to a urban road network where we also varied the number of equipped junctions. Other indicators are shown for road traffic performances in the road network case, where high gains are experienced in the simulation results.

eess.SY↗

arXiv2017

M/G/c/c state dependent queuing model for a road traffic system of two sections in tandem

We propose in this article a M/G/c/c state dependent queuing model for road traffic flow. The model is based on finite capacity queuing theory which captures the stationary density-flow relationships. It is also inspired from the deterministic Godunov scheme for the road traffic simulation. We first present a reformulation of the existing linear case of M/G/c/c state dependent model, in order to use flow rather than speed variables. We then extend this model in order to consider upstream traffic demand and downstream traffic supply. After that, we propose the model for two road sections in tandem where both sections influence each other. In order to deal with this mutual dependence, we solve an implicit system given by an algebraic equation. Finally, we derive some performance measures (throughput and expected travel time). A comparison with results predicted by the M/G/c/c state dependent queuing networks shows that the model we propose here captures really the dynamics of the road traffic.

math.OC↗