arXiv subjects

Muhammad Ahmed Atif

Publications and source records attributed to Muhammad Ahmed Atif.

3 recordsLinked to original sources

arXiv2026

A Controlled Study of Double DQN and Dueling DQN Under Cross-Environment Transfer

Transfer learning in deep reinforcement learning is often motivated by improved stability and reduced training cost, but it can also fail under substantial domain shift. This paper presents a controlled empirical study examining how architectural differences between Double Deep Q-Networks (DDQN) and Dueling DQN influence transfer behavior across environments. Using CartPole as a source task and LunarLander as a structurally distinct target task, we evaluate a fixed layer-wise representation transfer protocol under identical hyperparameters and training conditions, with baseline agents trained from scratch used to contextualize transfer effects. Empirical results show that DDQN consistently avoids negative transfer under the examined setup and maintains learning dynamics comparable to baseline performance in the target environment. In contrast, Dueling DQN consistently exhibits negative transfer under identical conditions, characterized by degraded rewards and unstable optimization behavior. Statistical analysis across multiple random seeds confirms a significant performance gap under transfer. These findings suggest that architectural inductive bias is strongly associated with robustness to cross-environment transfer in value-based deep reinforcement learning under the examined transfer protocol.

cs.LG↗

arXiv2026

Embodiment-Induced Coordination Regimes in Tabular Multi-Agent Q-Learning

Centralized value learning underlies a broad class of multi-agent reinforcement learning methods, but its claimed advantage is typically evaluated in settings that confound coordination structure with function approximation and partial observability. We isolate coordination structure in a fully tabular 8x8 predator-prey gridworld with explicit speed and stamina constraints, comparing all four pairings of Independent and Centralized Q-Learning across three kinematic regimes over 10 seeds. Fully independent learning (IQL-IQL) yields shorter episodes and higher predator returns than the fully centralized configuration (CQL-CQL) in every regime and every seed (Wilcoxon p = 0.00195, Cliff's delta = 1.0). Asymmetric IQL-CQL pairings produce coordination breakdowns that persist across the 40,000 episode training budget rather than resolving as transient instability. A best-response test against frozen IQL-IQL predators shows that even the strongest configuration has not converged to equilibrium at 40k episodes, so between-configuration differences reflect learning dynamics under a shared budget rather than end-state performance. We propose a mechanism we call temporal synchronization lock: a shared value function couples all agent decisions, so when one agent is stamina-limited the joint Q-value forces capable partners into suboptimal waits, while independent learners continue asynchronous pursuit. Because this pathology arises from credit assignment rather than function approximation, we conjecture analogous effects may arise in deep MARL methods that centralize credit during training. Centralized coordination is not uniformly beneficial; its advantage over independent learning shrinks or disappears under embodiment constraints, and mixed centralized-independent pairings can perform worse than either uniform choice.

cs.MA↗

arXiv2025

Bridging the Gap Between Theoretical and Practical Reinforcement Learning in Undergraduate Education

This innovative practice category paper presents an innovative framework for teaching Reinforcement Learning (RL) at the undergraduate level. Recognizing the challenges posed by the complex theoretical foundations of the subject and the need for hands-on algorithmic practice, the proposed approach integrates traditional lectures with interactive lab-based learning. Drawing inspiration from effective pedagogical practices in computer science and engineering, the framework engages students through real-time coding exercises using simulated environments such as OpenAI Gymnasium. The effectiveness of this approach is evaluated through student surveys, instructor feedback, and course performance metrics, demonstrating improvements in understanding, debugging, parameter tuning, and model evaluation. Ultimately, the study provides valuable insight into making Reinforcement Learning more accessible and engaging, thereby equipping students with essential problem-solving skills for real-world applications in Artificial Intelligence.

cs.CY↗