arXiv subjects

Marc Walden

Publications and source records attributed to Marc Walden.

3 recordsLinked to original sources

arXiv2026

Enhancing the MADDPG Algorithm for Multi-Agent Learning via Action Inference and Importance Sampling

We investigate multi-agent deep reinforcement learning and propose two enhancements to the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm. First, we introduce a novel Action Inference mechanism that enables each agent to predict other agents' intended actions, thereby improving the accuracy and stability of its own policy. Second, we apply an importance sampling strategy, using geometric distribution, in the replay buffer to prioritize more recent and informative experiences, which helps mitigate the non-stationarity inherent in multi-agent environments. We evaluate both modifications on the discrete-action Predator-Prey task provided by the PettingZoo library, a flexible Python interface for general multi-agent reinforcement learning benchmarks. Our results indicate that Action Inference is effective in improving learning stability and inter-agent cooperation and that importance sampling using geometric distribution can lead to significant improvements in exploration efficiency over standard MADDPG. Code available at https://github.com/shaashwathsivakumar/MARL_Proj

cs.LG↗

arXiv2026

Momentum Further Constrains Sharpness at the Edge of Stochastic Stability

Recent work suggests that (stochastic) gradient descent self-organizes near an instability boundary, shaping both optimization and the solutions found. Momentum and mini-batch gradients are widely used in practical deep learning optimization, but it remains unclear whether they operate in a comparable regime of instability. We demonstrate that SGD with momentum exhibits an Edge of Stochastic Stability (EoSS)-like regime with batch-size-dependent behavior that cannot be explained by a single momentum-adjusted stability threshold. Batch Sharpness (the expected directional mini-batch curvature) stabilizes in two distinct regimes: at small batch sizes it converges to a lower plateau $2(1-\beta)/\eta$, reflecting amplification of stochastic fluctuations by momentum and favoring flatter regions than vanilla SGD; at large batch sizes it converges to a higher plateau $2(1+\beta)/\eta$, where momentum recovers its classical stabilizing effect and favors sharper regions consistent with full-batch dynamics. We further show that this aligns with linear stability thresholds and discuss the implications for hyperparameter tuning and coupling.

cs.LG↗

arXiv2026

Modeling Accessibility-Constrained Networks with Time-Weighted Graphs

Accessibility for the physically disabled is a prevalent issue on university campuses, where stairs and steep slopes make navigating campus arduous. Our work proposes a pipeline to model a college campus as a network by combining Strava and other APIs with depth-first search to derive insights into wheelchair-accessible paths. We then develop a custom Least Resistance algorithm to compute optimal paths between selected nodes and benchmark it against Dijkstra's algorithm. We highlight crucial nodes on campus using centrality measures, demonstrating that wheelchair users are significantly constrained by the lack of mobility options and accommodations. Our pipeline is designed to support future expansion in scope and accuracy, while enabling the proposal of engineering solutions to improve campus accessibility for physically disabled individuals.

physics.soc-ph↗