arXiv subjects

Henry Wong

Publications and source records attributed to Henry Wong.

5 recordsLinked to original sources

arXiv2026

GRPO Training Dynamics for Small Language Models

Group Relative Policy Optimization (GRPO) has emerged as a memory-efficient reinforcement fine-tuning (RFT) technique for reasoning-intensive tasks. How- ever, GRPO training dynamics on small language models (SLMs) remain poorly understood, limiting its reliable adoption and reproducibility in open and resource- constrained environments. In this work, we present a systematic study of GRPO fine-tuning for SLMs ranging from 1.5B to 7B parameters under a practical single- node 8xA100 compute budget. Our study spans multiple model families and reasoning domains, including mathematics, coding, and multiple-choice question answering (MCQ) in science. Across these settings, we analyze how group size affects policy convergence, training stability, and downstream benchmark per- formance. We further characterize tensor-level update dynamics during GRPO training and investigate whether the choice of LoRA target modules and layers can improve the performance of GRPO-tuned models. While our initial GRPO-tuned models outperform their base counterparts on approximately 80% of mathematical benchmark evaluations, they demonstrate limited capability on MCQ and code reasoning tasks. Guided by our mechanistic evaluations, we refined our LoRA and reward-shaping configurations to improve performance in latter domains. These findings provide practical guidance for GRPO training for SLMs.

cs.LG↗

arXiv2026

Predictable GRPO: A Closed-Form Model of Training Dynamics

We develop a first-principles reduced-order model of these dynamics. Under a single mean-field assumption that summarizes the policy by its expected reward, we reduce the GRPO update to a stochastically-forced damped oscillator whose mass, damping, and stiffness are fixed in closed form by the optimizer hyperparameters together with a single measured curvature scale -- momentum supplies the inertia, off-policy lag erodes the damping, and the group size enters, to leading order, as a noise temperature. The reduction has three consequences. First, it subsumes the empirical single-exponential saturation law as its overdamped limit, recasting the fitted plateau, timescale, and size exponent as the fixed point, inverse stiffness, and curvature-scaling exponent of the underlying potential, and adding, through the retained inertial term, the slow-start phase the single exponential cannot represent. Second, it yields predictions tied to independently measurable quantities rather than fitted ones: group-size invariance of the deterministic trajectory with a $1/G$ stationary fluctuation, a sharp stability threshold in the refresh interval, and an overdamped-to-oscillatory transition. Third, it furnishes diagnostics that separate failure modes a reward curve alone conflates -- reward hacking, advantage degeneracy, policy concentration, and dynamical instability. Across three models and two group sizes, the closed-form trajectory fits training reward to $R^2 \geq 0.91$ and the mean trajectory is group-size invariant to leading order -- on both the reward curve and out-of-distribution transfer to eight math benchmarks -- while the within-group reward spread retains a residual $G$-dependence that the leading-order temperature picture does not capture.

cs.LG↗

arXiv2025

Attacking Autonomous Driving Agents with Adversarial Machine Learning: A Holistic Evaluation with the CARLA Leaderboard

To autonomously control vehicles, driving agents use outputs from a combination of machine-learning (ML) models, controller logic, and custom modules. Although numerous prior works have shown that adversarial examples can mislead ML models used in autonomous driving contexts, it remains unclear if these attacks are effective at producing harmful driving actions for various agents, environments, and scenarios. To assess the risk of adversarial examples to autonomous driving, we evaluate attacks against a variety of driving agents, rather than against ML models in isolation. To support this evaluation, we leverage CARLA, an urban driving simulator, to create and evaluate adversarial examples. We create adversarial patches designed to stop or steer driving agents, stream them into the CARLA simulator at runtime, and evaluate them against agents from the CARLA Leaderboard, a public repository of best-performing autonomous driving agents from an annual research competition. Unlike prior work, we evaluate attacks against autonomous driving systems without creating or modifying any driving-agent code and against all parts of the agent included with the ML model. We perform a case-study investigation of two attack strategies against three open-source driving agents from the CARLA Leaderboard across multiple driving scenarios, lighting conditions, and locations. Interestingly, we show that, although some attacks can successfully mislead ML models into predicting erroneous stopping or steering commands, some driving agents use modules, such as PID control or GPS-based rules, that can overrule attacker-manipulated predictions from ML models.

cs.CR↗

arXiv2008

Sedimentation-consolidation of a double porosity material

This paper studies the sedimentation-consolidation of a double porosity material, such as lumpy clay. Large displacements and finite strains are accounted for in a multidimensional setting. Fundamental equations are derived using a phenomenological approach and non-equilibrium thermodynamics, as set out by Coussy [Coussy, Poromechanics, Wiley, Chichester, 2004]. These equations particularise to three non-linear partial differential equations in one dimensional context. Numerical implementation in a finite element code is currently being undertaken.

physics.class-ph↗

arXiv2007

Aftershock identification

Earthquake aftershock identification is closely related to the question "Are aftershocks different from the rest of earthquakes?" We give a positive answer to this question and introduce a general statistical procedure for clustering analysis of seismicity that can be used, in particular, for aftershock detection. The proposed approach expands the analysis of Baiesi and Paczuski [PRE, 69, 066106 (2004)] based on the space-time-magnitude nearest-neighbor distance $η$ between earthquakes. We show that for a homogeneous Poisson marked point field with exponential marks, the distance $η$ has Weibull distribution, which bridges our results with classical correlation analysis for unmarked point fields. We introduce a 2D distribution of spatial and temporal components of $η$, which allows us to identify the clustered part of a point field. The proposed technique is applied to several synthetic seismicity models and to the observed seismicity of Southern California.

physics.geo-ph↗