Supercool with PPO: Exploring Supercooled Phase Transitions via Reinforcement Learning

Abstract

Gravitational waves from cosmological first-order phase transitions provide a powerful probe of hidden sectors and beyond the Standard Model physics. However, identifying phenomenologically relevant benchmark points remains computationally challenging, since viable and detectable signals typically occupy only a small fraction of the scanned parameter space. In this work, we introduce a reinforcement learning strategy based on Proximal Policy Optimization (PPO) to accelerate the search for gravitational wave signals from supercooled phase transitions in a minimal dark $U(1)_x$ sector. We construct a numerical reinforcement learning environment that maps the microscopic model parameters to the corresponding phase transition and gravitational wave observables, using a gauge-independent low-temperature formulation of the effective action. Several reward designs are developed to guide the agent toward parameter regions producing large gravitational wave amplitudes, broad frequency coverage, and detector sensitive benchmark points. We compare the PPO scans with conventional Monte Carlo scans in both narrow and broad windows of the $U(1)_x$ vacuum expectation value. Our results demonstrate that PPO provides an efficient goal-directed search strategy for gravitational wave phenomenology and offers a broadly applicable framework for learning-assisted exploration of high-dimensional scientific parameter spaces.