Exploring Ultralight Dark Matter Self-Coupling via the Gravitational Wave Background

Abstract

Supermassive black hole (SMBH) binary mergers are a primary source of the stochastic gravitational wave background (SGWB) in the nanohertz band, now being actively probed by pulsar timing arrays (PTAs). We investigate how ultralight dark matter (ULDM) with a quartic self-interaction, forming solitonic cores around SMBHs, modifies the SGWB through dynamical friction on the inspiralling binary. Solving the Gross--Pitaevskii--Poisson (GPP) equations numerically for halo masses $M_{\rm halo}\sim 10^{13}\,M_{\odot}$, we compute self-consistent solitonic density profiles across a range of dimensionless self-coupling parameters $\hat{\lambda}$, covering both attractive and repulsive self-interactions. We find that soliton-induced dynamical friction imprints a characteristic suppression feature in the GW strain spectrum, whose frequency location and depth are sensitive to both the ULDM mass $m$ and the coupling $\hat\lambda$. For SMBH masses $M_{\bullet}\sim 10^{8.5}\,M_{\odot}$ and ULDM masses in the range $m\sim(3\times10^{-22}-2\times10^{-21})\,\mathrm{eV}$, current PTA observations from NANOGrav, EPTA, and PPTA may place qualitative limits on the attractive and repulsive self-coupling of ULDM particles. ULDM masses exceeding ${\sim}\,1.6\times10^{-21}\,\mathrm{eV}$ are independently excluded by soliton accretion-timescale constraints. These results demonstrate that the nanohertz GW spectrum provides a complementary probe of ULDM self-interactions, operating in a regime distinct from conventional particle-scattering or rotation-curve analyses.