Exotic Higgs Decays at a Muon Collider

Abstract

We study the sensitivity of a future muon collider to exotic Higgs decays in a minimal scenario of Standard Model (SM) augmented with a light singlet scalar $S$. We consider the decay $h\to SS$ and $S$'s subsequently decay back to SM. In particular, we focus on final states with four bottom quarks ($4b$), or two bottom quarks and two muons ($2b2\mu$). Analyses are performed for two muon collider benchmark configurations: center-of-mass collision energy $\sqrt{s}=3~\mathrm{TeV}$ with $1~\mathrm{ab}^{-1}$ data and $\sqrt{s}=10~\mathrm{TeV}$ with $10~\mathrm{ab}^{-1}$ data. Machine-learning techniques are applied to suppress backgrounds and mitigate jet-combinatorics effects in both channels. We find that the $4b$ mode could be sensitive to the branching ratio, BR$(h \to SS \to 4b)$, of ${\cal O}(10^{-2})$ at 3 TeV and ${\cal O}(10^{-3})$ at 10 TeV, significantly improving upon high-luminosity LHC projections. In the Higgs-portal model with $S$ coupling to SM only through mixing with the Higgs, the sensitivities to BR$(h \to SS)$ remain at the same level given ${\cal O}(1)$ branching fraction of $S$ decaying into $b$-quarks. The $2b2\mu$ mode benefits from a clean dimuon resonance and can probe BR$(h\to SS\to 2b2\mu)$ down to $10^{-5}$ level at a 10 TeV muon collider. But the sensitivity to BR$(h \to SS)$ will be significantly reduced due to the small branching fraction of $S$ decaying into muons in the Higgs portal model.