Enhancing the sensitivity to FCNC top decays $t\to cH $ and $t\to cS $ in the boosted regime at CLIC

Abstract

The top quark, having the largest Yukawa coupling to the Higgs sector, provides a unique window into electroweak symmetry breaking and possible new physics beyond the Standard Model. Searches for rare top-quark processes are thus powerful probes of new physics. In this work, we investigate the flavor-changing neutral-current (FCNC) top-quark decays $t\to cH$ and $t\to cS$, where $S$ denotes a light scalar, at the Compact Linear Collider (CLIC) with a center-of-mass energy of $\sqrt{s}=1.5~\mathrm{TeV}$. Our analysis focuses on a kinematic regime distinct from most previous studies, in which the top quarks are typically highly boosted. To enhance signal discrimination in the boosted regime, we construct multi-channel jet images and employ a convolutional neural network (CNN) classifier to capture jet-substructure patterns relevant to the FCNC signals. Assuming an integrated luminosity of $4~\mathrm{ab}^{-1}$, we obtain the expected $95\%$ C.L. upper limit $\mathrm{BR}(t\to cH)\times \mathrm{BR}(H\to b\bar b)<5.27\times10^{-5}$. For the exotic scalar singlet, expected $95\%$ C.L. upper limits between $3.25\times10^{-5}$ and $5.26\times10^{-5}$ are obtained for $\mathrm{BR}(t\to cS)\times \mathrm{BR}(S\to b\bar b)$, for scalar masses between $30$ and $80~\mathrm{GeV}$.