Higgs mediated flavor violating top quark decays t --> u_i H, u_i gamma, u_i gamma gamma, and the process gamma gamma --> t c in effective theories

Abstract

The rare top quark couplings $tu_i\gamma$ and $tu_i\gamma \gamma$ ($u_i=u,c$) induced at the one-loop level by a flavor violating $tu_iH$ vertex are studied within the context of an effective Yukawa sector that incorporates $SU_L(2)\times U_Y(1)$-invariant operators of up to dimension six. Data on the recently observed $D^0-\overline{D^0}$ mixing are employed to constrain the $tu_iH$ vertex, which is then used to predict the $t\to u_iH$, $t\to u_i\gamma$, and $t\to u_i\gamma \gamma$ decays, as well as the $\gamma \gamma \to \bar{t}u_i+t\bar{u}_i$ reaction in the context of the ILC. It is found that the $t\to cH$ and $t\to c\gamma \gamma$ decays can reach sizable branching ratios as high as $5\times 10^{-3}$ and $10^{-4}$, respectively. As for the $t\to c\gamma$ decay, it can have a branching ratio of $5\times 10^{-8}$ that is about 6 orders of magnitude larger than the standard model prediction, which, however, is still very small to be detected. As for $tc$ production, it is found that, due to the presence of a resonant effect in the convoluted cross section $\sigma(e^+e^-\to \gamma \gamma \to t\bar{c}+\bar{t}c)$, about $(0.5 - 2.7)\times 10^{3}$ $tc$ events may be produced at the ILC for a value of the Higgs mass near to the top mass.