arXiv2020
We present the results of the first Dalitz plot analysis of the decay $D^{0}\to K^{-}π^{+}η$. The analysis is performed on a data set corresponding to an integrated luminosity of 953 $\rm{fb}^{-1}$ collected by the Belle detector at the asymmetric-energy $e^{+}e^{-}$ KEKB collider. The Dalitz plot is well described by a combination of the six resonant decay channels $\bar{K}^{*}(892)^0η$, $K^{-}a_0(980)^+$, $K^{-}a_2(1320)^+$, $\bar{K}^{*}(1410)^0η$, $K^{*}(1680)^-π^{+}$ and $K_2^{*}(1980)^-π^{+}$, together with $Kπ$ and $Kη$ S-wave components. The decays $K^{*}(1680)^{-}\to K^{-}η$ and $K_{2}^{*}(1980)^{-}\to K^{-}η$ are observed for the first time. We measure ratio of the branching fractions, $\frac{\mathcal{B}(D^{0}\to K^{-}π^{+}η)}{\mathcal{B}(D^{0}\to K^{-}π^{+})}=0.500\pm0.002{\rm(stat)}\pm0.020{\rm(syst)}\pm0.003{\rm (\mathcal{B}_{PDG})}$. Using the Dalitz fit result, the ratio $\frac{\mathcal{B}(K^{*}(1680)\to Kη)}{\mathcal{B}(K^{*}(1680)\to Kπ)}$ is measured to be $0.11\pm0.02{\rm(stat)}^{+0.06}_{-0.04}{\rm(syst)}\pm0.04{\rm(\mathcal{B}_{\text{PDG}})}$; this is much lower than the theoretical expectations ($\approx1$) made under the assumption that $K^{*}(1680)$ is a pure $1^{3}D_1$ state. The product branching fraction $\mathcal{B}(D^0\to [K_2^{*}(1980)^-\to K^{-}η]π^{+})=(2.2^{+1.7}_{-1.9})\times10^{-4}$ is determined. In addition, the $πη^{\prime}$ contribution to the $a_0(980)^{\pm}$ resonance shape is confirmed with 10.1$σ$ statistical significance using the three-channel Flatté model. We also measure $\mathcal{B}(D^0\to\bar{K}^{*}(892)^0η)=(1.41^{+0.13}_{-0.12})\%$. This is consistent with, and more precise than, the current world average $(1.02\pm0.30)\%$, deviates with a significance of more than $3σ$ from the theoretical predictions of (0.51-0.92)%.