Catalogue of Phonon Instabilities in Symmetry Group 191 Kagome MT$_6$Z$_6$ Materials

Abstract

Kagome materials manifest rich physical properties due to the emergence of abundant electronic phases. Here, we carry out a high-throughput first-principles study of the kagome 1:6:6 family MT$_6$Z$_6$ materials in space group 191, focusing on their phonon instability and electronic flat bands. Different MT$_6$Z$_6$ kagome candidates reveal a remarkable variety of kagome flat bands ranging from unfilled, partially filled, to fully filled. Notably, the Mn/Fe-166 compounds exhibit partially filled flat bands with a pronounced sharp peak in the density of states near the Fermi level, leading to magnetic orders that polarize the bands and stabilize the otherwise unstable phonon. When the flat bands are located away from the Fermi level, we find a large number of phonon instabilities, which can be classified into three types, based on the phonon dispersion and vibrational modes. Type-I instabilities involve the in-plane distortion of kagome nets, while type-II and type-III present out-of-plane distortion of trigonal M and Z atoms. We take MgNi$_6$Ge$_6$ and HfNi$_6$In$_6$ as examples to illustrate the possible CDW structures derived from the emergent type-I and type-II instabilities. The type-I instability in MgNi$_6$Ge$_6$ suggests a nematic phase transition, governed by the local twisting of kagome nets. The type-II instability in HfNi$_6$In$_6$ may result in a hexagonal-to-orthorhombic transition, offering insight into the formation of MT$_6$Z$_6$ in other space groups. Additionally, the predicted ScNb$_6$Sn$_6$ is analyzed as an example of the type-III instability. Our predictions suggest a vast kagome family with rich properties induced by the flat bands, possible CDW transitions, and their interplay with magnetism.