Constrained convex clustering for interpretable spatial domain detection in spot-based spatial transcriptomics
Popular technologies for generating spatially resolved transcriptomic data measure gene expression at the resolution of a "spot", i.e., a small tissue region 55 microns in diameter. Each spot can contain many cells of different types. In typical analyses, researchers are interested in using these data to identify and profile discrete spatial domains in tissue. In this paper, we propose a new method, DUET, which simultaneously identifies discrete spatial domains and estimates each spot's expected cell-type proportion. This allows the identified spatial domains to be characterized in terms of the underlying expected cell-type proportions, which affords interpretability and biological insight. DUET utilizes a constrained version of model-based convex clustering, and as such, can accommodate Poisson, negative binomial, normal, and other types of expression data. Moreover, our convex clustering-type criterion allows for both the number of clusters and degree of spatial smoothness to be controlled by a single tuning parameter, which can be chosen in a data-driven fashion. Through simulation studies and a real data application, we show that DUET can achieve better clustering and deconvolution performance than some existing methods.