scMEDAL for the interpretable analysis of single-cell transcriptomics data with batch effect visualization using a deep mixed effects autoencoder

Abstract

Single-cell RNA sequencing enables high-resolution analysis of cellular heterogeneity, yet disentangling biological signal from batch effects remains a major challenge. Existing batch-correction algorithms suppress or discard batch-related variation rather than modeling it. We propose scMEDAL, single-cell Mixed Effects Deep Autoencoder Learning, a framework that separately models batch-invariant and batch-specific effects using two complementary subnetworks. The principal innovation, scMEDAL-RE, is a random-effects Bayesian autoencoder that learns batch-specific representations while preserving biologically meaningful information confounded with batch effects signal often lost under standard correction. Complementing it, the fixed-effects subnetwork, scMEDAL-FE, trained via adversarial learning provides a default batch-correction component. Evaluations across diverse conditions (autism, leukemia, cardiovascular), cell types, and technical and biological effects show that scMEDAL-RE produces interpretable, batch-specific embeddings that complement both scMEDAL-FE and established correction methods (scVI, Scanorama, Harmony, SAUCIE), yielding more accurate prediction of disease status, donor group, and tissue. scMEDAL also provides generative visualizations, including counterfactual reconstructions of a cell's expression as if acquired in another batch. The framework allows substitution of the fixed-effects component with other correction methods, while retaining scMEDAL-RE's enhanced predictive power and visualization. Overall, scMEDAL is a versatile, interpretable framework that complements existing correction, providing enhanced insight into cellular heterogeneity and data acquisition.