Learning Joint Graphical Model with Computational Efficiency, Dynamic Regularization, and Adaptation

Abstract

Multi-sourced datasets are common in studies of variable interactions, for example, individual-level fMRI integration, cross-domain recommendation, etc, where each source induces a related but distinct dependency structure. Joint learning of multiple graphical models (i.e., multiple precision matrices) has emerged as an important tool in analyzing such data. Unlike separate learning, joint learning can leverage shared structural patterns across graphs to yield more accurate results. In this paper, we present an efficient and adaptive method named MIGHT (\textbf{M}ulti-task \textbf{I}terative \textbf{G}raphical \textbf{H}ard \textbf{T}hresholding) to estimate multiple graphs jointly. We reformulate the joint model into a series of multi-task learning problems through a column-by-column manner, and solve these problems using a dynamic regularized algorithm based on iterative hard thresholding. This framework is inherently parallelizable and therefore efficient in computation. Theoretically, we derive the non-asymptotic error bound for the resulting estimator. Furthermore, the proposed algorithm is adaptive to heterogeneous column-wise signal strengths: for nodes with strong signals, our estimator achieves improved error bounds and selection consistency adaptively, and also exhibits asymptotic normality -- properties rarely explored in existing joint learning methods. The performance of our method is illustrated through numerical simulations and real data analysis on a cancer gene-expression RNA-seq dataset.