CRUX: Clinical-Text Anchoring for Cross-Institutional CT-Pathology Fusion in Non-Small Cell Lung Cancer Survival Prediction

Abstract

Accurate prognosis in non-small cell lung cancer (NSCLC) benefits from integrating computed tomography (CT) and histopathological whole-slide images (WSIs), which capture complementary macroscopic and microscopic disease characteristics. Fusing these modalities is difficult: their features come from different pretrained foundation-model encoders and lack spatial correspondence, and both are sensitive to institution-specific acquisition and staining, causing multimodal models to overfit site-specific artifacts and generalize poorly. We present CRUX (CRoss-modal Unification via teXt anchoring), a fusion framework that uses clinical text---encoded by a biomedical language model---as an institution-invariant semantic anchor. Because clinical variables follow standardized classification systems, their embeddings occupy a representation space that is consistent across institutions, providing a stable reference frame for aligning heterogeneous imaging features. CRUX introduces a text-mediated cross-modal bridge in which CT- and pathology-derived text-query summaries exchange information within a compact text semantic space through a learnable hub token, avoiding direct image-space cross-attention that would propagate domain-specific artifacts. We evaluate on 2,157 NSCLC patients from five hospitals, holding out one hospital (N=446) entirely for external validation. CRUX attains an external concordance index (C-index) of 0.761, the highest among all unimodal, bimodal, and trimodal configurations. Notably, direct CT-WSI fusion degrades under distribution shift, whereas text anchoring restores and improves cross-institutional generalization. These results indicate that language-model-encoded clinical text is an effective structured prior for robust, generalizable multimodal cancer prognosis.