3D Cell Morphology--Clinical Text Contrastive Learning with Voxel Swin Transformer and Finding Encoder Domain-Adaptation
Our prior work adapted a 2D CLIP dual encoder with LoRA to a multi-source cell-microscopy corpus, reaching anomaly-scoring ROC-AUC up to 0.985. This paper advances the same contrastive-learning domain-adaptation methodology from 2D microscope images to 3D microscope volumes, pairing a 3D Swin Transformer voxel encoder (MONAI's SwinUNETR backbone) with ClinicalBERT as the finding-text encoder. We train on 3,000 single-cell 3D confocal fluorescence volumes (4 tagged organelle structures) from the Allen Institute's public hiPSC single-cell image dataset, paired with template-generated finding text, with mitotic (dividing) cells oversampled as an abnormal-like proxy class validated by ROC-AUC. Naively reusing work's freeze backbone, LoRA the attention projections recipe for the Voxel Encoder collapses voxel embeddings to near-input-invariance, since LoRA's low-rank delta cannot compensate for an otherwise-frozen, randomly initialized (not pretrained) 3D Swin backbone, even as InfoNCE loss decreases healthily. Switching the Voxel Encoder to full fine-tuning resolves this: 4-way zero-shot structure accuracy rises from 25.0% (chance) to 48.6%, and voxel anomaly-scoring ROC-AUC improves from 0.486 to a peak of 0.628 (still below the 0.70 sanity threshold), while bidirectional retrieval recall stays low (R@10 $\le$3.8%). We further introduce a nearest-neighbor diagnosis-reason retrieval evaluation, reaching 29.8% hit-rate@{1,5} at its best scale while scoring 0.0% elsewhere. The central finding of this MVP is architectural: LoRA's parameter-efficiency assumption presumes a pretrained backbone worth preserving, and breaks down when the backbone is randomly initialized.