arXiv subjects

Shaun Porwal

Publications and source records attributed to Shaun Porwal.

3 recordsLinked to original sources

arXiv2026

The Patient is not a Moving Document: A World Model Training Paradigm for Longitudinal EHR

Large language models (LLMs) trained with next-word-prediction have achieved success as clinical foundation models. Representations from these language backbones yield strong linear probe performance across biomedical tasks, suggesting that patient semantics emerge from next-token prediction at scale. However, this paradigm treats patients as a document to be summarized rather than a dynamical system to be simulated; a patient's trajectory emerges from their state evolving under interventions and time, requiring models that simulate dynamics rather than predict tokens. To address this, we introduce SMB-Structure, a world model for structured EHR that grounds a joint-embedding prediction architecture (JEPA) with next-token prediction (SFT). SFT grounds our model to reconstruct future patient states in token space, while JEPA predicts those futures in latent space from the initial patient representation alone, forcing trajectory dynamics to be encoded before the next state is observed. We validate across two large-scale cohorts: Memorial Sloan Kettering (23,319 oncology patients; 323,000+ patient-years) and INSPECT (19,402 pulmonary embolism patients). Using a linear probe evaluated at multiple points along the disease trajectory, we demonstrate that our training paradigm learns embeddings that capture disease dynamics not recoverable by autoregressive baselines, enabling SMB-Structure to achieve competitive performance on complex tasks characterized by high patient heterogeneity. Model weights are available at https://huggingface.co/standardmodelbio/SMB-v1-1.7B-Structure.

cs.AI↗

arXiv2025

GenVarFormer: Predicting gene expression from long-range mutations in cancer

Distinguishing the rare "driver" mutations that fuel cancer progression from the vast background of "passenger" mutations in the non-coding genome is a fundamental challenge in cancer biology. A primary mechanism that non-coding driver mutations contribute to cancer is by affecting gene expression, potentially from millions of nucleotides away. However, existing predictors of gene expression from mutations are unable to simultaneously handle interactions spanning millions of base pairs, the extreme sparsity of somatic mutations, and generalize to unseen genes. To overcome these limitations, we introduce GenVarFormer (GVF), a novel transformer-based architecture designed to learn mutation representations and their impact on gene expression. GVF efficiently predicts the effect of mutations up to 8 million base pairs away from a gene by only considering mutations and their local DNA context, while omitting the vast intermediate sequence. Using data from 864 breast cancer samples from The Cancer Genome Atlas, we demonstrate that GVF predicts gene expression with 26-fold higher correlation across samples than current models. In addition, GVF is the first model of its kind to generalize to unseen genes and samples simultaneously. Finally, we find that GVF patient embeddings are more informative than ground-truth gene expression for predicting overall patient survival in the most prevalent breast cancer subtype, luminal A. GVF embeddings and gene expression yielded concordance indices of $0.706^{\pm0.136}$ and $0.573^{\pm0.234}$, respectively. Our work establishes a new state-of-the-art for modeling the functional impact of non-coding mutations in cancer and provides a powerful new tool for identifying potential driver events and prognostic biomarkers.

q-bio.GN↗

arXiv2025

Patient-specific Biomolecular Instruction Tuning

Proteomics data is essential to pathogenic understanding of a disease phenotype. In cancer, analysis of molecular signatures enables precision medicine through the identification of biological processes that drive individualized tumor progression, therapeutic resistance, and clinical heterogeneity. Recent advances in multimodal large language models (LLMs) have shown remarkable capacity to integrate and reason across heterogeneous data modalities. However, performing multi-modal language modeling for molecular understanding of patient-specific proteomics remains a significant challenge due to two barriers: (1) the lack of instruction-tuning datasets that enable clinical interpretation from proteomics data, and (2) the absence of language modeling architectures designed to capture the rich heterogeneity of molecular data. In this work, we introduce CPTAC-PROTSTRUCT, the first instruction tuning dataset for molecular understanding of oncology, comprising over 400k open-ended examples derived from individualized proteomic profiles curated from the largest national proteomics cancer study (CPTAC). Additionally, we propose KRONOS (Knowledge Representation of patient Omics Networks in Oncology via Structured tuning), a novel graph-LLM framework that leverages molecular interaction topology with proteomics to learn patient-specific graph representations for enhanced clinical reasoning. We show that KRONOS achieves competitive performance across benchmark clinical tasks, including molecular classification, temporal trajectory modeling, and tumor stage prediction from proteomics data. Ultimately, this approach empowers LLMs to understand patient-level pathogenesis, advancing precision medicine through more accurate diagnosis, prognosis, and treatment stratification.

q-bio.QM↗