spora: A Unified Multimodal Dataset and Platform for Spatial Proteomics

Benedikt von Querfurth; Johann Wenckstern; Eeshaan Jain; Lukas Klein; Phil Cheng; Yexiang Cheng; Cedric Vincent-Cuaz; Martina Haberecker; Pascal Frossard; Andreas Wicki; Olivier Michielin; Charlotte Bunne

arXiv · 2610.24002

spora: A Unified Multimodal Dataset and Platform for Spatial Proteomics

Benedikt von Querfurth·Johann Wenckstern·Eeshaan Jain·Lukas Klein·Phil Cheng·Yexiang Cheng·Cedric Vincent-Cuaz·Martina Haberecker·Pascal Frossard·Andreas Wicki·Olivier Michielin·Charlotte Bunne

View original source ↗Read source document ↗Export metadata ↓

Abstract

Spatial proteomics (SP) captures the molecular composition and spatial organization of tissues at single-cell resolution, opening a direct view of the cellular ecosystems that drive disease. Protein panels, acquisition platforms, and tissue contexts differ from one cohort to the next, and a coherent picture of tissue biology can only emerge from models trained to bridge this heterogeneity. Foundation models offer the natural path forward, but their training depends on large, harmonized corpora that span platforms, panels, and cohorts. No public resource currently approaches that scale; available data sit in small, narrowly scoped releases with incompatible formats and conventions. We present $\texttt{spora}$, a large-scale multimodal SP dataset of 13,546 standardized multiplex images from 5,477 patients across 35 cohorts and five major SP technologies. Curated with biomedical experts, $\texttt{spora}$ provides model training-ready formats together with cell and tissue segmentations, structured clinical metadata, and, where available, paired H&E enabling the training of virtual staining models. It is complemented by $\texttt{spora [io]}$, a unified interface for tiling, sampling, and standardization across all cohorts and modalities, and $\texttt{spora [bench]}$, a benchmark suite spanning cell phenotyping, diagnostic and therapeutic prediction as well as virtual staining tasks across twelve cohorts that places current SP foundation models on common ground against task specific baselines.

Explore related subjects

cs.CV cs.AI

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Benedikt von Querfurth, Johann Wenckstern, Eeshaan Jain, Lukas Klein, Phil Cheng, Yexiang Cheng, Cedric Vincent-Cuaz, Martina Haberecker, Pascal Frossard, Andreas Wicki, Olivier Michielin, Charlotte Bunne. 2026-10-15. spora: A Unified Multimodal Dataset and Platform for Spatial Proteomics. https://arxiv.org/abs/2610.24002

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

MLLM-Guided Autonomous Object Detection for Open-World Understanding

Object detection typically relies on predefined category vocabularies, while open-world and open-vocabulary approaches remain limited by generic unknown labels or externally specified detection concepts. We propose LAOD, an MLLM-Guided Autonomous Object Detection framework in which a Multimodal Large Language Model (MLLM) generates an image-specific detection vocabulary that is grounded by an open-vocabulary detector. This decomposition separates semantic concept discovery from spatial localization and removes the need to manually enumerate target categories at inference time. We evaluate LAOD using Class-Agnostic Average Precision (CAAP), Semantic Naming Average Precision (SNAP), and Semantic Localization Average Precision (SemLoc-AP), which measure localization, semantic naming, and their correct association, respectively. Experiments on COCO-Val, LVIS-minival, and COCO-OOD across multiple MLLMs and detectors demonstrate strong performance across common, long-tailed, and out-of-distribution objects, while showing that grounding quality remains a major determinant of end-to-end performance. LAOD also substantially outperforms both direct MLLM-based object detection and dedicated models trained specifically for open-ended detection. Overall, the results support a modular formulation of autonomous object detection in which MLLMs determine what to detect and specialized open-vocabulary detectors determine where those concepts occur.

cs.CV↗

arXiv2026

Shape Distribution Matters: Shape-Specific Mixture-of-Experts for Amodal Segmentation Under Diverse Occlusions

Amodal segmentation aims to predict complete object masks covering both visible and occluded regions, serving as a critical foundation for robust multimedia scene understanding and interactive media editing. This task poses significant challenges in complex visual media due to severe occlusions and extreme shape variation, ranging from rigid furniture to highly deformable clothing. Existing one-size-fits-all approaches struggle to reason about diverse amodal shapes in multimedia content due to limited representation capacity. While a mixture-of-experts framework is a natural solution, naively applying it without considering the underlying shape distribution leads to mismatched routing and redundant experts. To address these issues, we introduce ShapeMoE, a shape-specific sparse mixture-of-experts framework designed for complex visual scene parsing. The key idea is to learn a latent shape distribution space and dynamically route each object to a lightweight expert tailored to its geometric characteristics. Specifically, ShapeMoE encodes each object into a compact Gaussian embedding that captures key shape features. A Shape-Aware Sparse Router then maps the object to the most suitable expert, enabling precise shape-aware routing. Each expert is lightweight and specializes in recovering occluded regions for specific shape patterns. ShapeMoE is designed for shape-specific expert specialization while maintaining computational efficiency. Extensive experiments on three challenging benchmarks demonstrate that ShapeMoE consistently outperforms state-of-the-art methods. Beyond advancing fundamental visual reasoning, ShapeMoE provides a robust and efficient visual prior for downstream multimedia applications, particularly in occlusion-heavy immersive environments.

cs.CV↗

arXiv2026

Generative Spatiotemporal Data Augmentation

We explore spatiotemporal data augmentation using video foundation models to diversify both camera viewpoints and scene dynamics. Unlike existing approaches based on simple geometric transforms or appearance perturbations, our method leverages off-the-shelf video diffusion models to generate realistic 3D spatial and temporal variations from a given image dataset. Incorporating these synthesized video clips as supplemental training data yields consistent performance gains in low-data settings, such as UAV-captured imagery where annotations are scarce. Beyond empirical improvements, we provide practical guidelines for (i) choosing an appropriate spatiotemporal generative setup, (ii) transferring annotations to synthetic frames, and (iii) addressing disocclusion - regions newly revealed and unlabeled in generated views. Experiments on COCO subsets and UAV-captured datasets show that, when applied judiciously, spatiotemporal augmentation broadens the data distribution along axes underrepresented by traditional and prior generative methods, offering an effective lever for improving model performance in data-scarce regimes.

cs.CV↗