Searcharxiv⌕ Search

arXiv subjects

Mert R. Sabuncu

Publications and source records attributed to Mert R. Sabuncu.

At least 19 recordsLinked to original sources

Destructive augmentations for prostate lesion segmentation on heterogeneous biparametric MRI data

Training prostate lesion segmentation models on biparametric MRI (bpMRI) is limited by small datasets with heterogeneous imaging acquisition protocols and annotations. In this work, we train a segmentation model on a combined cohort of 1,579 lesion-containing training cases from five datasets, representing, to our knowledge, the largest collection of prostate lesion annotations used for this task. Additionally, we investigate two complementary improvements to the widely used nnU-Net framework. First, we introduce a random wavelet augmentation and show that random sym4 (Symlet) wavelets outperform random convolution augmentation in terms of average Dice score, with only a modest reduction in F1 score due to lower precision. Second, we evaluate instance-level losses and show that Connected-Component-based Dice (CC-Dice)-CE loss complements BackSplit, yielding better performance than BackSplit alone. Together, these improvements achieve the best overall performance, reaching a Dice score of 0.614 on the public Prostate158 test set and establishing a new state-of-the-art on this benchmark. We publicly release our training code and model weights.

eess.IV↗

Improving PI-RADS Prediction and Prostate Cancer Detection from Multiparametric MRI with Anatomical Guidance

Accurate prediction of Prostate Imaging Reporting and Data System (PI-RADS) scores from multiparametric MRI (mpMRI) remains challenging, more so than clinically significant prostate cancer (csPCa) detection. Given the importance of lesion location in PI-RADS scoring, we evaluated whether incorporating prostate zonal masks improves PI-RADS prediction and csPCa detection across multiple architectures (3D ResNet-18, ResNet-50, and ViT) using a large-scale internal dataset. Anatomical guidance generally improved performance across architectures, with the best results achieved by ResNet-18 with 2-zone guidance for PI-RADS prediction (QWK = 0.511) and ResNet-50 with 5-zone guidance for csPCa detection (AUC = 0.809). Fine-tuned results on external datasets further demonstrated the benefits of anatomical guidance. Furthermore, models with anatomical guidance improved the utility of model-predicted PI-RADS scores for PI-RADS-based csPCa detection (positive defined as PI-RADS>=3). Combining model-predicted and expert-assigned PI-RADS scores significantly increased the AUC from 0.711 to 0.756 (p < 0.05), suggesting that AI-derived PI-RADS scores may provide complementary information to expert assessment for csPCa detection. Overall, our findings highlight the value of incorporating prostate zonal anatomy into deep learning-based prostate MRI assessment.

cs.CV↗

NeuroVolve: Evolving personalized visual stimuli toward user-defined brain response targets

A central question in neuroscience is what visual features drive brain responses across regions. Classic approaches rely on handcrafted stimuli to test specific hypotheses, limiting discovery to features defined before experiments. Here we introduce NeuroVolve, a data-driven generative framework for synthesizing images that achieve flexible, user-defined patterns of brain activation in individuals. NeuroVolve recovers established selectivity across visual cortical regions and, paired with subject-specific brain encoding models, generates personalized stimuli reflecting individual differences in neural preference. Beyond single-region selectivity, NeuroVolve uncovers multi-region response selectivity, characterizing how regions jointly represent visual content during naturalistic perception. We further demonstrate generalization beyond visual cortex: in a prospective fMRI experiment, NeuroVolve-generated stimuli targeting the amygdala, a classically non-sensory region, drove measured activation consistent with predictions in novel participants. Together, these results establish NeuroVolve as a less constrained, data-driven tool for probing individual-level neural selectivity across the brain.

cs.CV↗

Interpretable Multiple-Instance Learning of Glomerular Pathomics and Subtype Ratios for eGFR Classification

Chronic kidney disease is characterized by structural alterations in renal tissue and progressive loss of kidney function. While glomerular subtype ratios provide a compact and clinically meaningful summary of glomerular composition, representing pathomic information using whole-slide image (WSI)-level mean values diminishes glomerulus-level heterogeneity and the contribution of less prevalent patterns. In this proof-of-concept study, we investigated whether modeling individual glomeruli using multiple-instance learning (MIL) could provide complementary information to conventional subtype ratios for estimated glomerular filtration rate (eGFR) classification. We analyzed 162 kidney-biopsy WSIs from 162 patients, categorizing renal function as eGFR < 60 or >= 60 mL/min/1.73 m^2. Quantitative size, shape, intensity, and texture features were extracted from segmented glomeruli, representing each patient as a bag of glomerular instances. An attention-based MIL model was trained to aggregate these heterogeneous pathomic profiles. We evaluated subtype ratios, pathomics, and their combination using ABMIL, with a mean-pooled multilayer perceptron baseline. In five-fold cross-validation, the subtype-ratio-only model achieved an AUC of 0.7262 +/- 0.0596. MIL using pathomics alone achieved an AUC of 0.7362 +/- 0.0788, whereas integrating pathomics with subtype ratios produced the highest mean AUC of 0.7630 +/- 0.0665. SHapley Additive exPlanations (SHAP) were used to examine individual feature contributions. These results demonstrate that MIL-based pathomics can complement established glomerular subtype ratios by capturing additional glomerulus-level information for quantitative kidney-biopsy analysis.

eess.IV↗

Toward Contrast-Free Breast MRI: Improving Cancer Detection With Virtual Contrast Enhanced Imaging

Although contrast-enhanced (CE) breast MRI is an important tool for cancer detection, intravenous gadolinium administration introduces additional considerations for patients and clinical workflows. AI based virtual contrast enhanced (VCE) MRI could help identify patients for whom acquired CE imaging may be avoidable without compromising diagnostic performance. We developed a framework that combines non contrast (NC) and VCE MRI predictions for cancer detection. On the held-out test set, cancer detection using NC MRI alone achieved an AUROC of 0.865. Combining the predictions from NC and VCE MRI improved AUROC to 0.901. Building on this complementary predictive value, we evaluated patient-level routing to progressively replace acquired CE examinations with the NC plus VCE pathway, prioritizing patients with the most confident NC predictions. This simulation suggests that diagnostic performance can be largely preserved while limiting acquired CE imaging to approximately 40% of patients. Together, these findings demonstrate the complementary value of VCE MRI and suggest its potential to support selective contrast use in breast cancer screening.

eess.IV↗

From Classification to Cross-Modal Understanding: Leveraging Vision-Language Models for Fine-Grained Renal Pathology

Fine-grained glomerular subtyping is essential for kidney biopsy interpretation, yet it presents a harder setting than the broad organ- or disease-level recognition tasks commonly used in vision-language model (VLM) evaluation. In this task, all categories arise within the same renal microanatomic unit, and clinically meaningful subtypes often differ by subtle, overlapping, and continuum-like morphological cues. Clinically valuable labels are also scarce, making fully supervised image-only paradigms difficult to translate to practice. Although VLMs are promising for data-efficient pathology learning, it remains unclear how their cross-modal representations should be adapted and evaluated for such within-structure discrimination under realistic few-shot constraints. To our knowledge, this study establishes the first fine-grained renal pathology benchmark for few-shot VLM adaptation and systematically examines how supervision level, model design, and adaptation strategy shape both classification performance and multimodal representation structure. Beyond accuracy, AUC, and F1, we assess image-text alignment, positive-negative similarity separation, embedding geometry, and calibration (ECE), asking when performance gains reflect more reliable cross-modal understanding. Across renal pathology cohorts, additional supervision consistently improves discrimination, but reliable adaptation is not explained by alignment alone; stronger performance is associated with larger class separation, more coherent embedding organization, and better awareness of confidence-related failure modes under domain variation. Within this retrospective two-cohort evaluation, these findings move assessment beyond coarse classification and accuracy alone and provide comparative evidence to inform model selection and future translational validation. Code: https://github.com/ddrrnn123/glo-vlms

cs.CV↗

PathLang: A Language-Centered Benchmark for Vision-Language Models in Computational Pathology

Pathology vision-language models (VLMs) have shown strong visual perception ability, but their robustness in the language domain remains poorly characterized. Existing pathology VLM benchmarks largely rely on canonical closed-set prompts or perturb only generic templates, treating language as a fixed evaluation component rather than a variable axis of model behavior. In clinical practice, however, diagnostic language varies across reports, institutions, and candidate diagnoses. We introduce PathLang, a language-centered and clinically grounded zero-shot benchmark. PathLang holds the underlying slides, ground-truth labels, and image-text evaluation direction fixed while systematically varying only the diagnostic language, so that performance differences reflect how a diagnosis is phrased rather than what is imaged. The language variation follows how pathologists actually rephrase diagnoses (terminology, specificity, and reporting style), and all prompts and candidate pools are validated by six board-certified pathologists. PathLang covers four task families: (1) zero-shot classification with image-text alignment analysis, (2) cross-modal retrieval, (3) paraphrase robustness, including semantic-equivalence paraphrases, length and reporting-style variation, and prompt ensembling, and (4) open-vocabulary diagnosis retrieval over four candidate pools with distinct forms of semantic competition. Across nine VLMs and five public datasets spanning four organs, we find that performance is highly sensitive to clinically equivalent paraphrases, varies substantially across forms of semantic competition, and that image-text alignment quality does not necessarily translate into inter-class separability. We release the prompt corpus, candidate pools, pre-computed text embeddings, and evaluation code at https://anonymous.4open.science/r/PathLang.

cs.CV↗

MAdam: Metric-Aware Multi-Objective Adam

Multi-objective optimization (MOO) underlies many machine learning problems, yet MOO solvers across the loss-balancing, gradient-balancing, and Pareto-based families almost universally hand their reconciled directions to Adam~\citep{kingma2015adam}. We show this coupling introduces two systematic gaps between the solver's intent and the optimizer's execution. The first is a weighting mismatch: Adam's second-moment denominator entangles the time-varying preference vector with gradient statistics, marginalizing the preference into a history average and collapsing distinct Pareto trade-offs toward a near-uniform mixture. The second is a geometric mismatch: Adam's adaptive metric distorts the Euclidean geometry MOO solvers assume, turning aligned objectives into apparent conflicts. To resolve both jointly, we introduce MAdam (Metric-Aware Multi-Objective Adam), a drop-in wrapper that leaves both solver and optimizer unchanged. MAdam preconditions the reconciled direction by the preference-conditioned curvature of the scalarized objective; on this whitened input, Adam's second moment collapses to identity, so the realized update is governed by the preference-conditioned metric. Across multi-task learning, Pareto-front recovery, physics-informed neural networks, and medical imaging, MAdam improves over Adam in aggregate for every solver family. Our code is available at https://github.com/lfb-1/MAdam.

cs.LG↗

Seeing Abnormal from Normal: Glomerular Abnormality in Representations of Normal Renal Morphology

Fine-grained evaluation of glomerular pathology must distinguish normal glomeruli from abnormalities such as global and segmental glomerulosclerosis, obsolescent, ischemic, solidified, disappearing, and atubular glomeruli. Supervised classification requires labeled examples of every category, which is impractical when subtypes are rare or absent from the training cohort. One-class anomaly detection offers an alternative by modeling normal data and scoring deviations, allowing previously unseen abnormalities to be detected. We use the frozen residual U-Net backbone of Omni-Seg, pretrained to segment structurally normal renal primitives without abnormal-subtype labels. We propose NoRDeC (Normal-Reference Detection and Characterization), a framework combining Mahalanobis normal-reference scoring with layer-wise representation analysis to determine whether and where glomerular pathology is encoded, how spatial aggregation affects detection, and whether abnormalities alter inter-layer relationships differently. Using glomerular images from two institutions, we evaluate backbone layers and aggregation strategies, compare NoRDeC with PaDiM and PatchCore, and analyze representations using centered kernel alignment (CKA). Layer 4 with Center-70 aggregation achieved a pooled AUROC of $0.926\pm0.013$. NoRDeC achieved the highest AUROC in six of seven abnormality categories and in the pooled analysis, while CKA suggested subtype-dependent changes in inter-layer relationships not captured by anomaly scores alone. The normal-reference model is fitted using only normal glomeruli; abnormality labels are used for configuration selection, evaluation, and grouping in the representation analysis. These results show that a frozen renal feature extractor can support both detection and representation-level characterization of glomerular abnormalities without using abnormal examples to fit the detector.

cs.CV↗

Benchmarking Active Spot Selection for Cost-Efficient Spatial Transcriptomics

Spatial transcriptomics (ST) measures gene expression in tissue context, but dense capture grids can be costly and may repeatedly sample morphologically similar regions. Most active learning strategies were developed for categorical labels and independent samples. We conduct a retrospective pool-based benchmark of active learning versus uniform Random sampling for ST, where expression vectors are high-dimensional and continuous and candidates are spatially correlated. Using two fully profiled public ST cohorts, we mask candidate expression vectors and simulate multi-round selection with uncertainty-based Monte Carlo dropout (MC-dropout) and temporal output discrepancy (TOD), and diversity-based CoreSet and TypiClust-inspired selection. We compare 160 completed configurations at 5%, 10%, 30%, and 50% of the fold-wide training spot pool under patient-level cross-validation, with a separate full-label reference. Within each budget, strategies share the selection schedule, morphology-to-expression predictor, and optimization protocol. We assess mean per-gene within-slide Pearson correlation coefficient (PCC), expression-cluster agreement, and Moran's I fidelity. On HER2-positive breast cancer, pooled mean PCC differences from Random across the four active strategies were -0.0176, -0.0117, +0.0056, and +0.0057 at 5%, 10%, 30%, and 50%, respectively. On cutaneous squamous cell carcinoma (cSCC), three strategies were below Random at 5%, and all four were below Random at 10%. On HER2-positive breast cancer, CoreSet and MC-dropout had lower PCC but higher expression-cluster agreement than Random at the two smallest budgets; this pattern did not reproduce on cSCC. Under the reported fixed training horizons, the evaluated active strategies do not consistently improve on Random at small budgets, and rankings depend on the evaluation measure.

cs.CV↗

StainBridge: Stain-Aware Pairwise Registration of Serial Renal Biopsy Whole-Slide Images Across Structural and Immunohistochemical Stains

Three-dimensional (3D) reconstruction of histopathology tissue requires accurate pairwise registration of serial whole-slide images (WSIs). Cross-stain benchmarks have advanced registration of differently stained histology, including structural-to-immunohistochemistry (IHC) pairs, but serial renal biopsy stacks remain difficult: they interleave several structural stains with diverse IHC markers whose expression can be sparse or absent, leaving few shared features to match. We present StainBridge, a stain-aware framework for registering serial renal biopsy WSIs across structural and IHC stains. StainBridge couples three preprocessing components, stain deconvolution, intensity normalization, and tissue-mask injection, with XFeat-based affine initialization and four nonrigid backends (VoxelMorph, ConvexAdam, FireANTs, and DeeperHistReg). We evaluate it on 23 cases comprising 338 WSIs, four structural stains, and ten IHC markers, with functional tissue units annotated on consecutive sections to give 1,468 landmark correspondences across 272 image pairs, and report tissue-mask Dice, functional-unit centroid error in micrometers, and tissue-restricted structural similarity. Nonrigid refinement improves on the affine initialization for three of four backends, VoxelMorph being the exception. DeeperHistReg, which computes its own initialization rather than relying on XFeat, gives the best pooled landmark accuracy and registers the most pairs, including every attempted structural-IHC pair. Preprocessing improves landmark accuracy for ConvexAdam and FireANTs in every stain-pairing category, and FireANTs shows both the largest single preprocessing gain on structural-IHC pairs and the best pooled tissue overlap. These results offer practical guidance for cross-stain registration and a foundation for integrated 3D analysis of renal tissue architecture and molecular expression.

eess.IV↗

ShapKO: Shapley-Adaptive Modality Knockout for Robust Multimodal Learning

Multimodal medical models often degrade when inputs are missing, a common scenario in real-world clinical workflows. Separately, even when all modalities are present, modality dominance is observed during training, where optimization over-relies on a highly predictive modality and undertrains complementary sources, resulting in poor robustness under partial availability. While training-time modality knockout improves missing-modality robustness, existing approaches use static masking rates that cannot adapt to evolving modality utility during training. We introduce ShapKO (Shapley-Adaptive Modality Knockout), a dynamic training strategy that learns modality-specific knockout probabilities based on validation utility. ShapKO periodically evaluates performance across modality subsets, estimates modality importance via Shapley values, and updates masking probabilities to suppress dominant modalities more frequently. This adaptive process promotes complementary representations, while requiring no architectural modifications. We evaluate ShapKO on three datasets covering multitask clinical classification, survival prediction, and cancer detection. ShapKO consistently improves performance under modality absence and yields interpretable trajectories of learned masking behavior. Code is available at: https://github.com/sumona00/ShapKO

cs.CV↗

BackSplit: The Importance of Sub-dividing the Background in Biomedical Lesion Segmentation

Segmenting small lesions in medical images remains notoriously difficult. Most prior work tackles this challenge by either designing better architectures, loss functions, or data augmentation schemes; and collecting more labeled data. We take a different view, arguing that part of the problem lies in how the background is modeled. Common lesion segmentation collapses all non-lesion pixels into a single "background" class, ignoring the rich anatomical context in which lesions appear. In reality, the background is highly heterogeneous-composed of tissues, organs, and other structures that can now be labeled manually or inferred automatically using existing segmentation models. In this paper, we argue that training with fine-grained labels that sub-divide the background class, which we call BackSplit, is a simple yet powerful paradigm that can offer a significant performance boost without increasing inference costs. From an information theoretic standpoint, we prove that BackSplit increases the expected Fisher Information relative to conventional binary training, leading to tighter asymptotic bounds and more stable optimization. With extensive experiments across multiple datasets and architectures, we empirically show that BackSplit consistently boosts small-lesion segmentation performance, even when auxiliary labels are generated automatically using pretrained segmentation models. Additionally, we demonstrate that auxiliary labels derived from interactive segmentation frameworks exhibit the same beneficial effect, demonstrating its robustness, simplicity, and broad applicability.

cs.CV↗

MORI-Seg: Learning Morphological Geometry for Instance Segmentation without Instance Annotations

Instance-level quantification of kidney functional units is essential for morphometric analysis, yet most publicly available pathology datasets provide only semantic segmentation annotations, where adjacent structures of the same class are merged into single regions. This prevents reliable instance-level analysis and limits downstream quantitative studies. Existing heuristic post-processing methods often yield suboptimal instance separation, particularly in crowded and adherent regions, while deep learning-based instance segmentation approaches typically require intensive instance-level annotations that are costly and labor-intensive to obtain. We propose MORI-Seg, a deep learning framework that enables instance segmentation without requiring instance-level annotations. Instead of heuristic splitting or instance supervision, MORI-Seg learns morphology-aware geometric representations directly from semantic masks by jointly modeling object-centric distance fields and boundary-band representations to encode interior structure and contact interfaces. A class-conditioned feature disentanglement module further promotes intra-instance coherence and inter-instance separation. Under semantic-only supervision, MORI-Seg decomposes connected semantic regions into distinct instance masks in an end-to-end manner. Experiments demonstrate improved instance separation accuracy and more reliable morphometric quantification compared with classical post-processing pipelines and representative semantic-to-instance learning approaches. The official implementation is publicly available at https://github.com/ddrrnn123/MORI-Seg.

cs.CV↗

Network-Aware Bilinear Tokenization for Brain Functional Connectivity Representation Learning

Masked autoencoders (MAEs) have recently shown promise for self-supervised representation learning of resting-state brain functional connectivity (FC). However, a fundamental question remains unresolved: how should FC matrices be tokenized to align with the intrinsic modular organization of large-scale brain networks? Existing approaches typically adopt region-centric or graph-based schemes that treat FC as structurally homogeneous elements and overlook the large-scale network brain organization. We introduce NERVE (Network-Aware Representations of Brain Functional Connectivity via Bilinear Tokenization), a self-supervised learning framework that redefines FC tokenization by partitioning FC matrices into patches of intra- and inter-network connectivity blocks. Unlike image-based MAE, where fixed-size patches share a common tokenizer, FC patches defined by network pairs are heterogeneous in size and correspond to distinct functional roles. To resolve this problem, NERVE embeds FC patches through a novel structured bilinear factorization. This formulation preserves network identity and reduces parameter complexity from quadratic to linear scaling in the number of networks. We evaluate NERVE across three large-scale developmental cohorts (ABCD, PNC, and CCNP) for behavior and psychopathology prediction. Compared to structurally agnostic MAE variants and graph-based self-supervised baselines, the proposed network-aware formulation yields more stable and transferable representations, particularly in cross-cohort evaluation. Ablation studies confirm that the proposed bilinear network embedding and anatomically grounded parcellation are critical for performance. These findings highlight the importance of incorporating domain-specific structural priors into self-supervised learning for functional connectomics. Code is available at: https://github.com/leomlck/NERVE.

cs.AI↗

Prediction of Rectal Cancer Regrowth from Longitudinal Endoscopy

Clinical trial studies indicate benefit of watch-and-wait (WW) surveillance for patients with rectal cancer showing a complete or near clinical response (CR) directly after treatment (restaging). However, there are no objectively accurate methods to early detect local tumor regrowth (LR) in patients undergoing WW from follow-up exams. Hence, we developed Temporal Rectal Endoscopy Cross-attention (TREX), a longitudinal deep learning approach that combines pairs of images acquired at restaging and follow-up to distinguish CR from LR. TREX uses pretrained Swin Transformers in a siamese setting to extract features from longitudinal images and dual cross-attention to combine the features without spatial co-registration between image pairs. TREX and Swin-based baselines were trained under two settings: (a) detecting LR or CR at the last available follow-up and (b) early detection of LR at 3--6, 6--12, and 12--24 months before clinical confirmation. TREX achieved the highest accuracy in detecting LR with a high sensitivity of 97% $\pm$ 6% and a balanced accuracy of 90% $\pm$ 3%, and outperformed all baselines in early detection at both 3--6 (74% $\pm$ 1%) and 6--12 months (62% $\pm$ 4%) prior to clinical detection. Clinical validation via a surgeon survey showed that TREX matched attending-level overall accuracy (TREX: 86.21% vs.\ Clinicians: 87.84% $\pm$ 1.28%). Finally, we explored TREX's ability to predict treatment response by combining pre-treatment (pre-TNT) and restaging endoscopies, achieving a balanced accuracy of 73% $\pm$ 12%. These results show that longitudinal deep learning analysis of endoscopy may improve surveillance and enable earlier identification of rectal cancer regrowth.

cs.CV↗

Spectral Vision Transformer for Efficient Tokenization with Limited Data

We propose a novel spectral vision transformer architecture for efficient tokenization in limited data, with an emphasis on medical imaging. We outline convenient theoretical properties arising from the choice of basis including spatial invariance and optimal signal-to-noise ratio. We show reduced complexity arising from the spectral projection compared to spatial vision transformers. We show equitable or superior performance with a reduced number of parameters as compared to a variety of models including compact and standard vision transformers, convolutional neural networks with attention, shifted window transformers, multi-layer perceptrons, and logistic regression. We include simulated, public, and clinical data in our analysis and release our code at: \verb+github.com/agr78/spectralViT+.

cs.CV↗

AI-Based Detection of Temporal Changes in MR-Linac Images Acquired During Routine Prostate Radiotherapy

Purpose: To investigate whether an AI-based method can detect subtle inter-fraction changes in MR-Linac images acquired during radiotherapy and explore the broader potential of MRLinac imaging. Methods: This retrospective study included longitudinal 0.35T MR-Linac images from 761 patients. To identify temporal changes, we employed a deep learning model using temporal ordering via pairwise comparison, previously shown effective for longitudinal imaging studies. The model was trained using first-to-last fraction pairs (F1-FL) and all pairs (All-pairs). Performance was assessed using quantitative metrics (accuracy and AUC) and compared against a radiologist's performance. Qualitative evaluation was performed using saliency maps, which identify anatomical regions associated with temporal imaging changes. Results: The F1-FL model demonstrated high performance (AUC=0.99, accuracy=0.95) and outperformed the radiologist in temporal ordering task. The All-pairs model also showed high performance (AUC=0.97, accuracy=0.91). Regions contributing to predictions included the prostate, bladder, and pubic symphysis. The performance was correlated to fractional intervals and was reduced for non-radiation-exposed timepoints (Sim and F1), suggesting that observed changes may reflect both temporal variation and radiation exposure. Conclusion: MR-Linac imaging appears capable of capturing subtle changes during prostate radiotherapy that can be detected by AI models, even over approximately two-day intervals. The model's high performance, together with quantitative and qualitative analyses, supports a potential role for MR-Linac in clinical applications beyond image guidance.

eess.IV↗