Searcharxiv⌕ Search

arXiv · 2611.18765

Mutual Colormap Dependence: How Recoloring Biomedical Images Changes Vision-Language Model Answers

Abstract

Vision-language models (VLMs) are being built to answer questions about biomedical images, yet they never see the measured intensities, only the colors that a colormap has assigned to them. For questions about image content, the colormap should not change the answer. We call the extent to which it does Mutual Colormap Dependence (MCD), the mutual information between the colormap and a model's answer, which is zero exactly when every colormap yields the same answer distribution. Because VLMs sample their answers, we compare recoloring with a control that uses the same number of calls to ask again about the unchanged gray image, so that ordinary resampling is not mistaken for an effect of the colormap. We test GPT-5.4, Gemini 3.1 Pro, Claude Sonnet 4.6, and Qwen3-VL-235B under six luminance-monotonic colormaps (gray, viridis, plasma, inferno, magma, and cividis) on 50 medical visual-question-answering items from 16 SLAKE images, 64 ChestX-ray14 chest radiographs, and 200 BBBC039 fluorescence-microscopy images for exact nucleus counting. Every prompt tells the model to ignore the colormap. Even so, with three answers per colormap on the chest radiographs, the answers of all four models depend on the colormap (all Holm-adjusted $p \leq 0.002$). For Claude, some colored map overturns the majority answer on 29 of the 64 radiographs, where resampling gray alone overturns about 4, and Qwen's accuracy falls from 0.54 under gray to between 0.32 and 0.41 under the colored maps, mostly because it abstains. With one answer per colormap, recoloring adds disagreement beyond repeated sampling in 5 of 12 model-panel cells under per-cell 95% intervals, and repeating the instruction to ignore color does not reliably reduce it. The colormap is therefore part of what a VLM sees, and it can change the answer.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Siddharth Vohra, Runmin Jiang, Min Xu. 2026-11-11. Mutual Colormap Dependence: How Recoloring Biomedical Images Changes Vision-Language Model Answers. https://searcharxiv.com/abs/2611.18765

Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite Searcharxiv by Indiciae: BibTeX · RIS.

KEEP EXPLORING

Related papers

Physics-Aware Zero-Shot Learning for Low-Dose CT from a Single Sinogram

Low-dose computed tomography (CT) imaging remains a persistent challenge, especially under heterogeneous acquisition conditions. Zero-shot learning, which enables models to learn directly from individual images without external supervision, represents a compelling approach to mitigate distribution mismatches arising from variations in scanner hardware, imaging protocols, and patient demographics. However, most existing zero-shot approaches employ down-sampling operations that reduce spatial resolution and cause domain shifts, which become particularly evident when applied to raw sinogram data. Here, we propose a physics-aware zero-shot CT denoising framework to exploit the conjugate symmetry property intrinsic to CT sinograms. Conjugate X-ray pencil beams traverse identical anatomical paths and ideally yield the same expected attenuation values but with independent noise realizations. By randomly mapping sinogram values between conjugate-ray pairs, we synthesize structurally consistent yet noise-diverse sinogram pairs, enabling self-supervised network training without external data. The resulting lightweight denoising network demonstrates strong generalization across low-dose simulation, clinical helical CT, and preclinical photon-counting CT datasets. Quantitative and qualitative results demonstrate competitive denoising performance, with favorable noise suppression and structural preservation across diverse acquisition settings. This work establishes a generalizable and data-efficient paradigm for CT denoising that integrates physical redundancy into learning-based image reconstruction. The code for the proposed method is openly available at: https://github.com/shiyongyi/Physics-Aware-Zero-Shot-CT-Sinogram-Denoising

eess.IV↗

QUTCC: Quantile Uncertainty Training and Conformal Calibration for Imaging Inverse Problems

While deep learning offers tremendous promise for scientific and medical imaging, any failures and hallucinations (predictions that do not coincide with reality) are hard to pinpoint and can have serious downstream consequences. Uncertainty estimation techniques, such as conformal prediction, can help by predicting statistically valid error bars for a model's prediction. However, popular conformal prediction methods were not designed for high-dimensional image-valued problems and do not take into account spatial correlations within an image during conformal calibration, resulting in larger-than-necessary uncertainty intervals. We propose a practical simultaneous quantile regression method that enables non-linear, spatially-adaptive scaling during conformal calibration. Our method, QUTCC uses a U-Net architecture with a quantile embedding to learn a full conditional quantile distribution during training, and then leverages this non-linear, learned function for spatially-adaptive conformal calibration. At test time, our method can efficiently estimate uncertainty intervals with pixel-marginal coverage guarantees. In addition, QUTCC can also predict pixel-wise conditional probability density estimates without any built-in distributional assumptions. We evaluate our method on several denoising problems, accelerated magnetic resonance imaging, and quantitative phase microscopy. Our method consistently produces tighter uncertainty intervals than prior conformal methods at the same coverage level, can predict plausible conditional distributions for different tasks, and in some cases, high-uncertainty regions can help us locate hallucinations in a model's prediction.

eess.IV↗

GroundGazer: An optical reference system for planar localization with millimeter accuracy at low cost

Highly accurate indoor localization systems with absolute mm positioning accuracy are currently expensive. They include laser trackers, total stations, and motion capture systems relying on multiple high-end cameras. In this work, we introduce a high-accuracy, planar indoor localization system named GroundGazer (GG) for autonomous mobile robots (AMRs). GG estimates the AMR's planar position with mm and its heading with sub-degree accuracy. The system requires only a monocular (fisheye) camera, a chessboard floor, and an optional laser diode. Our system is simple and low-cost due to the chessboard floor, robust, scalable to multiple robots, and extendable to 3D position and orientation estimation.

eess.IV↗