SearcharxivSearch

arXiv subjects

Rohit Bhargava

Publications and source records attributed to Rohit Bhargava.

17 recordsLinked to original sources

Seamless Whole Slide Label-Free Virtual Staining

Label-free virtual staining offers a compelling, non-destructive alternative to standard histopathology; however, its clinical adoption is hindered by the computational bottlenecks inherent to processing gigapixel Whole Slide Images (WSIs). Current deep learning approaches require patch-based inference to avoid memory constraints, which disrupts global tissue continuity and introduces tiling artifacts--displaying visible seams and color shifts. To address this, we introduce the Consistency Memory Bank (COMB), a novel label-free virtual staining framework that enforces spatial and channel consistency across tiles without memory bottlenecks. COMB decouples context storage from computation, utilizing a dynamic retrieval mechanism to fetch feature representations from adjacent tiles. This enables a retrieval-based context integration strategy that adopts local padding to resolve spatial discontinuities and neighbor-aware channel attention to stabilize statistical drift. Further optimized with a sliding window schedule to ensure minimal memory overhead, our method demonstrates superior performance over state-of-the-art baselines, achieving significant improvements in both perceptual fidelity and tiling consistency, while suggesting its downstream utility in tumor segmentation. Code is available at https://github.com/dou0000/COMB.

eess.IV

Anatomy of a failure: When, how, and why deep vision fails in scientific domains

Mirroring its ubiquity in popular media and all human activities, the use of deep learning (DL) is rapidly growing in scientific imaging modalities. However, unlike everyday RGB pictures, pixels encode precise physicochemical properties in scientific imaging across potentially thousands of channels. While DL is well validated on human-centric RGB perceptual tasks, its effectiveness for scientific imaging remains uncertain. Here, we show that the naive application of DL frameworks to scientific images can lead to critical failures. We evaluate the use of DL for pathology, comparing RGB images of stained tissue with the quantitative and information-rich biochemical signatures of infrared (IR) imaging. Despite this informational advantage, DL models trained on IR data paradoxically underperform. We investigate this discrepancy to find that IR data priors interact poorly with the simplicity bias of DL, causing models to collapse to one-dimensional predictions. This constitutes a catastrophic DL failure because the model's representational capacity remains largely unused, while furthermore raising AI safety concerns and undermining the advantages of such scientific modalities. Notably, this problem persists even with state-of-the-art DL robustification strategies, which are primarily designed and validated for RGB imagery and thus inherit the same prior-bias mismatch. This work establishes a framework for understanding the limitations of generic DL in science and advocates for the study of modality-specific failure modes to guide the development of specialized, safe AI algorithms.

cs.CV

Needle Biopsy And Fiber-Optic Compatible Robotic Insertion Platform

Tissue biopsy is the gold standard for diagnosing many diseases, involving the extraction of diseased tissue for histopathology analysis by expert pathologists. However, this procedure has two main limitations: 1) Manual sampling through tissue biopsy is prone to inaccuracies; 2) The extraction process is followed by a time-consuming pathology test. To address these limitations, we present a compact, accurate, and maneuverable robotic insertion platform to overcome the limitations in traditional histopathology. Our platform is capable of steering a variety of tools with different sizes, including needle for tissue extraction and optical fibers for vibrational spectroscopy applications. This system facilitates the guidance of end-effector to the tissue and assists surgeons in navigating to the biopsy target area for multi-modal diagnosis. In this paper, we outline the general concept of our device, followed by a detailed description of its mechanical design and control scheme. We conclude with the validation of the system through a series of tests, including positioning accuracy, admittance performance, and tool insertion efficacy.

cs.RO

Finer Disentanglement of Aleatoric Uncertainty Can Accelerate Chemical Histopathology Imaging

Label-free chemical imaging holds significant promise for improving digital pathology workflows, but data acquisition speed remains a limiting factor. To address this gap, we propose an adaptive strategy-initially scan the low information (LI) content of the entire tissue quickly, identify regions with high aleatoric uncertainty (AU), and selectively re-image them at better quality to capture higher information (HI) details. The primary challenge lies in distinguishing between high-AU regions mitigable through HI imaging and those that are not. However, since existing uncertainty frameworks cannot separate such AU subcategories, we propose a fine-grained disentanglement method based on post-hoc latent space analysis to unmix resolvable from irresolvable high-AU regions. We apply our approach to streamline infrared spectroscopic imaging of breast tissues, achieving superior downstream segmentation performance. This marks the first study focused on fine-grained AU disentanglement within dynamic image spaces (LI-to-HI), with novel application to streamline histopathology.

cs.CV

Hallucination Detection in Virtually-Stained Histology: A Latent Space Baseline

Histopathologic analysis of stained tissue remains central to biomedical research and clinical care. Virtual staining (VS) offers a promising alternative, with potential to reduce costs and streamline workflows, yet hallucinations pose serious risks to clinical reliability. Here, we formalize the problem of hallucination detection in VS and propose a scalable post-hoc method: Neural Hallucination Precursor (NHP), which leverages the generator's latent space to preemptively flag hallucinations. Extensive experiments across diverse VS tasks show NHP is both effective and robust. Critically, we also find that models with fewer hallucinations do not necessarily offer better detectability, exposing a gap in current VS evaluation and underscoring the need for hallucination detection benchmarks.

eess.IV

Label-free correlative morpho-chemical tomography of 3D kidney mesangial cells

Label-free characterization of biological specimens seeks to supplement existing imaging techniques and avoid the need for contrast agents that can disturb the native state of living samples. Conventional label-free optical imaging techniques are compatible with living samples but face challenges such as poor sectioning capability, fragmentary morphology, and lack chemical specific information. Here, we combined simultaneous label-free autofluorescence multi-harmonic (SLAM) microscopy and gradient light interference microscopy (GLIM) to extract both chemical specific and morphological tomography of 3D cultured kidney mesangial cells. Imaging 3D in vitro kidney models is essential to understand kidney function and pathology. Our correlative approach enables imaging and quantification of these cells to extract both morphology and chemical-specific signals that is crucial for understanding kidney function. In our approach, SLAM offers a nonlinear imaging platform with a single-excitation source to simultaneously acquire autofluorescence (FAD and NAD(P)H), second, and third harmonic signal from the 3D cultured cells. Complementarily, GLIM acquires high-contrast quantitative phase information to quantify structural changes in samples with thickness of up to 250 micron. Our correlative imaging results demonstrate a versatile and hassle-free platform for morpho-chemical cellular tomography to investigate functions such as metabolism and matrix deposition of kidney mesangial cells in 3D under controlled physiological conditions.

physics.optics

Endoscopic Fourier-transform infrared spectroscopy through a fiber microprobe

Fourier-transform infrared spectroscopy (FTIR) is a powerful analytical method for not only the chemical identification of solid, liquid, and gas species, but also the quantification of their concentration. However, the chemical quantification capability of FTIR is significantly hindered when the analyte is surrounded by a strong IR absorbing medium, such as liquid solutions. To overcome this limit, here we develop an IR fiber microprobe that can be inserted into liquid medium, and obtain full FTIR spectra at points of interest. To benchmark this endoscopic FTIR method, we insert the microprobe into bulk water covering a ZnSe substrate and measure the IR transmittance of water as a function of the probe-substrate distance. The obtained vibrational modes, overall transmittance vs z profiles, quantitative absorption coefficients, and micro z-section IR transmittance spectra are all consistent with the standard IR absorption properties of water. The results pave the way for endoscopic chemical profiling inside bulk liquid solutions, promising for applications in many biological, chemical, and electrochemical systems.

physics.ins-det

Transformer Based Tissue Classification in Robotic Needle Biopsy

Image-guided minimally invasive robotic surgery is commonly employed for tasks such as needle biopsies or localized therapies. However, the nonlinear deformation of various tissue types presents difficulties for surgeons in achieving precise needle tip placement, particularly when relying on low-fidelity biopsy imaging systems. In this paper, we introduce a method to classify needle biopsy interventions and identify tissue types based on a comprehensive needle-tissue contact model that incorporates both position and force parameters. We trained a transformer model using a comprehensive dataset collected from a formerly developed robotics platform, which consists of synthetic and porcine tissue from various locations (liver, kidney, heart, belly, hock) marked with interaction phases (pre-puncture, puncture, post-puncture, neutral). This model achieves a significant classification accuracy of 0.93. Our demonstrated method can assist surgeons in identifying transitions to different tissues, aiding surgeons with tissue awareness.

eess.SP

Are We Ready for Out-of-Distribution Detection in Digital Pathology?

The detection of semantic and covariate out-of-distribution (OOD) examples is a critical yet overlooked challenge in digital pathology (DP). Recently, substantial insight and methods on OOD detection were presented by the ML community, but how do they fare in DP applications? To this end, we establish a benchmark study, our highlights being: 1) the adoption of proper evaluation protocols, 2) the comparison of diverse detectors in both a single and multi-model setting, and 3) the exploration into advanced ML settings like transfer learning (ImageNet vs. DP pre-training) and choice of architecture (CNNs vs. transformers). Through our comprehensive experiments, we contribute new insights and guidelines, paving the way for future research and discussion.

cs.CV

Large fluorescence enhancement via lossless all-dielectric spherical mesocavities

Nano- and microparticles are popular media to enhance optical signals, including fluorescence from a dye proximal to the particle. Here we show that homogeneous, lossless, all-dielectric spheres with diameters in the mesoscale range, between nano- ($\lesssim 100$~nm) and micro- ($\gtrsim 1$ $μ$m) scales, can offer surprisingly large fluorescence enhancements, up to $F\sim 10^4$. With the absence of nonradiative Ohmic losses inherent to plasmonic particles, we show that $F$ can increase, decrease or even stay the same with increasing intrinsic quantum yield $q_0$, for suppressed, enhanced or intact radiative decay rates of a fluorophore, respectively. Further, the fluorophore may be located inside or outside the particle, providing additional flexibility and opportunities to design fit for purpose particles. The presented analysis with simple dielectric spheres should spur further interest in this less-explored scale of particles and experimental investigations to realize their potential for applications in imaging, molecular sensing, light coupling, and quantum information processing.

physics.optics

Deepfake histological images for enhancing digital pathology

An optical microscopic examination of thinly cut stained tissue on glass slides prepared from a FFPE tissue blocks is the gold standard for tissue diagnostics. In addition, the diagnostic abilities and expertise of any pathologist is dependent on their direct experience with common as well as rarer variant morphologies. Recently, deep learning approaches have been used to successfully show a high level of accuracy for such tasks. However, obtaining expert-level annotated images is an expensive and time-consuming task and artificially synthesized histological images can prove greatly beneficial. Here, we present an approach to not only generate histological images that reproduce the diagnostic morphologic features of common disease but also provide a user ability to generate new and rare morphologies. Our approach involves developing a generative adversarial network model that synthesizes pathology images constrained by class labels. We investigated the ability of this framework in synthesizing realistic prostate and colon tissue images and assessed the utility of these images in augmenting diagnostic ability of machine learning methods as well as their usability by a panel of experienced anatomic pathologists. Synthetic data generated by our framework performed similar to real data in training a deep learning model for diagnosis. Pathologists were not able to distinguish between real and synthetic images and showed a similar level of inter-observer agreement for prostate cancer grading. We extended the approach to significantly more complex images from colon biopsies and showed that the complex microenvironment in such tissues can also be reproduced. Finally, we present the ability for a user to generate deepfake histological images via a simple markup of sematic labels.

eess.IV

A generative adversarial approach to facilitate archival-quality histopathologic diagnoses from frozen tissue sections

In clinical diagnostics and research involving histopathology, formalin fixed paraffin embedded (FFPE) tissue is almost universally favored for its superb image quality. However, tissue processing time (more than 24 hours) can slow decision-making. In contrast, fresh frozen (FF) processing (less than 1 hour) can yield rapid information but diagnostic accuracy is suboptimal due to lack of clearing, morphologic deformation and more frequent artifacts. Here, we bridge this gap using artificial intelligence. We synthesize FFPE-like images ,virtual FFPE, from FF images using a generative adversarial network (GAN) from 98 paired kidney samples derived from 40 patients. Five board-certified pathologists evaluated the results in a blinded test. Image quality of the virtual FFPE data was assessed to be high and showed a close resemblance to real FFPE images. Clinical assessments of disease on the virtual FFPE images showed a higher inter-observer agreement compared to FF images. The nearly instantaneously generated virtual FFPE images can not only reduce time to information but can facilitate more precise diagnosis from routine FF images without extraneous costs and effort.

eess.IV

Composition-Aware Spectroscopic Tomography

Chemical imaging provides information about the distribution of chemicals within a target. When combined with structural information about the target, in situ chemical imaging opens the door to applications ranging from tissue classification to industrial process monitoring. The combination of infrared spectroscopy and optical microscopy is a powerful tool for chemical imaging of thin targets. Unfortunately, extending this technique to targets with appreciable depth is prohibitively slow. We combine confocal microscopy and infrared spectroscopy to provide chemical imaging in three spatial dimensions. Interferometric measurements are acquired at a small number of focal depths, and images are formed by solving a regularized inverse scattering problem. A low-dimensional signal model is key to our approach: we assume the target comprises a finite number of distinct chemical species. We establish conditions on the constituent spectra and the number of measurements needed for unique recovery of the target. Simulations illustrate imaging of cellular phantoms and sub-wavelength targets from noisy measurements.

eess.IV

High-resolution label-free imaging of tissue morphology with confocal phase microscopy

Label-free imaging approaches seek to simplify and augment histopathologic assessment by replacing the current practice of staining by dyes to visualize tissue morphology with quantitative optical measurements. Quantitative phase imaging (QPI) operates with visible/UV light and thus provides a resolution matched to current practice. Here we introduce and demonstrate confocal QPI for label-free imaging of tissue sections and assess its utility for manual histopathologic inspection. Imaging cancerous and normal adjacent human breast and prostate, we show that tissue structural organization can be resolved with high spatial detail comparable to conventional H&E stains. Our confocal QPI images are found to be free of halo, solving this common problem in QPI. We further describe and apply a virtual imaging system based on Finite-Difference Time-Domain (FDTD) calculations to quantitatively compare confocal with wide-field QPI methods and explore performance limits using numerical tissue phantoms.

physics.med-ph

A deep learning framework for morphologic detail beyond the diffraction limit in infrared spectroscopic imaging

Infrared (IR) microscopes measure spectral information that quantifies molecular content to assign the identity of biomedical cells but lack the spatial quality of optical microscopy to appreciate morphologic features. Here, we propose a method to utilize the semantic information of cellular identity from IR imaging with the morphologic detail of pathology images in a deep learning-based approach to image super-resolution. Using Generative Adversarial Networks (GANs), we enhance the spatial detail in IR imaging beyond the diffraction limit while retaining their spectral contrast. This technique can be rapidly integrated with modern IR microscopes to provide a framework useful for routine pathology.

eess.IV

Freeform Assembly Planning

3D printing enables the fabrication of complex architectures by automating long sequences of additive steps. The increasing sophistication of printers, materials, and generative design promises to make geometric complexity a non-issue in manufacturing; however, this complexity can only be realized if a design can be translated into a physically executable sequence of printing operations. We investigate this planning problem for freeform direct-write assembly, in which filaments of material are deposited through a nozzle translating along a 3D path to create sparse, frame-like structures. We enumerate the process constraints for different variants of the freeform assembly process and show that, in the case where material stiffens via a glass transition, determining whether a feasible sequence exists is NP-complete. Nonetheless, for topologies typically encountered in real-world applications, finding a feasible or even optimal sequence is a tractable problem. We develop a sequencing algorithm that maximizes the fidelity of the printed part and minimizes the probability of print failure by modeling the assembly as a linear, elastic frame. We implement the algorithm and validate our approach experimentally, printing objects composed of thousands of sugar alcohol filaments with diameters of 100-200 microns. The assembly planner allows the freeform process to be applied to arbitrarily complex parts, from tissue engineering and microfluidics at the micrometer scale, to vascularized functional materials and soft robots at the millimeter scale, to structural components at the meter scale, thus opening a variety of assembly possibilities.

cs.CE

Competition between extinction and enhancement in surface enhanced Raman spectroscopy

Conjugated metallic nanoparticles are a promising means to achieve ultrasensitive and multiplexed sensing in intact three-dimensional samples, especially for biological applications, via surface enhanced Raman scattering (SERS). We show that enhancement and extinction are linked and compete in a collection of metallic nanoparticles. Counterintuitively, the Raman signal vanishes when nanoparticles are excited at their plasmon resonance, while increasing nanoparticle concentrations at off-resonance excitation sometimes leads to decreased signal. We develop an effective medium theory that explains both phenomena. Optimal choices of excitation wavelength, individual particle enhancement factor and concentrations are indicated.

physics.optics