arXiv subjects

Katrin Amunts

Publications and source records attributed to Katrin Amunts.

At least 19 recordsLinked to original sources

arXiv2026

3D scattered light imaging: extracting 3D fiber orientations from 1D line profiles in brain imaging

Understanding the 3D fiber architecture of the brain at the microscopic scale is essential for revealing its structural connectivity and function. Polarization-based optical imaging (3D-PLI) techniques enable high-fidelity reconstruction of single nerve fiber orientations but struggle to resolve fiber crossings, which are critical for recovering the full connectome. Scattering-based imaging provides access to the structure factor of three-dimensionally oriented fibers. By probing a fixed scattering angle under multiple azimuthal illumination angles, in-plane fiber orientations and crossings can be recovered using computational scattered light imaging (SLI). However, despite containing 3D information, a theoretical framework to extract full 3D orientations has been lacking. In this talk, a simple analogical approximation of Rayleigh-Gans scattering theory is introduced to extract the 3D orientation of nerve fibers from one-dimensional SLI measurements. The theory is validated using tilted microscopic glass phantoms consisting of 2 um-thick rod lattices fabricated by two-photon lithography. Finally, the method is applied to brain tissue samples and compared with 3D-PLI.

physics.optics↗

arXiv2026

Cytoarchitecture in Words: Weakly Supervised Vision-Language Modeling for Human Brain Microscopy

Vision foundation models increasingly support interactive scientific workflows, but natural-language interaction requires coupling visual representations to language. Curated image-text pairs for this coupling are scarce in many biomedical domains. This is the case for cell-body-stained histological sections of the human brain, where microscopic image patches encode cytoarchitecture: cellular density, morphology, laminar structure, and areal organization. We propose retrieve-and-enrich supervision, a weakly supervised scheme for training image-conditioned language models without curated image-text pairs. The method retrieves label-level text from the literature via a shared anatomical label, then enriches it with image-specific properties such as cortical layer thickness and cell density. Labels provide training targets only, not model inputs. We use this scheme to couple CytoNet, a cytoarchitectonic vision foundation model, to an open-weight large language model via a lightweight Flamingo-style adapter. Across 57 brain areas, the resulting model produces plausible cytoarchitectonic descriptions, supports open-set use by rejecting out-of-scope areas, and predicts the correct area for in-scope patches with 90.6% accuracy. Removing explicit area names from generated text still leaves descriptions sufficient for a language model to recover the area in an 8-way test with 68.6% accuracy. A second instruction-tuned model trained with image-specific targets recovers cortical layer thickness and cell-densities from individual patches, allowing the model to read out information beyond canonical area descriptions. Our results show that retrieve-and-enrich supervision offers a practical route to vision-language training in specialized imaging domains where labels and expert text exist but image-level captions do not.

cs.CV↗

arXiv2025

CytoNet: A Foundation Model for the Human Cerebral Cortex at Cellular Resolution

Studying the cellular architecture of the human cerebral cortex is essential for understanding how the brain is organized from the micro to the macro level, and how it functions. However, investigating complex texture patterns in histological images using automatic methods that can be scaled across whole brains remains a challenge. Here we introduce CytoNet, a foundation model trained on 1 million unlabeled microscopic image patches from over 4,000 histological sections from nine postmortem brains, and evaluated on over 2,000 sections from five additional brains excluded from self-supervised pretraining. Using co-localization in the cortical sheet for self-supervision, CytoNet learns to encode complex cellular patterns into expressive and anatomically meaningful feature representations. CytoNet supports multiple downstream applications, including area classification, laminar segmentation, quantification of microarchitectural variation, and exploratory mapping of cortical subdivisions. Functional parcellation analyses provided parcellation-dependent evidence for links between cytoarchitecture and macroscale functional organization. Together, these results establish CytoNet as a unified framework for scalable analysis of cortical microarchitecture and for testing links between cellular architecture and structure-function organization in the human cerebral cortex.

q-bio.NC↗

arXiv2025

From Fibers to Cells: Fourier-Based Registration Enables Virtual Cresyl Violet Staining From 3D Polarized Light Imaging

Comprehensive assessment of the various aspects of the brain's microstructure requires the use of complementary imaging techniques. This includes measuring the spatial distribution of cell bodies (cytoarchitecture) and nerve fibers (myeloarchitecture). The gold standard for cytoarchitectonic analysis is light microscopic imaging of cell-body stained tissue sections. To reveal the 3D orientations of nerve fibers, 3D Polarized Light Imaging (3D-PLI) has been introduced, a method that is label-free and allows subsequent staining of sections after 3D-PLI measurement. By post-staining for cell bodies, a direct link between fiber- and cytoarchitecture can potentially be established in the same section. However, inevitable distortions introduced during the staining process make a costly nonlinear and cross-modal registration necessary in order to study the detailed relationships between cells and fibers in the images. In addition, the complexity of processing histological sections for post-staining only allows for a limited number of such samples. In this work, we take advantage of deep learning methods for image-to-image translation to generate a virtual staining of 3D-PLI that is spatially aligned at the cellular level. We use a supervised setting, building on a unique dataset of brain sections, to which Cresyl violet staining has been applied after 3D-PLI measurement. To ensure high correspondence between both modalities, we address the misalignment of training data using Fourier-based registration. In this way, registration can be efficiently calculated during training for local image patches of target and predicted staining. We demonstrate that the proposed method can predict a Cresyl violet staining from 3D-PLI, resulting in a virtual staining that exhibits plausible patterns of cell organization in gray matter, with larger cell bodies being localized at their expected positions.

eess.IV↗

arXiv2024

Scattering polarimetry enables correlative nerve fiber imaging and multimodal analysis

Mapping the intricate network of nerve fibers is crucial for understanding brain function. Three-Dimensional Polarized Light Imaging (3D-PLI) and Computational Scattered Light Imaging (ComSLI) map dense nerve fibers in brain sections with micrometer resolution using visible light. 3D-PLI reconstructs 3D-fiber orientations, while ComSLI disentangles multiple directions per pixel. So far, these imaging techniques have been realized in separate setups. A combination within a single device would facilitate faster measurements, pixelwise mapping, cross-validation of fiber orientations, and leverage the advantages of each technique while mitigating their limitations. Here, we introduce the Scattering Polarimeter, a microscope that facilitates correlative large-area scans by integrating 3D-PLI and ComSLI measurements into a single system. Based on a Mueller polarimeter, it incorporates variable retarders and a large-area light source for direct and oblique illumination, enabling combined 3D-PLI and ComSLI measurements. Applied to human and vervet monkey brain sections, the Scattering Polarimeter generates results comparable to state-of-the-art 3D-PLI and ComSLI setups and creates a multimodal fiber direction map, integrating the robust fiber orientations obtained from 3D-PLI with fiber crossings from ComSLI. Furthermore, we discuss applications of the Scattering Polarimeter for unprecedented correlative and multimodal brain imaging.

physics.med-ph↗

arXiv2024

Analyzing Regional Organization of the Human Hippocampus in 3D-PLI Using Contrastive Learning and Geometric Unfolding

Understanding the cortical organization of the human brain requires interpretable descriptors for distinct structural and functional imaging data. 3D polarized light imaging (3D-PLI) is an imaging modality for visualizing fiber architecture in postmortem brains with high resolution that also captures the presence of cell bodies, for example, to identify hippocampal subfields. The rich texture in 3D-PLI images, however, makes this modality particularly difficult to analyze and best practices for characterizing architectonic patterns still need to be established. In this work, we demonstrate a novel method to analyze the regional organization of the human hippocampus in 3D-PLI by combining recent advances in unfolding methods with deep texture features obtained using a self-supervised contrastive learning approach. We identify clusters in the representations that correspond well with classical descriptions of hippocampal subfields, lending validity to the developed methodology.

cs.CV↗

arXiv2024

Self-Supervised Representation Learning for Nerve Fiber Distribution Patterns in 3D-PLI

A comprehensive understanding of the organizational principles in the human brain requires, among other factors, well-quantifiable descriptors of nerve fiber architecture. Three-dimensional polarized light imaging (3D-PLI) is a microscopic imaging technique that enables insights into the fine-grained organization of myelinated nerve fibers with high resolution. Descriptors characterizing the fiber architecture observed in 3D-PLI would enable downstream analysis tasks such as multimodal correlation studies, clustering, and mapping. However, best practices for observer-independent characterization of fiber architecture in 3D-PLI are not yet available. To this end, we propose the application of a fully data-driven approach to characterize nerve fiber architecture in 3D-PLI images using self-supervised representation learning. We introduce a 3D-Context Contrastive Learning (CL-3D) objective that utilizes the spatial neighborhood of texture examples across histological brain sections of a 3D reconstructed volume to sample positive pairs for contrastive learning. We combine this sampling strategy with specifically designed image augmentations to gain robustness to typical variations in 3D-PLI parameter maps. The approach is demonstrated for the 3D reconstructed occipital lobe of a vervet monkey brain. We show that extracted features are highly sensitive to different configurations of nerve fibers, yet robust to variations between consecutive brain sections arising from histological processing. We demonstrate their practical applicability for retrieving clusters of homogeneous fiber architecture, performing classification with minimal annotations, and query-based retrieval of characteristic components of fiber architecture such as U-fibers.

cs.CV↗

arXiv2023

Denoising Diffusion Probabilistic Models for Image Inpainting of Cell Distributions in the Human Brain

Recent advances in imaging and high-performance computing have made it possible to image the entire human brain at the cellular level. This is the basis to study the multi-scale architecture of the brain regarding its subdivision into brain areas and nuclei, cortical layers, columns, and cell clusters down to single cell morphology Methods for brain mapping and cell segmentation exploit such images to enable rapid and automated analysis of cytoarchitecture and cell distribution in complete series of histological sections. However, the presence of inevitable processing artifacts in the image data caused by missing sections, tears in the tissue, or staining variations remains the primary reason for gaps in the resulting image data. To this end we aim to provide a model that can fill in missing information in a reliable way, following the true cell distribution at different scales. Inspired by the recent success in image generation, we propose a denoising diffusion probabilistic model (DDPM), trained on light-microscopic scans of cell-body stained sections. We extend this model with the RePaint method to impute missing or replace corrupted image data. We show that our trained DDPM is able to generate highly realistic image information for this purpose, generating plausible cell statistics and cytoarchitectonic patterns. We validate its outputs using two established downstream task models trained on the same data.

eess.IV↗

arXiv2022

GORDA: Graph-based ORientation Distribution Analysis of SLI scatterometry Patterns of Nerve Fibres

Scattered Light Imaging (SLI) is a novel approach for microscopically revealing the fibre architecture of unstained brain sections. The measurements are obtained by illuminating brain sections from different angles and measuring the transmitted (scattered) light under normal incidence. The evaluation of scattering profiles commonly relies on a peak picking technique and feature extraction from the peaks, which allows quantitative determination of parallel and crossing in-plane nerve fibre directions for each image pixel. However, the estimation of the 3D orientation of the fibres cannot be assessed with the traditional methodology. We propose an unsupervised learning approach using spherical convolutions for estimating the 3D orientation of neural fibres, resulting in a more detailed interpretation of the fibre orientation distributions in the brain.

eess.IV↗

arXiv2022

Automated computation of nerve fibre inclinations from 3D polarised light imaging measurements of brain tissue

The method 3D polarised light imaging (3D-PLI) measures the birefringence of histological brain sections to determine the spatial course of nerve fibres (myelinated axons). While the in-plane fibre directions can be determined with high accuracy, the computation of the out-of-plane fibre inclinations is more challenging because they are derived from the strength of the birefringence signals (retardation), which depends e.g. on the amount of nerve fibres. One possibility to improve the accuracy is to consider the average transmitted light intensity (transmittance weighting). The current procedure requires effortful manual adjustment of parameters and anatomical knowledge. Here, we introduce an automated, optimised computation of the fibre inclinations, allowing for a much faster, reproducible determination of fibre orientations in 3D-PLI. Depending on the degree of myelination, the algorithm uses different models (transmittance-weighted, unweighted, or a linear combination), allowing to go beyond traditional definitions of white and grey matter and account for regionally specific behaviour. As the algorithm is parallelised and GPU optimised, and uses images from standard 3D-PLI (retardation and transmittance), it can be applied to large data sets, also from previous measurements. The functionality is demonstrated on unstained coronal and sagittal histological sections of vervet monkey and rat brains.

physics.bio-ph↗

arXiv2021

Scatterometry Measurements with Scattered Light Imaging Enable New Insights into the Nerve Fiber Architecture of the Brain

The correct reconstruction of individual (crossing) nerve fibers is a prerequisite when constructing a detailed network model of the brain. The recently developed technique Scattered Light Imaging (SLI) allows the reconstruction of crossing nerve fiber pathways in whole brain tissue samples with micrometer resolution: The individual fiber orientations are determined by illuminating unstained histological brain sections from different directions, measuring the transmitted scattered light under normal incidence, and studying the light intensity profiles of each pixel in the resulting image series. So far, SLI measurements were performed with a fixed polar angle of illumination and a small number of illumination directions, providing only an estimate of the nerve fiber directions and limited information about the underlying tissue structure. Here, we use an LED display with individually controllable light-emitting diodes to measure the full distribution of scattered light behind the sample (scattering pattern) for each image pixel at once, enabling scatterometry measurements of whole brain tissue samples. We compare our results to coherent Fourier scatterometry (raster-scanning the sample with a non-focused laser beam) and previous SLI measurements with fixed polar angle of illumination, using sections from a vervet monkey brain and human optic tracts. Finally, we present SLI scatterometry measurements of a human brain section with 3 $μ$m in-plane resolution, demonstrating that the technique is a powerful approach to gain new insights into the nerve fiber architecture of the human brain.

physics.bio-ph↗

arXiv2021

Contour Proposal Networks for Biomedical Instance Segmentation

We present a conceptually simple framework for object instance segmentation called Contour Proposal Network (CPN), which detects possibly overlapping objects in an image while simultaneously fitting closed object contours using an interpretable, fixed-sized representation based on Fourier Descriptors. The CPN can incorporate state of the art object detection architectures as backbone networks into a single-stage instance segmentation model that can be trained end-to-end. We construct CPN models with different backbone networks, and apply them to instance segmentation of cells in datasets from different modalities. In our experiments, we show CPNs that outperform U-Nets and Mask R-CNNs in instance segmentation accuracy, and present variants with execution times suitable for real-time applications. The trained models generalize well across different domains of cell types. Since the main assumption of the framework are closed object contours, it is applicable to a wide range of detection problems also outside the biomedical domain. An implementation of the model architecture in PyTorch is freely available.

cs.CV↗

arXiv2021

Brain Modelling as a Service: The Virtual Brain on EBRAINS

The Virtual Brain (TVB) is now available as open-source cloud ecosystem on EBRAINS, a shared digital research platform for brain science. It offers services for constructing, simulating and analysing brain network models (BNMs) including the TVB network simulator; magnetic resonance imaging (MRI) processing pipelines to extract structural and functional connectomes; multiscale co-simulation of spiking and large-scale networks; a domain specific language for automatic high-performance code generation from user-specified models; simulation-ready BNMs of patients and healthy volunteers; Bayesian inference of epilepsy spread; data and code for mouse brain simulation; and extensive educational material. TVB cloud services facilitate reproducible online collaboration and discovery of data assets, models, and software embedded in scalable and secure workflows, a precondition for research on large cohort data sets, better generalizability and clinical translation.

cs.CE↗

arXiv2021

2D histology meets 3D topology: Cytoarchitectonic brain mapping with Graph Neural Networks

Cytoarchitecture describes the spatial organization of neuronal cells in the brain, including their arrangement into layers and columns with respect to cell density, orientation, or presence of certain cell types. It allows to segregate the brain into cortical areas and subcortical nuclei, links structure with connectivity and function, and provides a microstructural reference for human brain atlases. Mapping boundaries between areas requires to scan histological sections at microscopic resolution. While recent high-throughput scanners allow to scan a complete human brain in the order of a year, it is practically impossible to delineate regions at the same pace using the established gold standard method. Researchers have recently addressed cytoarchitectonic mapping of cortical regions with deep neural networks, relying on image patches from individual 2D sections for classification. However, the 3D context, which is needed to disambiguate complex or obliquely cut brain regions, is not taken into account. In this work, we combine 2D histology with 3D topology by reformulating the mapping task as a node classification problem on an approximate 3D midsurface mesh through the isocortex. We extract deep features from cortical patches in 2D histological sections which are descriptive of cytoarchitecture, and assign them to the corresponding nodes on the 3D mesh to construct a large attributed graph. By solving the brain mapping problem on this graph using graph neural networks, we obtain significantly improved classification results. The proposed framework lends itself nicely to integration of additional neuroanatomical priors for mapping.

eess.IV↗

arXiv2021

Scattered Light Imaging: Resolving the substructure of nerve fiber crossings in whole brain sections with micrometer resolution

For developing a detailed network model of the brain based on image reconstructions, it is necessary to spatially resolve crossing nerve fibers. The accuracy hereby depends on many factors, including the spatial resolution of the imaging technique. 3D Polarized Light Imaging (3D-PLI) allows the three-dimensional reconstruction of nerve fiber tracts in whole brain sections with micrometer in-plane resolution, but leaves uncertainties in pixels containing crossing fibers. Here we introduce Scattered Light Imaging (SLI) to resolve the substructure of nerve fiber crossings. The measurement is performed on the same unstained histological brain sections as in 3D-PLI. By illuminating the brain sections from different angles and measuring the transmitted (scattered) light under normal incidence, SLI provides information about the underlying nerve fiber structure. A fully automated evaluation of the resulting light intensity profiles has been developed, allowing the user to extract various characteristics, like the individual directions of in-plane crossing nerve fibers, for each image pixel at once. We validate the reconstructed nerve fiber directions against results from previous simulation studies, scatterometry measurements, and fiber directions obtained from 3D-PLI. We demonstrate in different brain samples (human optic tracts, vervet monkey brain, rat brain) that the 2D fiber directions can be reliably reconstructed for up to three crossing nerve fiber bundles in each image pixel with an in-plane resolution of up to 6.5 $μ$m. We show that SLI also yields reliable fiber directions in brain regions with low 3D-PLI signals coming from regions with a low density of myelinated nerve fibers or out-of-plane fibers. In combination with 3D-PLI, the technique can be used for a full reconstruction of the three-dimensional nerve fiber architecture in the brain.

physics.med-ph↗

arXiv2021

Contrastive Representation Learning for Whole Brain Cytoarchitectonic Mapping in Histological Human Brain Sections

Cytoarchitectonic maps provide microstructural reference parcellations of the brain, describing its organization in terms of the spatial arrangement of neuronal cell bodies as measured from histological tissue sections. Recent work provided the first automatic segmentations of cytoarchitectonic areas in the visual system using Convolutional Neural Networks. We aim to extend this approach to become applicable to a wider range of brain areas, envisioning a solution for mapping the complete human brain. Inspired by recent success in image classification, we propose a contrastive learning objective for encoding microscopic image patches into robust microstructural features, which are efficient for cytoarchitectonic area classification. We show that a model pre-trained using this learning task outperforms a model trained from scratch, as well as a model pre-trained on a recently proposed auxiliary task. We perform cluster analysis in the feature space to show that the learned representations form anatomically meaningful groups.

eess.IV↗

arXiv2020

Convolutional Neural Networks for cytoarchitectonic brain mapping at large scale

Human brain atlases provide spatial reference systems for data characterizing brain organization at different levels, coming from different brains. Cytoarchitecture is a basic principle of the microstructural organization of the brain, as regional differences in the arrangement and composition of neuronal cells are indicators of changes in connectivity and function. Automated scanning procedures and observer-independent methods are prerequisites to reliably identify cytoarchitectonic areas, and to achieve reproducible models of brain segregation. Time becomes a key factor when moving from the analysis of single regions of interest towards high-throughput scanning of large series of whole-brain sections. Here we present a new workflow for mapping cytoarchitectonic areas in large series of cell-body stained histological sections of human postmortem brains. It is based on a Deep Convolutional Neural Network (CNN), which is trained on a pair of section images with annotations, with a large number of un-annotated sections in between. The model learns to create all missing annotations in between with high accuracy, and faster than our previous workflow based on observer-independent mapping. The new workflow does not require preceding 3D-reconstruction of sections, and is robust against histological artefacts. It processes large data sets with sizes in the order of multiple Terabytes efficiently. The workflow was integrated into a web interface, to allow access without expertise in deep learning and batch computing. Applying deep neural networks for cytoarchitectonic mapping opens new perspectives to enable high-resolution models of brain areas, introducing CNNs to identify borders of brain areas.

eess.IV↗

arXiv2019

Finite-Difference Time-Domain simulations of transmission microscopy enable a better interpretation of 3D nerve fiber architectures in the brain

In many laboratories, conventional bright-field transmission microscopes are available to study the structure and organization principles of fibrous tissue samples, but they usually provide only 2D information. To access the third (out-of-plane) dimension, more advanced techniques are employed. An example is 3D Polarized Light Imaging (3D-PLI), which measures the birefringence of histological brain sections to derive the spatial nerve fiber orientations. Here, we show how light scattering in transmission microscopy measurements can be leveraged to gain 3D structural information about fibrous tissue samples like brain tissue. For this purpose, we developed a simulation framework using finite-difference time-domain (FDTD) simulations and high performance computing, which can easily be adapted to other microscopy techniques and tissue types with comparable fibrous structures (e.g., muscle fibers, collagen, or artificial fibers). As conventional bright-field transmission microscopy provides usually only 2D information about tissue structures, a three-dimensional reconstruction of fibers across several sections is difficult. By combining our simulations with experimental studies, we show that the polarization-independent transmitted light intensity (transmittance) contains 3D information: We demonstrate in several experimental studies on brain sections from different species (rodent, monkey, human) that the transmittance decreases significantly (by more than 50%) with the increasing out-of-plane angle of the nerve fibers. Our FDTD simulations show that this decrease is mainly caused by polarization-independent light scattering in combination with the finite numerical aperture of the imaging system. This allows to use standard transmission microscopy techniques to obtain 3D information about the fiber inclination and to detect steep fibers, without need for additional measurements.

physics.med-ph↗