arXiv subjects

Jianlong Yang

Publications and source records attributed to Jianlong Yang.

At least 19 recordsLinked to original sources

arXiv2024

Computer-generated holography enables high-uniformity, high-efficiency depth-of-focus extension in endoscopic OCT

Fiber-form optics extends the high-resolution tomographic imaging capabilities of Optical Coherence Tomography (OCT) to the inside of the human body, i.e., endoscopic OCT. However, it still faces challenges due to the trade-off between probe size, resolution, and Depth Of Focus (DOF). Here we introduce a method for extending the DOF in endoscopic OCT with high uniformity and efficiency. On the basis of multi-level diffractive optics, we leverage the multi-dimensional light field modulation capabilities of Computer-Generated Holography (CGH), to achieve precise control of the intensity distribution of the off-axis portion of the OCT probe light. Our method eliminates the need for an objective lens, allowing for direct fabrication at the distal facet of a single-mode fiber using femtosecond laser two-photon 3D printing. The superiority of our method has been verified through numerical simulation, beam measurement, and imaging results obtained with our home-built endoscopic OCT system.

physics.optics↗

arXiv2024

Cross-attention learning enables real-time nonuniform rotational distortion correction in OCT

Nonuniform rotational distortion (NURD) correction is vital for endoscopic optical coherence tomography (OCT) imaging and its functional extensions, such as angiography and elastography. Current NURD correction methods require time-consuming feature tracking or cross-correlation calculations and thus sacrifice temporal resolution. Here we propose a cross-attention learning method for the NURD correction in OCT. Our method is inspired by the recent success of the self-attention mechanism in natural language processing and computer vision. By leveraging its ability to model long-range dependencies, we can directly obtain the correlation between OCT A-lines at any distance, thus accelerating the NURD correction. We develop an end-to-end stacked cross-attention network and design three types of optimization constraints. We compare our method with two traditional feature-based methods and a CNN-based method, on two publicly-available endoscopic OCT datasets and a private dataset collected on our home-built endoscopic OCT system. Our method achieved a $\sim3\times$ speedup to real time ($26\pm 3$ fps), and superior correction performance.

eess.IV↗

arXiv2023

Towards ultra-low-cost smartphone microscopy

The outbreak of COVID-19 exposed the inadequacy of our technical tools for home health surveillance, and recent studies have shown the potential of smartphones as a universal optical microscopic imaging platform for such applications. However, most of them use laboratory-grade optomechanical components and transmitted illuminations to ensure focus tuning capability and imaging quality, which keeps the cost of the equipment high. Here we propose an ultra-low-cost solution for smartphone microscopy. To realize focus tunability, we designed a seesaw-like structure capable of converting large displacements on one side into small displacements on the other (reduced to ~9.1%), which leverages the intrinsic flexibility of 3D printing materials. We achieved a focus-tuning accuracy of ~5 micron, which is 40 times higher than the machining accuracy of the 3D-printed lens holder itself. For microscopic imaging, we use an off-the-shelf smartphone camera lens as the objective and the built-in flashlight as the illumination. To compensate for the resulting image quality degradation, we developed a learning-based image enhancement method. We use the CycleGAN architecture to establish the mapping from smartphone microscope images to benchtop microscope images without pairing. We verified the imaging performance on different biomedical samples. Except for the smartphone, we kept the full costs of the device under 4 USD. We think these efforts to lower the costs of smartphone microscopes will benefit their applications in various scenarios, such as point-of-care testing, on-site diagnosis, and home health surveillance.

eess.IV↗

arXiv2023

Annotation-efficient learning for OCT segmentation

Deep learning has been successfully applied to OCT segmentation. However, for data from different manufacturers and imaging protocols, and for different regions of interest (ROIs), it requires laborious and time-consuming data annotation and training, which is undesirable in many scenarios, such as surgical navigation and multi-center clinical trials. Here we propose an annotation-efficient learning method for OCT segmentation that could significantly reduce annotation costs. Leveraging self-supervised generative learning, we train a Transformer-based model to learn the OCT imagery. Then we connect the trained Transformer-based encoder to a CNN-based decoder, to learn the dense pixel-wise prediction in OCT segmentation. These training phases use open-access data and thus incur no annotation costs, and the pre-trained model can be adapted to different data and ROIs without re-training. Based on the greedy approximation for the k-center problem, we also introduce an algorithm for the selective annotation of the target data. We verified our method on publicly-available and private OCT datasets. Compared to the widely-used U-Net model with 100% training data, our method only requires ~10% of the data for achieving the same segmentation accuracy, and it speeds the training up to ~3.5 times. Furthermore, our proposed method outperforms other potential strategies that could improve annotation efficiency. We think this emphasis on learning efficiency may help improve the intelligence and application penetration of OCT-based technologies. Our code and pre-trained model are publicly available at https://github.com/SJTU-Intelligent-Optics-Lab/Annotation-efficient-learning-for-OCT-segmentation.

cs.CV↗

arXiv2021

Proxy-bridged Image Reconstruction Network for Anomaly Detection in Medical Images

Anomaly detection in medical images refers to the identification of abnormal images with only normal images in the training set. Most existing methods solve this problem with a self-reconstruction framework, which tends to learn an identity mapping and reduces the sensitivity to anomalies. To mitigate this problem, in this paper, we propose a novel Proxy-bridged Image Reconstruction Network (ProxyAno) for anomaly detection in medical images. Specifically, we use an intermediate proxy to bridge the input image and the reconstructed image. We study different proxy types, and we find that the superpixel-image (SI) is the best one. We set all pixels' intensities within each superpixel as their average intensity, and denote this image as SI. The proposed ProxyAno consists of two modules, a Proxy Extraction Module and an Image Reconstruction Module. In the Proxy Extraction Module, a memory is introduced to memorize the feature correspondence for normal image to its corresponding SI, while the memorized correspondence does not apply to the abnormal images, which leads to the information loss for abnormal image and facilitates the anomaly detection. In the Image Reconstruction Module, we map an SI to its reconstructed image. Further, we crop a patch from the image and paste it on the normal SI to mimic the anomalies, and enforce the network to reconstruct the normal image even with the pseudo abnormal SI. In this way, our network enlarges the reconstruction error for anomalies. Extensive experiments on brain MR images, retinal OCT images and retinal fundus images verify the effectiveness of our method for both image-level and pixel-level anomaly detection.

eess.IV↗

arXiv2020

ROSE: A Retinal OCT-Angiography Vessel Segmentation Dataset and New Model

Optical Coherence Tomography Angiography (OCT-A) is a non-invasive imaging technique, and has been increasingly used to image the retinal vasculature at capillary level resolution. However, automated segmentation of retinal vessels in OCT-A has been under-studied due to various challenges such as low capillary visibility and high vessel complexity, despite its significance in understanding many eye-related diseases. In addition, there is no publicly available OCT-A dataset with manually graded vessels for training and validation. To address these issues, for the first time in the field of retinal image analysis we construct a dedicated Retinal OCT-A SEgmentation dataset (ROSE), which consists of 229 OCT-A images with vessel annotations at either centerline-level or pixel level. This dataset has been released for public access to assist researchers in the community in undertaking research in related topics. Secondly, we propose a novel Split-based Coarse-to-Fine vessel segmentation network (SCF-Net), with the ability to detect thick and thin vessels separately. In the SCF-Net, a split-based coarse segmentation (SCS) module is first introduced to produce a preliminary confidence map of vessels, and a split-based refinement (SRN) module is then used to optimize the shape/contour of the retinal microvasculature. Thirdly, we perform a thorough evaluation of the state-of-the-art vessel segmentation models and our SCF-Net on the proposed ROSE dataset. The experimental results demonstrate that our SCF-Net yields better vessel segmentation performance in OCT-A than both traditional methods and other deep learning methods.

eess.IV↗

arXiv2020

CS2-Net: Deep Learning Segmentation of Curvilinear Structures in Medical Imaging

Automated detection of curvilinear structures, e.g., blood vessels or nerve fibres, from medical and biomedical images is a crucial early step in automatic image interpretation associated to the management of many diseases. Precise measurement of the morphological changes of these curvilinear organ structures informs clinicians for understanding the mechanism, diagnosis, and treatment of e.g. cardiovascular, kidney, eye, lung, and neurological conditions. In this work, we propose a generic and unified convolution neural network for the segmentation of curvilinear structures and illustrate in several 2D/3D medical imaging modalities. We introduce a new curvilinear structure segmentation network (CS2-Net), which includes a self-attention mechanism in the encoder and decoder to learn rich hierarchical representations of curvilinear structures. Two types of attention modules - spatial attention and channel attention - are utilized to enhance the inter-class discrimination and intra-class responsiveness, to further integrate local features with their global dependencies and normalization, adaptively. Furthermore, to facilitate the segmentation of curvilinear structures in medical images, we employ a 1x3 and a 3x1 convolutional kernel to capture boundary features. ...

eess.IV↗

arXiv2020

Automatic Segmentation and Visualization of Choroid in OCT with Knowledge Infused Deep Learning

The choroid provides oxygen and nourishment to the outer retina thus is related to the pathology of various ocular diseases. Optical coherence tomography (OCT) is advantageous in visualizing and quantifying the choroid in vivo. (1) The lower boundary of the choroid (choroid-sclera interface) in OCT is fuzzy, which makes the automatic segmentation difficult and inaccurate. (2) The visualization of the choroid is hindered by the vessel shadows from the superficial layers of the inner retina. In this paper, we propose to incorporate medical and imaging prior knowledge with deep learning to address these two problems. We propose a biomarker infused global-to-local network for the choroid segmentation. It leverages the choroidal thickness, a primary biomarker in clinic, as a constraint to improve the segmentation accuracy. We also design a global-to-local strategy in the choroid segmentation: a global module is used to segment all the retinal and choroidal layers simultaneously for suppressing overfitting and providing global structure information, then a local module is used to refine the segmentation with the biomarker infusion. To eliminate the retinal vessel shadows, we propose a pipeline that firstly use anatomical and OCT imaging knowledge to locate the shadows using their projection on the retinal pigment epithelium layer, then the contents of the choroidal vasculature at the shadow locations are predicted with an edge-to-texture generative adversarial inpainting network. The experiments show our method outperforms the existing methods on both the segmentation and shadow elimination tasks. We further apply the proposed method in a clinical prospective study for understanding the pathology of glaucoma by detecting the structure and vascular changes of the choroid related to the elevation of intra-ocular pressure.

eess.IV↗

arXiv2020

Encoding Structure-Texture Relation with P-Net for Anomaly Detection in Retinal Images

Anomaly detection in retinal image refers to the identification of abnormality caused by various retinal diseases/lesions, by only leveraging normal images in training phase. Normal images from healthy subjects often have regular structures (e.g., the structured blood vessels in the fundus image, or structured anatomy in optical coherence tomography image). On the contrary, the diseases and lesions often destroy these structures. Motivated by this, we propose to leverage the relation between the image texture and structure to design a deep neural network for anomaly detection. Specifically, we first extract the structure of the retinal images, then we combine both the structure features and the last layer features extracted from original health image to reconstruct the original input healthy image. The image feature provides the texture information and guarantees the uniqueness of the image recovered from the structure. In the end, we further utilize the reconstructed image to extract the structure and measure the difference between structure extracted from original and the reconstructed image. On the one hand, minimizing the reconstruction difference behaves like a regularizer to guarantee that the image is corrected reconstructed. On the other hand, such structure difference can also be used as a metric for normality measurement. The whole network is termed as P-Net because it has a ``P'' shape. Extensive experiments on RESC dataset and iSee dataset validate the effectiveness of our approach for anomaly detection in retinal images. Further, our method also generalizes well to novel class discovery in retinal images and anomaly detection in real-world images.

eess.IV↗

arXiv2020

Open-Narrow-Synechiae Anterior Chamber Angle Classification in AS-OCT Sequences

Anterior chamber angle (ACA) classification is a key step in the diagnosis of angle-closure glaucoma in Anterior Segment Optical Coherence Tomography (AS-OCT). Existing automated analysis methods focus on a binary classification system (i.e., open angle or angle-closure) in a 2D AS-OCT slice. However, clinical diagnosis requires a more discriminating ACA three-class system (i.e., open, narrow, or synechiae angles) for the benefit of clinicians who seek better to understand the progression of the spectrum of angle-closure glaucoma types. To address this, we propose a novel sequence multi-scale aggregation deep network (SMA-Net) for open-narrow-synechiae ACA classification based on an AS-OCT sequence. In our method, a Multi-Scale Discriminative Aggregation (MSDA) block is utilized to learn the multi-scale representations at slice level, while a ConvLSTM is introduced to study the temporal dynamics of these representations at sequence level. Finally, a multi-level loss function is used to combine the slice-based and sequence-based losses. The proposed method is evaluated across two AS-OCT datasets. The experimental results show that the proposed method outperforms existing state-of-the-art methods in applicability, effectiveness, and accuracy. We believe this work to be the first attempt to classify ACAs into open, narrow, or synechia types grading using AS-OCT sequences.

cs.CV↗

arXiv2020

High signal-to-noise ratio reconstruction of low bit-depth optical coherence tomography using deep learning

Reducing the bit-depth is an effective approach to lower the cost of optical coherence tomography (OCT) systems and increase the transmission efficiency in data acquisition and telemedicine. However, a low bit-depth will lead to the degeneration of the detection sensitivity thus reduce the signal-to-noise ratio (SNR) of OCT images. In this paper, we propose to use deep learning for the reconstruction of the high SNR OCT images from the low bit-depth acquisition. Its feasibility was preliminarily evaluated by applying the proposed method to the quantized $3\sim8$-bit data from native 12-bit interference fringes. We employed a pixel-to-pixel generative adversarial network architecture in the low to high bit-depth OCT image transition. Retinal OCT data of a healthy subject from a homemade spectral-domain OCT system was used in the study. Extensively qualitative and quantitative results show this deep-learning-based approach could significantly improve the SNR of the low bit-depth OCT images especially at the choroidal region. Superior similarity and SNR between the reconstructed images and the original 12-bit OCT images could be derived when the bit-depth $\geq 5$. This work demonstrates the proper integration of OCT and deep learning could benefit the development of healthcare in low-resource settings.

eess.IV↗

arXiv2020

Sparse-GAN: Sparsity-constrained Generative Adversarial Network for Anomaly Detection in Retinal OCT Image

With the development of convolutional neural network, deep learning has shown its success for retinal disease detection from optical coherence tomography (OCT) images. However, deep learning often relies on large scale labelled data for training, which is oftentimes challenging especially for disease with low occurrence. Moreover, a deep learning system trained from data-set with one or a few diseases is unable to detect other unseen diseases, which limits the practical usage of the system in disease screening. To address the limitation, we propose a novel anomaly detection framework termed Sparsity-constrained Generative Adversarial Network (Sparse-GAN) for disease screening where only healthy data are available in the training set. The contributions of Sparse-GAN are two-folds: 1) The proposed Sparse-GAN predicts the anomalies in latent space rather than image-level; 2) Sparse-GAN is constrained by a novel Sparsity Regularization Net. Furthermore, in light of the role of lesions for disease screening, we present to leverage on an anomaly activation map to show the heatmap of lesions. We evaluate our proposed Sparse-GAN on a publicly available dataset, and the results show that the proposed method outperforms the state-of-the-art methods.

cs.CV↗

arXiv2020

Universal digital filtering for denoising volumetric retinal OCT and OCT angiography in 3D shearlet domain

Retinal optical coherence tomography (OCT) and OCT angiography (OCTA) suffer from the degeneration of image quality due to speckle noise and bulk-motion noise, respectively. Because the cross-sectional retina has distinct features in OCT and OCTA B-scans, existing digital filters that can denoise OCT efficiently are unable to handle the bulk-motion noise in OCTA. In this Letter, we propose a universal digital filtering approach that is capable of minimizing both types of noise. Considering the retinal capillaries in OCTA are hard to differentiate in B-scans while having distinct curvilinear structures in 3D volumes, we decompose the volumetric OCT and OCTA data with 3D shearlets thus efficiently separate the retinal tissue and vessels from the noise in this transform domain. Compared with wavelets and curvelets, the shearlets provide better representation of the layer edges in OCT and the vasculature in OCTA. Qualitative and quantitative results show the proposed method outperforms the state-of-the-art OCT and OCTA denoising methods. Besides, the superiority of 3D denoising is demonstrated by comparing the 3D shearlet filtering with its 2D counterpart.

eess.IV↗

arXiv2020

Digital resolution enhancement in low transverse sampling optical coherence tomography angiography using deep learning

Optical coherence tomography angiography (OCTA) requires high transverse sampling density for visualizing retinal and choroidal capillaries. Low transverse sampling causes resolution degradation, such as the angiograms in wide-field OCTA. In this paper, we propose to address this problem using deep learning. We conducted extensive experiments on converting the centrally cropped 3 x 3 mm2 field of view (FOV) of the 8 x 8 mm2 foveal OCTA images (a sampling density of 22.9 $μ$m) to the native 3 x 3 mm2 en face OCTA images (a sampling density of 12.2 $μ$m). We employed a cycle-consistent adversarial network architecture in this conversion. The quantitative analysis using the perceptual similarity measures shows the generated OCTA images are closer to the native 3 x 3 mm2 scans. Besides, the results show the proposed method could also enhance signal-to-noise ratio. We further applied our method to enhance diseased cases and calculate vascular biomarkers, which demonstrates its generalization performance and clinical perspective.

eess.IV↗

arXiv2019

BioNet: Infusing Biomarker Prior into Global-to-Local Network for Choroid Segmentation in Optical Coherence Tomography Images

Choroid is the vascular layer of the eye, which is directly related to the incidence and severity of many ocular diseases. Optical Coherence Tomography (OCT) is capable of imaging both the cross-sectional view of retina and choroid, but the segmentation of the choroid region is challenging because of the fuzzy choroid-sclera interface (CSI). In this paper, we propose a biomarker infused global-to-local network (BioNet) for choroid segmentation, which segments the choroid with higher credibility and robustness. Firstly, our method trains a biomarker prediction network to learn the features of the biomarker. Then a global multi-layers segmentation module is applied to segment the OCT image into 12 layers. Finally, the global multi-layered result and the original OCT image are fed into a local choroid segmentation module to segment the choroid region with the biomarker infused as regularizer. We conducted comparison experiments with the state-of-the-art methods on a dataset (named AROD). The experimental results demonstrate the superiority of our method with $90.77\%$ Dice-index and 6.23 pixels Average-unsigned-surface-detection-error, etc.

cs.CV↗

arXiv2019

Perceptual-assisted Adversarial Adaptation for Choroid Segmentation in optical coherence tomography

Accurate choroid segmentation in optical coherence tomography (OCT) image is vital because the choroid thickness is a major quantitative biomarker of many ocular diseases. Deep learning has shown its superiority in the segmentation of the choroid region but subjects to the performance degeneration caused by the domain discrepancies (e.g., noise level and distribution) among datasets obtained from the OCT devices of different manufacturers. In this paper, we present an unsupervised perceptual-assisted adversarial adaptation (PAAA) framework for efficiently segmenting the choroid area by narrowing the domain discrepancies between different domains. The adversarial adaptation module in the proposed framework encourages the prediction structure information of the target domain to be similar to that of the source domain. Besides, a perceptual loss is employed for matching their shape information (the curvatures of Bruch's membrane and choroid-sclera interface) which can result in a fine boundary prediction. The results of quantitative experiments show that the proposed PAAA segmentation framework outperforms other state-of-the-art methods.

eess.IV↗

arXiv2019

Assessment of Generative Adversarial Networks Model for Synthetic Optical Coherence Tomography Images of Retinal Disorders

Purpose: To assess whether a generative adversarial network (GAN) could synthesize realistic optical coherence tomography (OCT) images that satisfactorily serve as the educational images for retinal specialists and the training datasets for the classification of various retinal disorders using deep learning (DL). Methods: The GANs architecture was adopted to synthesis high-resolution OCT images training on a publicly available OCT dataset including urgent referrals (choroidal neovascularization and diabetic macular edema) and non-urgent referrals (normal and drusen). 400 real and synthetic OCT images were evaluated by 2 retinal specialists to assess image quality. We further trained 2 DL models on either real or synthetic datasets and compared the performance of urgent vs nonurgent referrals diagnosis tested on a local (1000 images from the public dataset) and clinical validation dataset (278 images from Shanghai Shibei Hospital). Results: The image quality of real vs synthetic OCT images was similar as assessed by 2 retinal specialists. The accuracy of discrimination as real vs synthetic OCT images was 59.50% for retinal specialist 1 and 53.67% for retinal specialist 2. For the local dataset, the DL model trained on real (DL_Model_R) and synthetic OCT images (DL_Model_S) had an area under the curve (AUC) of 0.99, and 0.98 respectively. For the clinical dataset, the AUC was 0.94 for DL_Model_R, 0.90 for DL_Model_S. Conclusions: The GAN-synthetic OCT images can be used by clinicians for educational purposes and developing DL algorithms. Translational Relevance: The medical image synthesis based on GANs is promising in human and machine to fulfill clinical tasks.

physics.med-ph↗

arXiv2019

Knowledge infused cascade convolutional neural network for segmenting retinal vessels in volumetric optical coherence tomography

We present a cascade deep neural network to segment retinal vessels in volumetric optical coherence tomography (OCT). Two types of knowledge are infused into the network for confining the searching regions. (1) Histology. The retinal vessels locate between the inner limiting membrane and the inner nuclear layer of human retina. (2) Imaging. The red blood cells inside the vessels scatter the OCT probe light forward and form projection shadows on the retinal pigment epithelium (RPE) layer, which is avascular thus perfect for localizing the retinal vessel in transverse plane. Qualitative and quantitative comparison results show that the proposed method outperforms the state-of-the-art deep learning and graph-based methods. This work demonstrates, instead of modifying the architectures of the deep networks, incorporating proper prior knowledge in the design of the image processing framework could be an efficient approach for handling such specific tasks.

eess.IV↗