arXiv subjects

Tao Song

Publications and source records attributed to Tao Song.

68 records · Page 4Linked to original sources

arXiv2022

TCR-GAN: Predicting tropical cyclone passive microwave rainfall using infrared imagery via generative adversarial networks

Tropical cyclones (TC) generally carry large amounts of water vapor and can cause large-scale extreme rainfall. Passive microwave rainfall (PMR) estimation of TC with high spatial and temporal resolution is crucial for disaster warning of TC, but remains a challenging problem due to the low temporal resolution of microwave sensors. This study attempts to solve this problem by directly forecasting PMR from satellite infrared (IR) images of TC. We develop a generative adversarial network (GAN) to convert IR images into PMR, and establish the mapping relationship between TC cloud-top bright temperature and PMR, the algorithm is named TCR-GAN. Meanwhile, a new dataset that is available as a benchmark, Dataset of Tropical Cyclone IR-to-Rainfall Prediction (TCIRRP) was established, which is expected to advance the development of artificial intelligence in this direction. Experimental results show that the algorithm can effectively extract key features from IR. The end-to-end deep learning approach shows potential as a technique that can be applied globally and provides a new perspective tropical cyclone precipitation prediction via satellite, which is expected to provide important insights for real-time visualization of TC rainfall globally in operations.

cs.LG↗

arXiv2022

SCPM-Net: An Anchor-free 3D Lung Nodule Detection Network using Sphere Representation and Center Points Matching

Lung nodule detection from 3D Computed Tomography scans plays a vital role in efficient lung cancer screening. Despite the SOTA performance obtained by recent anchor-based detectors using CNNs for this task, they require predetermined anchor parameters such as the size, number, and aspect ratio of anchors, and have limited robustness when dealing with lung nodules with a massive variety of sizes. To overcome these problems, we propose a 3D sphere representation-based center-points matching detection network that is anchor-free and automatically predicts the position, radius, and offset of nodules without the manual design of nodule/anchor parameters. The SCPM-Net consists of two novel components: sphere representation and center points matching. First, to match the nodule annotation in clinical practice, we replace the commonly used bounding box with our proposed bounding sphere to represent nodules with the centroid, radius, and local offset in 3D space. A compatible sphere-based intersection over-union loss function is introduced to train the lung nodule detection network stably and efficiently. Second, we empower the network anchor-free by designing a positive center-points selection and matching process, which naturally discards pre-determined anchor boxes. An online hard example mining and re-focal loss subsequently enable the CPM process to be more robust, resulting in more accurate point assignment and mitigation of class imbalance. In addition, to better capture spatial information and 3D context for the detection, we propose to fuse multi-level spatial coordinate maps with the feature extractor and combine them with 3D squeeze-and-excitation attention modules. Experimental results on the LUNA16 dataset showed that our proposed framework achieves superior performance compared with existing anchor-based and anchor-free methods for lung nodule detection.

cs.CV↗

arXiv2021

Efficient Semi-Supervised Gross Target Volume of Nasopharyngeal Carcinoma Segmentation via Uncertainty Rectified Pyramid Consistency

Gross Target Volume (GTV) segmentation plays an irreplaceable role in radiotherapy planning for Nasopharyngeal Carcinoma (NPC). Despite that Convolutional Neural Networks (CNN) have achieved good performance for this task, they rely on a large set of labeled images for training, which is expensive and time-consuming to acquire. In this paper, we propose a novel framework with Uncertainty Rectified Pyramid Consistency (URPC) regularization for semi-supervised NPC GTV segmentation. Concretely, we extend a backbone segmentation network to produce pyramid predictions at different scales. The pyramid predictions network (PPNet) is supervised by the ground truth of labeled images and a multi-scale consistency loss for unlabeled images, motivated by the fact that prediction at different scales for the same input should be similar and consistent. However, due to the different resolution of these predictions, encouraging them to be consistent at each pixel directly has low robustness and may lose some fine details. To address this problem, we further design a novel uncertainty rectifying module to enable the framework to gradually learn from meaningful and reliable consensual regions at different scales. Experimental results on a dataset with 258 NPC MR images showed that with only 10% or 20% images labeled, our method largely improved the segmentation performance by leveraging the unlabeled images, and it also outperformed five state-of-the-art semi-supervised segmentation methods. Moreover, when only 50% images labeled, URPC achieved an average Dice score of 82.74% that was close to fully supervised learning.

cs.CV↗

arXiv2021

Fast and Accurate Scene Parsing via Bi-direction Alignment Networks

In this paper, we propose an effective method for fast and accurate scene parsing called Bidirectional Alignment Network (BiAlignNet). Previously, one representative work BiSeNet~\cite{bisenet} uses two different paths (Context Path and Spatial Path) to achieve balanced learning of semantics and details, respectively. However, the relationship between the two paths is not well explored. We argue that both paths can benefit each other in a complementary way. Motivated by this, we propose a novel network by aligning two-path information into each other through a learned flow field. To avoid the noise and semantic gaps, we introduce a Gated Flow Alignment Module to align both features in a bidirectional way. Moreover, to make the Spatial Path learn more detailed information, we present an edge-guided hard pixel mining loss to supervise the aligned learning process. Our method achieves 80.1\% and 78.5\% mIoU in validation and test set of Cityscapes while running at 30 FPS with full resolution inputs. Code and models will be available at \url{https://github.com/jojacola/BiAlignNet}.

cs.CV↗

arXiv2021

MIDeepSeg: Minimally Interactive Segmentation of Unseen Objects from Medical Images Using Deep Learning

Segmentation of organs or lesions from medical images plays an essential role in many clinical applications such as diagnosis and treatment planning. Though Convolutional Neural Networks (CNN) have achieved the state-of-the-art performance for automatic segmentation, they are often limited by the lack of clinically acceptable accuracy and robustness in complex cases. Therefore, interactive segmentation is a practical alternative to these methods. However, traditional interactive segmentation methods require a large amount of user interactions, and recently proposed CNN-based interactive segmentation methods are limited by poor performance on previously unseen objects. To solve these problems, we propose a novel deep learning-based interactive segmentation method that not only has high efficiency due to only requiring clicks as user inputs but also generalizes well to a range of previously unseen objects. Specifically, we first encode user-provided interior margin points via our proposed exponentialized geodesic distance that enables a CNN to achieve a good initial segmentation result of both previously seen and unseen objects, then we use a novel information fusion method that combines the initial segmentation with only few additional user clicks to efficiently obtain a refined segmentation. We validated our proposed framework through extensive experiments on 2D and 3D medical image segmentation tasks with a wide range of previous unseen objects that were not present in the training set. Experimental results showed that our proposed framework 1) achieves accurate results with fewer user interactions and less time compared with state-of-the-art interactive frameworks and 2) generalizes well to previously unseen objects.

cs.CV↗

arXiv2020

CA-Net: Comprehensive Attention Convolutional Neural Networks for Explainable Medical Image Segmentation

Accurate medical image segmentation is essential for diagnosis and treatment planning of diseases. Convolutional Neural Networks (CNNs) have achieved state-of-the-art performance for automatic medical image segmentation. However, they are still challenged by complicated conditions where the segmentation target has large variations of position, shape and scale, and existing CNNs have a poor explainability that limits their application to clinical decisions. In this work, we make extensive use of multiple attentions in a CNN architecture and propose a comprehensive attention-based CNN (CA-Net) for more accurate and explainable medical image segmentation that is aware of the most important spatial positions, channels and scales at the same time. In particular, we first propose a joint spatial attention module to make the network focus more on the foreground region. Then, a novel channel attention module is proposed to adaptively recalibrate channel-wise feature responses and highlight the most relevant feature channels. Also, we propose a scale attention module implicitly emphasizing the most salient feature maps among multiple scales so that the CNN is adaptive to the size of an object. Extensive experiments on skin lesion segmentation from ISIC 2018 and multi-class segmentation of fetal MRI found that our proposed CA-Net significantly improved the average segmentation Dice score from 87.77% to 92.08% for skin lesion, 84.79% to 87.08% for the placenta and 93.20% to 95.88% for the fetal brain respectively compared with U-Net. It reduced the model size to around 15 times smaller with close or even better accuracy compared with state-of-the-art DeepLabv3+. In addition, it has a much higher explainability than existing networks by visualizing the attention weight maps. Our code is available at https://github.com/HiLab-git/CA-Net

eess.IV↗

arXiv2020

Automatic Ischemic Stroke Lesion Segmentation from Computed Tomography Perfusion Images by Image Synthesis and Attention-Based Deep Neural Networks

Ischemic stroke lesion segmentation from Computed Tomography Perfusion (CTP) images is important for accurate diagnosis of stroke in acute care units. However, it is challenged by low image contrast and resolution of the perfusion parameter maps, in addition to the complex appearance of the lesion. To deal with this problem, we propose a novel framework based on synthesized pseudo Diffusion-Weighted Imaging (DWI) from perfusion parameter maps to obtain better image quality for more accurate segmentation. Our framework consists of three components based on Convolutional Neural Networks (CNNs) and is trained end-to-end. First, a feature extractor is used to obtain both a low-level and high-level compact representation of the raw spatiotemporal Computed Tomography Angiography (CTA) images. Second, a pseudo DWI generator takes as input the concatenation of CTP perfusion parameter maps and our extracted features to obtain the synthesized pseudo DWI. To achieve better synthesis quality, we propose a hybrid loss function that pays more attention to lesion regions and encourages high-level contextual consistency. Finally, we segment the lesion region from the synthesized pseudo DWI, where the segmentation network is based on switchable normalization and channel calibration for better performance. Experimental results showed that our framework achieved the top performance on ISLES 2018 challenge and: 1) our method using synthesized pseudo DWI outperformed methods segmenting the lesion from perfusion parameter maps directly; 2) the feature extractor exploiting additional spatiotemporal CTA images led to better synthesized pseudo DWI quality and higher segmentation accuracy; and 3) the proposed loss functions and network structure improved the pseudo DWI synthesis and lesion segmentation performance.

eess.IV↗

arXiv2020

KLDivNet: An unsupervised neural network for multi-modality image registration

Multi-modality image registration is one of the most underlined processes in medical image analysis. Recently, convolutional neural networks (CNNs) have shown significant potential in deformable registration. However, the lack of voxel-wise ground truth challenges the training of CNNs for an accurate registration. In this work, we propose a cross-modality similarity metric, based on the KL-divergence of image variables, and implement an efficient estimation method using a CNN. This estimation network, referred to as KLDivNet, can be trained unsupervisedly. We then embed the KLDivNet into a registration network to achieve the unsupervised deformable registration for multi-modality images. We employed three datasets, i.e., AAL Brain, LiTS Liver and Hospital Liver, with both the intra- and inter-modality image registration tasks for validation. Results showed that our similarity metric was effective, and the proposed registration network delivered superior performance compared to the state-of-the-art methods.

cs.CV↗

arXiv2019

Generative Model-Based Ischemic Stroke Lesion Segmentation

CT perfusion (CTP) has been used to triage ischemic stroke patients in the early stage, because of its speed, availability, and lack of contraindications. Perfusion parameters including cerebral blood volume (CBV), cerebral blood flow (CBF), mean transit time (MTT) and time of peak (Tmax) could also be computed from CTP data. However, CTP data or the perfusion parameters, are ambiguous to locate the infarct core or tissue at risk (penumbra), which is normally confirmed by the follow-up Diffusion Weighted Imaging (DWI) or perfusion diffusion mismatch. In this paper, we propose a novel generative modelbased segmentation framework composed of an extractor, a generator and a segmentor for ischemic stroke lesion segmentation. First, an extractor is used to directly extract the representative feature images from the CTP feature images. Second, a generator is used to generate the clinical relevant DWI images using the output from the extractor and perfusion parameters. Finally, the segmentor is used to precisely segment the ischemic stroke lesion using the generated DWI from the generator. Meanwhile, a novel pixel-region loss function, generalized dice combined with weighted cross entropy, is used to handle data unbalance problem which is commonly encountered in medical image segmentation. All networks are trained end-to-end from scratch using the 2018 Ischemic Stroke Lesion Segmentation Challenge (ISLES) dataset and our method won the first place in the 2018 ischemic stroke lesions segmentation challenge in the test stage.

eess.IV↗

arXiv2018

Small-scale Pedestrian Detection Based on Somatic Topology Localization and Temporal Feature Aggregation

A critical issue in pedestrian detection is to detect small-scale objects that will introduce feeble contrast and motion blur in images and videos, which in our opinion should partially resort to deep-rooted annotation bias. Motivated by this, we propose a novel method integrated with somatic topological line localization (TLL) and temporal feature aggregation for detecting multi-scale pedestrians, which works particularly well with small-scale pedestrians that are relatively far from the camera. Moreover, a post-processing scheme based on Markov Random Field (MRF) is introduced to eliminate ambiguities in occlusion cases. Applying with these methodologies comprehensively, we achieve best detection performance on Caltech benchmark and improve performance of small-scale objects significantly (miss rate decreases from 74.53% to 60.79%). Beyond this, we also achieve competitive performance on CityPersons dataset and show the existence of annotation bias in KITTI dataset.

cs.CV↗

arXiv2017

Mechanism of cellular effect directly induced by magnetic nanoparticles under magnetic fields

The interaction of magnetic nanoparticles (MNPs) with various magnetic fields could directly induce cellular effects. Many scattered investigations have got involved in these cellular effects, analyzed their relative mechanisms and extended their biomedical uses in magnetic hyperthermia and cell regulation. This review reports these cellular effects and their important applications in biomedical area. More importantly, we highlight the underlying mechanisms behind these direct cellular effects in the review from the thermal energy and mechanical force. Recently, some physical analyses showed that the mechanisms of heat and mechanical force in cellular effects are controversial. Although the physical principle plays an important role in these cellular effects, some chemical reactions such as free radical reaction also existed in the interaction of MNPs with magnetic fields, which provides the possible explanation for the current controversy. It's anticipated that the review here could provide readers a deeper understanding of mechanisms of how MNPs contribute to the direct cellular effects and thus their biomedical applications under various magnetic fields.

physics.bio-ph↗

arXiv2016

Multi-fold Darboux transformations of the extended bigraded Toda hierarchy

With the extended logarithmic flow equations and some extended Vertex operators in generalized Hirota bilinear equations, extended bigraded Toda hierarchy(EBTH) was proved to govern the Gromov-Witten theory of orbiford $c_{NM}$ in literature. The generating function of these Gromov-Witten invariants is one special solution of the EBTH. In this paper, the multi-fold Darboux transformations and their determinant representations of the EBTH are given with two different gauge transformation operators. The two Darboux transformations in different directions are used to generate new solutions from known solutions which include soliton solutions of $(N,N)$-EBTH, i.e. the EBTH when $N=M$. From the generation of new solutions, one can find the big difference between the EBTH and the extended Toda hierarchy(ETH). Meanwhile we plotted the soliton graphs of the $(N,N)$-EBTH from which some approximation analysis will be given. From the analysis on velocities of soliton solutions, the difference between the extended flows and other flows are shown. The two different Darboux transformations constructed by us might be useful in Gromov-Witten theory of orbiford $c_{NM}$.

nlin.SI↗

arXiv2012

Optimal Contrast Greyscale Visual Cryptography Schemes with Reversing

Visual cryptography scheme (VCS) is an encryption technique that utilizes human visual system in recovering secret image and it does not require any complex calculation. However, the contrast of the reconstructed image could be quite low. A number of reversing-based VCSs (or VCSs with reversing) (RVCS) have been proposed for binary secret images, allowing participants to perform a reversing operation on shares (or shadows). This reversing operation can be easily implemented by current copy machines. Some existing traditional VCS schemes without reversing (nRVCS) can be extended to RVCS with the same pixel expansion for binary image, and the RVCS can achieve ideal contrast, significantly higher than that of the corresponding nRVCS. In the application of greyscale VCS, the contrast is much lower than that of the binary cases. Therefore, it is more desirable to improve the contrast in the greyscale image reconstruction. However, when greyscale images are involved, one cannot take advantage of this reversing operation so easily. Many existing greyscale nRVCS cannot be directly extended to RVCS. In this paper, we first give a new greyscale nRVCS with minimum pixel expansion and propose an optimal-contrast greyscale RVCS (GRVCS) by using basis matrices of perfect black nRVCS. Also, we propose an optimal GRVCS even though the basis matrices are not perfect black. Finally, we design an optimal-contrast GRVCS with minimum number of shares held by each participant. The proposed schemes can satisfy different user requirement, previous RVCSs for binary images can be viewed as special cases in the schemes proposed here.

cs.CR↗

arXiv2012

Avian magnetoreception model realized by coupling magnetite-based mechanism with radical-pair-based mechanism

Many animal species were verified to use geomagnetic field for their navigation, but the biophysical mechanism of magnetoreception has remained enigmatic. This paper presents a special biophysical model that consists of magnetite-based and radical-pair-based mechanisms for avian magnetoreception. The amplitude of the resultant magnetic field around the magnetic particles corresponds to the geomagnetic field direction and affects the yield of singlet/triplet state products in the radical-pair reactions. Therefore, in the proposed model, the singlet/triplet state product yields are related to the geomagnetic field information for orientational detection. The resultant magnetic fields corresponding to two materials with different magnetic properties were analyzed under different geomagnetic field directions. The results showed that ferromagnetic particles in organisms can provide more significant changes in singlet state products than superparamagnetic particles, and the period of variation for the singlet state products with an included angle in the geomagnetic field is approximately 180° when the magnetic particles are ferromagnetic materials, consistent with the experimental results obtained from avian magnetic compass. Further, the calculated results of the singlet state products in a reception plane showed that the proposed model can explain the avian magnetoreception mechanism with an inclination compass.

physics.bio-ph↗