arXiv subjects

Linyuan Wang

Publications and source records attributed to Linyuan Wang.

At least 37 records · Page 2Linked to original sources

arXiv2021

Improving the Transferability of Adversarial Examples with New Iteration Framework and Input Dropout

Deep neural networks(DNNs) is vulnerable to be attacked by adversarial examples. Black-box attack is the most threatening attack. At present, black-box attack methods mainly adopt gradient-based iterative attack methods, which usually limit the relationship between the iteration step size, the number of iterations, and the maximum perturbation. In this paper, we propose a new gradient iteration framework, which redefines the relationship between the above three. Under this framework, we easily improve the attack success rate of DI-TI-MIM. In addition, we propose a gradient iterative attack method based on input dropout, which can be well combined with our framework. We further propose a multi dropout rate version of this method. Experimental results show that our best method can achieve attack success rate of 96.2\% for defense model on average, which is higher than the state-of-the-art gradient-based attacks.

cs.LG↗

arXiv2021

Dynamic Defense Approach for Adversarial Robustness in Deep Neural Networks via Stochastic Ensemble Smoothed Model

Deep neural networks have been shown to suffer from critical vulnerabilities under adversarial attacks. This phenomenon stimulated the creation of different attack and defense strategies similar to those adopted in cyberspace security. The dependence of such strategies on attack and defense mechanisms makes the associated algorithms on both sides appear as closely reciprocating processes. The defense strategies are particularly passive in these processes, and enhancing initiative of such strategies can be an effective way to get out of this arms race. Inspired by the dynamic defense approach in cyberspace, this paper builds upon stochastic ensemble smoothing based on defense method of random smoothing and model ensemble. Proposed method employs network architecture and smoothing parameters as ensemble attributes, and dynamically change attribute-based ensemble model before every inference prediction request. The proposed method handles the extreme transferability and vulnerability of ensemble models under white-box attacks. Experimental comparison of ASR-vs-distortion curves with different attack scenarios shows that even the attacker with the highest attack capability cannot easily exceed the attack success rate associated with the ensemble smoothed model, especially under untargeted attacks.

cs.CR↗

arXiv2020

Dual-energy CT Reconstruction from Dual Quarter Scans

Compared with conventional single-energy computed tomography (CT), dual-energy CT (DECT) provides better material differentiation but most DECT imaging systems require dual full-angle projection data at different X-ray spectra. Relaxing the requirement of data acquisition is a particularly attractive research to promote the applications of DECT in a wide range of imaging areas. In this work, we design a novel DECT imaging scheme with dual quarter scans and propose an efficient method to reconstruct the desired DECT images from dual limited-angle projection data, which enables DECT on imaging configurations with half-scan and largely reduces scanning angles and radiation doses. We first study the characteristics of image artifacts under dual quarter scans scheme, and find that the directional limited-angle artifacts of DECT images are complementarily distributed in image domain because the corresponding X-rays of high- and low-energy scans are orthogonal. Inspired by this finding, a fusion CT image is generated by integrating the limited-angle DECT images of dual quarter scans. This strategy largely reduces the limited-angle artifacts and preserves the image edges and inner structures. Utilizing the capability of neural network in the modeling of nonlinear problem, a novel Anchor network with single-entry double-out architecture is designed in this work to yield the desired DECT images from the generated fusion CT image. Experimental results on the simulated and real data verify the effectiveness of the proposed method.

physics.med-ph↗

arXiv2020

RPT: Learning Point Set Representation for Siamese Visual Tracking

While remarkable progress has been made in robust visual tracking, accurate target state estimation still remains a highly challenging problem. In this paper, we argue that this issue is closely related to the prevalent bounding box representation, which provides only a coarse spatial extent of object. Thus an effcient visual tracking framework is proposed to accurately estimate the target state with a finer representation as a set of representative points. The point set is trained to indicate the semantically and geometrically significant positions of target region, enabling more fine-grained localization and modeling of object appearance. We further propose a multi-level aggregation strategy to obtain detailed structure information by fusing hierarchical convolution layers. Extensive experiments on several challenging benchmarks including OTB2015, VOT2018, VOT2019 and GOT-10k demonstrate that our method achieves new state-of-the-art performance while running at over 20 FPS.

cs.CV↗

arXiv2020

Generative Adversarial Network-Based Sinogram Super-Resolution for Computed Tomography Imaging

Compared with the conventional 1*1 acquisition mode of projection in computed tomography (CT) image reconstruction, the 2*2 acquisition mode improves the collection efficiency of the projection and reduces the X-ray exposure time. However, the collected projection based on the 2*2 acquisition mode has low resolution (LR) and the reconstructed image quality is poor, thus limiting the use of this mode in CT imaging systems. In this study, a novel sinogram-super-resolution generative adversarial network (SSR-GAN) model is proposed to obtain high-resolution (HR) sinograms from LR sinograms, thereby improving the reconstruction image quality under the 2*2 acquisition mode. The proposed generator is based on the residual network for LR sinogram feature extraction and super-resolution (SR) sinogram generation. A relativistic discriminator is designed to render the network capable of obtaining more realistic SR sinograms. Moreover, we combine the cycle consistency loss, sinogram domain loss, and reconstruction image domain loss in the total loss function to supervise SR sinogram generation. Then, a trained model can be obtained by inputting the paired LR/HR sinograms into the network. Finally, the classic FBP reconstruction algorithm is used for CT image reconstruction based on the generated SR sinogram. The qualitative and quantitative results of evaluations on digital and real data illustrate that the proposed model not only obtains clean SR sinograms from noisy LR sinograms but also outperforms its counterparts.

physics.med-ph↗

arXiv2020

Dissociable neural representations of adversarially perturbed images in convolutional neural networks and the human brain

Despite the remarkable similarities between convolutional neural networks (CNN) and the human brain, CNNs still fall behind humans in many visual tasks, indicating that there still exist considerable differences between the two systems. Here, we leverage adversarial noise (AN) and adversarial interference (AI) images to quantify the consistency between neural representations and perceptual outcomes in the two systems. Humans can successfully recognize AI images as corresponding categories but perceive AN images as meaningless noise. In contrast, CNNs can correctly recognize AN images but mistakenly classify AI images into wrong categories with surprisingly high confidence. We use functional magnetic resonance imaging to measure brain activity evoked by regular and adversarial images in the human brain, and compare it to the activity of artificial neurons in a prototypical CNN-AlexNet. In the human brain, we find that the representational similarity between regular and adversarial images largely echoes their perceptual similarity in all early visual areas. In AlexNet, however, the neural representations of adversarial images are inconsistent with network outputs in all intermediate processing layers, providing no neural foundations for perceptual similarity. Furthermore, we show that voxel-encoding models trained on regular images can successfully generalize to the neural responses to AI images but not AN images. These remarkable differences between the human brain and AlexNet in the representation-perception relation suggest that future CNNs should emulate both behavior and the internal neural presentations of the human brain.

cs.CV↗

arXiv2020

AdvJND: Generating Adversarial Examples with Just Noticeable Difference

Compared with traditional machine learning models, deep neural networks perform better, especially in image classification tasks. However, they are vulnerable to adversarial examples. Adding small perturbations on examples causes a good-performance model to misclassify the crafted examples, without category differences in the human eyes, and fools deep models successfully. There are two requirements for generating adversarial examples: the attack success rate and image fidelity metrics. Generally, perturbations are increased to ensure the adversarial examples' high attack success rate; however, the adversarial examples obtained have poor concealment. To alleviate the tradeoff between the attack success rate and image fidelity, we propose a method named AdvJND, adding visual model coefficients, just noticeable difference coefficients, in the constraint of a distortion function when generating adversarial examples. In fact, the visual subjective feeling of the human eyes is added as a priori information, which decides the distribution of perturbations, to improve the image quality of adversarial examples. We tested our method on the FashionMNIST, CIFAR10, and MiniImageNet datasets. Adversarial examples generated by our AdvJND algorithm yield gradient distributions that are similar to those of the original inputs. Hence, the crafted noise can be hidden in the original inputs, thus improving the attack concealment significantly.

cs.CV↗

arXiv2020

Neural encoding and interpretation for high-level visual cortices based on fMRI using image caption features

On basis of functional magnetic resonance imaging (fMRI), researchers are devoted to designing visual encoding models to predict the neuron activity of human in response to presented image stimuli and analyze inner mechanism of human visual cortices. Deep network structure composed of hierarchical processing layers forms deep network models by learning features of data on specific task through big dataset. Deep network models have powerful and hierarchical representation of data, and have brought about breakthroughs for visual encoding, while revealing hierarchical structural similarity with the manner of information processing in human visual cortices. However, previous studies almost used image features of those deep network models pre-trained on classification task to construct visual encoding models. Except for deep network structure, the task or corresponding big dataset is also important for deep network models, but neglected by previous studies. Because image classification is a relatively fundamental task, it is difficult to guide deep network models to master high-level semantic representations of data, which causes into that encoding performance for high-level visual cortices is limited. In this study, we introduced one higher-level vision task: image caption (IC) task and proposed the visual encoding model based on IC features (ICFVEM) to encode voxels of high-level visual cortices. Experiment demonstrated that ICFVEM obtained better encoding performance than previous deep network models pre-trained on classification task. In addition, the interpretation of voxels was realized to explore the detailed characteristics of voxels based on the visualization of semantic words, and comparative analysis implied that high-level visual cortices behaved the correlative representation of image content.

cs.CV↗

arXiv2020

BigGAN-based Bayesian reconstruction of natural images from human brain activity

In the visual decoding domain, visually reconstructing presented images given the corresponding human brain activity monitored by functional magnetic resonance imaging (fMRI) is difficult, especially when reconstructing viewed natural images. Visual reconstruction is a conditional image generation on fMRI data and thus generative adversarial network (GAN) for natural image generation is recently introduced for this task. Although GAN-based methods have greatly improved, the fidelity and naturalness of reconstruction are still unsatisfactory due to the small number of fMRI data samples and the instability of GAN training. In this study, we proposed a new GAN-based Bayesian visual reconstruction method (GAN-BVRM) that includes a classifier to decode categories from fMRI data, a pre-trained conditional generator to generate natural images of specified categories, and a set of encoding models and evaluator to evaluate generated images. GAN-BVRM employs the pre-trained generator of the prevailing BigGAN to generate masses of natural images, and selects the images that best matches with the corresponding brain activity through the encoding models as the reconstruction of the image stimuli. In this process, the semantic and detailed contents of reconstruction are controlled by decoded categories and encoding models, respectively. GAN-BVRM used the Bayesian manner to avoid contradiction between naturalness and fidelity from current GAN-based methods and thus can improve the advantages of GAN. Experimental results revealed that GAN-BVRM improves the fidelity and naturalness, that is, the reconstruction is natural and similar to the presented image stimuli.

cs.CV↗

arXiv2019

CNN-based Automatic Detection of Bone Conditions via Diagnostic CT Images for Osteoporosis Screening

Purpose: The purpose is to design a novelty automatic diagnostic method for osteoporosis screening by using the potential capability of convolutional neural network (CNN) in feature representation and extraction, which can be incorporated into the procedure of routine CT diagnostic in physical examination thereby improving the osteoporosis diagnosis and reducing the patient burden. Methods: The proposed convolutional neural network-based method mainly comprises two functional modules to perform automatic detection of bone condition by analyzing the diagnostic CT image. The first functional module aims to locate and segment the ROI of diagnostic CT image, called Mark-Segmentation-Network (MS-Net). The second functional module is used to determine the category of bone condition by the features of ROI, called Bone-Conditions-Classification-Network (BCC-Net). The trained MS-Net can get the mark image of inputted original CT image, thereby obtain the segmentation image. The trained BCC-Net can obtain the probability value of normal bone mass, low bone mass, and osteoporosis by inputting the segmentation image. On the basis of network results, the radiologists can provide preliminary qualitative diagnosis results of bone condition. Results: The proposed MS-Net has an excellent segmentation precision on the shape preservation of different lumbar vertebra. The numerical indicators of segmentation results are greater than 0.998. The proposed BCC-Net was evaluated with 3,024 clinical images and achieved an accuracy of 76.65% and an area under the receiver operating characteristic curve of 0.9167. Conclusions: This study proposed a novel two-step method for automatic detection of bone conditions via diagnostic CT images and it has great potential in clinical applications for osteoporosis screening. The method can potentially reduce the manual burden to radiologists and diagnostic cost to patients.

physics.med-ph↗

arXiv2019

Effective and efficient ROI-wise visual encoding using an end-to-end CNN regression model and selective optimization

Recently, visual encoding based on functional magnetic resonance imaging (fMRI) have realized many achievements with the rapid development of deep network computation. Visual encoding model is aimed at predicting brain activity in response to presented image stimuli. Currently, visual encoding is accomplished mainly by firstly extracting image features through convolutional neural network (CNN) model pre-trained on computer vision task, and secondly training a linear regression model to map specific layer of CNN features to each voxel, namely voxel-wise encoding. However, the two-step manner model, essentially, is hard to determine which kind of well features are well linearly matched for beforehand unknown fMRI data with little understanding of human visual representation. Analogizing computer vision mostly related human vision, we proposed the end-to-end convolution regression model (ETECRM) in the region of interest (ROI)-wise manner to accomplish effective and efficient visual encoding. The end-to-end manner was introduced to make the model automatically learn better matching features to improve encoding performance. The ROI-wise manner was used to improve the encoding efficiency for many voxels. In addition, we designed the selective optimization including self-adapting weight learning and weighted correlation loss, noise regularization to avoid interfering of ineffective voxels in ROI-wise encoding. Experiment demonstrated that the proposed model obtained better predicting accuracy than the two-step manner of encoding models. Comparative analysis implied that end-to-end manner and large volume of fMRI data may drive the future development of visual encoding.

q-bio.NC↗

arXiv2019

Cycle-Consistent Adversarial GAN: the integration of adversarial attack and defense

In image classification of deep learning, adversarial examples where inputs intended to add small magnitude perturbations may mislead deep neural networks (DNNs) to incorrect results, which means DNNs are vulnerable to them. Different attack and defense strategies have been proposed to better research the mechanism of deep learning. However, those research in these networks are only for one aspect, either an attack or a defense, not considering that attacks and defenses should be interdependent and mutually reinforcing, just like the relationship between spears and shields. In this paper, we propose Cycle-Consistent Adversarial GAN (CycleAdvGAN) to generate adversarial examples, which can learn and approximate the distribution of original instances and adversarial examples. For CycleAdvGAN, once the Generator and are trained, can generate adversarial perturbations efficiently for any instance, so as to make DNNs predict wrong, and recovery adversarial examples to clean instances, so as to make DNNs predict correct. We apply CycleAdvGAN under semi-white box and black-box settings on two public datasets MNIST and CIFAR10. Using the extensive experiments, we show that our method has achieved the state-of-the-art adversarial attack method and also efficiently improve the defense ability, which make the integration of adversarial attack and defense come true. In additional, it has improved attack effect only trained on the adversarial dataset generated by any kind of adversarial attack.

cs.CV↗

arXiv2019

Projection Super-resolution Based on Convolutional Neural Network for Computed Tomography

The improvement of computed tomography (CT) image resolution is beneficial to the subsequent medical diagnosis, but it is usually limited by the scanning devices and great expense. Convolutional neural network (CNN)-based methods have achieved promising ability in super-resolution. However, existing methods mainly focus on the super-resolution of reconstructed image and do not fully explored the approach of super-resolution from projection-domain. In this paper, we studied the characteristic of projection and proposed a CNN-based super-resolution method to establish the mapping relationship of low- and high-resolution projection. The network label is high-resolution projection and the input is its corresponding interpolation data after down sampling. FDK algorithm is utilized for three-dimensional image reconstruction and one slice of reconstruction image is taken as an example to evaluate the performance of the proposed method. Qualitative and quantitative results show that the proposed method is potential to improve the resolution of projection and enables the reconstructed image with higher quality.

physics.med-ph↗

arXiv2019

Category decoding of visual stimuli from human brain activity using a bidirectional recurrent neural network to simulate bidirectional information flows in human visual cortices

Recently, visual encoding and decoding based on functional magnetic resonance imaging (fMRI) have realized many achievements with the rapid development of deep network computation. Despite the hierarchically similar representations of deep network and human vision, visual information flows from primary visual cortices to high visual cortices and vice versa based on the bottom-up and top-down manners, respectively. Inspired by the bidirectional information flows, we proposed a bidirectional recurrent neural network (BRNN)-based method to decode the categories from fMRI data. The forward and backward directions in the BRNN module characterized the bottom-up and top-down manners, respectively. The proposed method regarded the selected voxels of each visual cortex region (V1, V2, V3, V4, and LO) as one node in the sequence fed into the BRNN module and combined the output of the BRNN module to decode the categories with the subsequent fully connected layer. This new method allows the efficient utilization of hierarchical information representations and bidirectional information flows in human visual cortices. Experiment results demonstrated that our method improved the accuracy of three-level category decoding than other methods, which implicitly validated the hierarchical and bidirectional human visual representations. Comparative analysis revealed that the category representations of human visual cortices were hierarchical, distributed, complementary, and correlative.

q-bio.NC↗

arXiv2019

A Sinogram Inpainting Method based on Generative Adversarial Network for Limited-angle Computed Tomography

Limited-angle computed tomography (CT) image reconstruction is a challenging reconstruction problem in the fields of CT. With the development of deep learning, the generative adversarial network (GAN) perform well in image restoration by approximating the distribution of training sample data. In this paper, we proposed an effective GAN-based inpainting method to restore the missing sinogram data for limited-angle scanning. To estimate the missing data, we design the generator and discriminator of the patch-GAN and train the network to learn the data distribution of the sinogram. We obtain the reconstructed image from the restored sinogram by filtered back projection and simultaneous algebraic reconstruction technique with total variation. Experimental results show that serious artifacts caused by missing projection data can be reduced by the proposed method, and it is hopeful to solve the reconstruction problem of 60 degrees limited scanning angle.

physics.med-ph↗

arXiv2019

A visual encoding model based on deep neural networks and transfer learning

Background: Building visual encoding models to accurately predict visual responses is a central challenge for current vision-based brain-machine interface techniques. To achieve high prediction accuracy on neural signals, visual encoding models should include precise visual features and appropriate prediction algorithms. Most existing visual encoding models employ hand-craft visual features (e.g., Gabor wavelets or semantic labels) or data-driven features (e.g., features extracted from deep neural networks (DNN)). They also assume a linear mapping between feature representation to brain activity. However, it remains unknown whether such linear mapping is sufficient for maximizing prediction accuracy. New Method: We construct a new visual encoding framework to predict cortical responses in a benchmark functional magnetic resonance imaging (fMRI) dataset. In this framework, we employ the transfer learning technique to incorporate a pre-trained DNN (i.e., AlexNet) and train a nonlinear mapping from visual features to brain activity. This nonlinear mapping replaces the conventional linear mapping and is supposed to improve prediction accuracy on brain activity. Results: The proposed framework can significantly predict responses of over 20% voxels in early visual areas (i.e., V1-lateral occipital region, LO) and achieve unprecedented prediction accuracy. Comparison with Existing Methods: Comparing to two conventional visual encoding models, we find that the proposed encoding model shows consistent higher prediction accuracy in all early visual areas, especially in relatively anterior visual areas (i.e., V4 and LO). Conclusions: Our work proposes a new framework to utilize pre-trained visual features and train non-linear mappings from visual features to brain activity.

cs.CV↗

arXiv2018

Image Domain Dual Material Decomposition for Dual-Energy CT using Butterfly Network

Dual-energy CT (DECT) has been increasingly used in imaging applications because of its capability for material differentiation. However, material decomposition suffers from magnified noise from two CT images of independent scans, leading to severe degradation of image quality. Existing algorithms achieve suboptimal decomposition performance since they fail to accurately depict the mapping relationship between DECT and the basis material images. Inspired by the impressive potential of CNN, we developed a new Butterfly network to perform the image domain dual material decomposition due to its strong approximation ability to the mapping functions in DECT. The Butterfly network is derived from the image domain DECT decomposition model by exploring the geometric relationship between mapping functions of data model and network components. The network is designed as the double-entry double-out crossover architecture based on the decomposition formulation. It enters a pair of dual-energy images as inputs and defines the ground true decomposed images as each label. The crossover architecture, which plays an important role in material decomposition, is designed to implement the information exchange between the two material generation pathways in the network. Network components exhibit different sensitivity to basis materials in the visualization. By analyzing their sensitivity to different basis materials, we determined the roles of network components in material decomposition. This visualization evaluation reveals what the network can learn and verifies the rationality of network design. The qualitative and quantitative evaluations in material decomposition of patient data indicate that the proposed network outperforms its counterpart.

physics.med-ph↗

arXiv2018

Constraint-free Natural Image Reconstruction from fMRI Signals Based on Convolutional Neural Network

In recent years, research on decoding brain activity based on functional magnetic resonance imaging (fMRI) has made remarkable achievements. However, constraint-free natural image reconstruction from brain activity is still a challenge. The existing methods simplified the problem by using semantic prior information or just reconstructing simple images such as letters and digitals. Without semantic prior information, we present a novel method to reconstruct nature images from fMRI signals of human visual cortex based on the computation model of convolutional neural network (CNN). Firstly, we extracted the units output of viewed natural images in each layer of a pre-trained CNN as CNN features. Secondly, we transformed image reconstruction from fMRI signals into the problem of CNN feature visualizations by training a sparse linear regression to map from the fMRI patterns to CNN features. By iteratively optimization to find the matched image, whose CNN unit features become most similar to those predicted from the brain activity, we finally achieved the promising results for the challenging constraint-free natural image reconstruction. As there was no use of semantic prior information of the stimuli when training decoding model, any category of images (not constraint by the training set) could be reconstructed theoretically. We found that the reconstructed images resembled the natural stimuli, especially in position and shape. The experimental results suggest that hierarchical visual features can effectively express the visual perception process of human brain.

cs.CV↗