arXiv subjects

Dawei Li

Publications and source records attributed to Dawei Li.

108 records · Page 6Linked to original sources

arXiv2019

RILOD: Near Real-Time Incremental Learning for Object Detection at the Edge

Object detection models shipped with camera-equipped edge devices cannot cover the objects of interest for every user. Therefore, the incremental learning capability is a critical feature for a robust and personalized object detection system that many applications would rely on. In this paper, we present an efficient yet practical system, RILOD, to incrementally train an existing object detection model such that it can detect new object classes without losing its capability to detect old classes. The key component of RILOD is a novel incremental learning algorithm that trains end-to-end for one-stage deep object detection models only using training data of new object classes. Specifically to avoid catastrophic forgetting, the algorithm distills three types of knowledge from the old model to mimic the old model's behavior on object classification, bounding box regression and feature extraction. In addition, since the training data for the new classes may not be available, a real-time dataset construction pipeline is designed to collect training images on-the-fly and automatically label the images with both category and bounding box annotations. We have implemented RILOD under both edge-cloud and edge-only setups. Experiment results show that the proposed system can learn to detect a new object class in just a few minutes, including both dataset construction and model training. In comparison, traditional fine-tuning based method may take a few hours for training, and in most cases would also need a tedious and costly manual dataset labeling step.

cs.CV↗

arXiv2019

Customized OCT images compression scheme with deep neural network

We customize an end-to-end image compression framework for retina OCT images based on deep convolutional neural networks (CNNs). The customized compression scheme consists of three parts: data Preprocessing, compression CNNs, and reconstruction CNNs. Data preprocessing module reduces the speckle noise of the OCT images and the segments out the region of interest. We added customized skip connections between the compression CNNs and the reconstruction CNNs to reserve the detail information and trained the two nets together with the semantic segmented image patches from data preprocessing module. To train the two networks sensitive to both low frequency information and high frequency information, we adopted an objective function with two parts: A PatchGAN discriminator to judge the high frequency information and a differentiable MS-SSIM penalty to evaluate the low frequency information. The proposed framework was trained and evaluated on a publicly available OCT dataset. The evaluation showed above 99% similarity in terms of multi-scale structural similarity (MS-SSIM) when the compression ratio is as high as 40. Furthermore, the reconstructed images of compression ratio 80 from the proposed framework even have better quality than that of compression ratio 20 from JPEG by visual comparison. The testing result outperforms JPEG in term of both of MS-SSIM and visualization, which is more obvious as the increase of compression ratio. Our preliminary result indicates the huge potential of deep neural networks on customized medical image compression.

eess.IV↗

arXiv2019

Inverse Structural Design of Graphene/Boron Nitride Hybrids by Regressional GAN

Inverse design of materials with desired properties is currently laborious and heavily relies on intuition of researchers through a trial-and-error process. The massive combinational spaces due to the constituent elements and their structural configurations are too overwhelming to be all searched even by high-throughput computations. Herein, we demonstrated a novel regressional generative adversarial network (RGAN) for inverse design of representative two-dimensional materials, graphene and boron-nitride (BN) hybrids. A significant novelty of the proposed RGAN is that it combines the supervised and regressional convolutional neural network (CNN) with the traditional unsupervised GAN, thus overcoming the common technical barrier in the traditional GANs, which cannot generate data associated with given continuous quantitative labels. The proposed RGAN enables to autonomously generate graphene/BN hybrids with any given bandgaps. Moreover, the generated structures exhibit high fidelity, yielding bandgaps within ~ 10% MAEF of the desired bandgaps as cross-validated by density functional theory (DFT) calculations. Further analysis by principle component analysis (PCA) and modified locally linear embedding (MLLE) methods on the latent features encoded by the regressor reveals that the generator has successfully generated structures that followed the statistical distribution of the real structures. It implies the possibility of the RGAN in recognizing physical rules hidden in the high-dimensional data. This new inverse design methodology would speed up the discovery and development of other 2D materials and beyond.

physics.comp-ph↗

arXiv2019

An overlapping-free leaf segmentation method for plant point clouds

Automatic leaf segmentation, as well as identification and classification methods that built upon it, are able to provide immediate monitoring for plant growth status to guarantee the output. Although 3D plant point clouds contain abundant phenotypic features, plant leaves are usually distributed in clusters and are sometimes seriously overlapped in the canopy. Therefore, it is still a big challenge to automatically segment each individual leaf from a highly crowded plant canopy in 3D for plant phenotyping purposes. In this work, we propose an overlapping-free individual leaf segmentation method for plant point clouds using the 3D filtering and facet region growing. In order to separate leaves with different overlapping situations, we develop a new 3D joint filtering operator, which integrates a Radius-based Outlier Filter (RBOF) and a Surface Boundary Filter (SBF) to help to separate occluded leaves. By introducing the facet over-segmentation and facet-based region growing, the noise in segmentation is suppressed and labeled leaf centers can expand to their whole leaves, respectively. Our method can work on point clouds generated from three types of 3D imaging platforms, and also suitable for different kinds of plant species. In experiments, it obtains a point-level cover rate of 97% for Epipremnum aureum, 99% for Monstera deliciosa, 99% for Calathea makoyana, and 87% for Hedera nepalensis sample plants. At the leaf level, our method reaches an average Recall at 100.00%, a Precision at 99.33%, and an average F-measure at 99.66%, respectively. The proposed method can also facilitate the automatic traits estimation of each single leaf (such as the leaf area, length, and width), which has potential to become a highly effective tool for plant research and agricultural engineering.

cs.CV↗

arXiv2019

Talking Face Generation by Conditional Recurrent Adversarial Network

Given an arbitrary face image and an arbitrary speech clip, the proposed work attempts to generating the talking face video with accurate lip synchronization while maintaining smooth transition of both lip and facial movement over the entire video clip. Existing works either do not consider temporal dependency on face images across different video frames thus easily yielding noticeable/abrupt facial and lip movement or are only limited to the generation of talking face video for a specific person thus lacking generalization capacity. We propose a novel conditional video generation network where the audio input is treated as a condition for the recurrent adversarial network such that temporal dependency is incorporated to realize smooth transition for the lip and facial movement. In addition, we deploy a multi-task adversarial training scheme in the context of video generation to improve both photo-realism and the accuracy for lip synchronization. Finally, based on the phoneme distribution information extracted from the audio clip, we develop a sample selection method that effectively reduces the size of the training dataset without sacrificing the quality of the generated video. Extensive experiments on both controlled and uncontrolled datasets demonstrate the superiority of the proposed approach in terms of visual quality, lip sync accuracy, and smooth transition of lip and facial movement, as compared to the state-of-the-art.

cs.CV↗

arXiv2019

An Integrated Image Filter for Enhancing Change Detection Results

Change detection is a fundamental task in computer vision. Despite significant advances have been made, most of the change detection methods fail to work well in challenging scenes due to ubiquitous noise and interferences. Nowadays, post-processing methods (e.g. MRF, and CRF) aiming to enhance the binary change detection results still fall short of the requirements on universality for distinctive scenes, applicability for different types of detection methods, accuracy, and real-time performance. Inspired by the nature of image filtering, which separates noise from pixel observations and recovers the real structure of patches, we consider utilizing image filters to enhance the detection masks. In this paper, we present an integrated filter which comprises a weighted local guided image filter and a weighted spatiotemporal tree filter. The spatiotemporal tree filter leverages the global spatiotemporal information of adjacent video frames and meanwhile the guided filter carries out local window filtering of pixels, for enhancing the coarse change detection masks. The main contributions are three: (i) the proposed filter can make full use of the information of the same object in consecutive frames to improve its current detection mask by computations on a spatiotemporal minimum spanning tree; (ii) the integrated filter possesses both advantages of local filtering and global filtering; it not only has good edge-preserving property but also can handle heavily textured and colorful foreground regions; and (iii) Unlike some popular enhancement methods (MRF, and CRF) that require either a priori background probabilities or a posteriori foreground probabilities for every pixel to improve the coarse detection masks, our method is a versatile enhancement filter that can be applied after many different types of change detection methods, and is particularly suitable for video sequences.

cs.CV↗

arXiv2019

Regularize, Expand and Compress: Multi-task based Lifelong Learning via NonExpansive AutoML

Lifelong learning, the problem of continual learning where tasks arrive in sequence, has been lately attracting more attention in the computer vision community. The aim of lifelong learning is to develop a system that can learn new tasks while maintaining the performance on the previously learned tasks. However, there are two obstacles for lifelong learning of deep neural networks: catastrophic forgetting and capacity limitation. To solve the above issues, inspired by the recent breakthroughs in automatically learning good neural network architectures, we develop a Multi-task based lifelong learning via nonexpansive AutoML framework termed Regularize, Expand and Compress (REC). REC is composed of three stages: 1) continually learns the sequential tasks without the learned tasks' data via a newly proposed multi-task weight consolidation (MWC) algorithm; 2) expands the network to help the lifelong learning with potentially improved model capability and performance by network-transformation based AutoML; 3) compresses the expanded model after learning every new task to maintain model efficiency and performance. The proposed MWC and REC algorithms achieve superior performance over other lifelong learning algorithms on four different datasets.

cs.CV↗

arXiv2019

MICIK: MIning Cross-Layer Inherent Similarity Knowledge for Deep Model Compression

State-of-the-art deep model compression methods exploit the low-rank approximation and sparsity pruning to remove redundant parameters from a learned hidden layer. However, they process each hidden layer individually while neglecting the common components across layers, and thus are not able to fully exploit the potential redundancy space for compression. To solve the above problem and enable further compression of a model, removing the cross-layer redundancy and mining the layer-wise inheritance knowledge is necessary. In this paper, we introduce a holistic model compression framework, namely MIning Cross-layer Inherent similarity Knowledge (MICIK), to fully excavate the potential redundancy space. The proposed MICIK framework simultaneously, (1) learns the common and unique weight components across deep neural network layers to increase compression rate; (2) preserves the inherent similarity knowledge of nearby layers and distant layers to minimize the accuracy loss and (3) can be complementary to other existing compression techniques such as knowledge distillation. Extensive experiments on large-scale convolutional neural networks demonstrate that MICIK is superior over state-of-the-art model compression approaches with 16X parameter reduction on VGG-16 and 6X on GoogLeNet, all without accuracy loss.

cs.CV↗

arXiv2018

Statistical Data Assimilation: Formulation and Examples from Neurobiology

For the Research Topic Data Assimilation and Control: Theory and Applications in Life Sciences we first review the formulation of statistical data assimilation (SDA) and discuss algorithms for exploring variational approximations to the conditional expected values of biophysical aspects of functional neural circuits. Then we report on the application of SDA to (1) the exploration of properties of individual neurons in the HVC nucleus of the avian song system, and (2) characterizing individual neurons formulated as very large scale integration (VLSI) analog circuits with a goal of building functional, biophysically realistic, VLSI representations of functional nervous systems. Networks of neurons pose a substantially greater challenge, and we comment on formulating experiments to probe the properties, especially the functional connectivity, in song command circuits within HVC.

q-bio.NC↗

arXiv2018

Dynamically Hierarchy Revolution: DirNet for Compressing Recurrent Neural Network on Mobile Devices

Recurrent neural networks (RNNs) achieve cutting-edge performance on a variety of problems. However, due to their high computational and memory demands, deploying RNNs on resource constrained mobile devices is a challenging task. To guarantee minimum accuracy loss with higher compression rate and driven by the mobile resource requirement, we introduce a novel model compression approach DirNet based on an optimized fast dictionary learning algorithm, which 1) dynamically mines the dictionary atoms of the projection dictionary matrix within layer to adjust the compression rate 2) adaptively changes the sparsity of sparse codes cross the hierarchical layers. Experimental results on language model and an ASR model trained with a 1000h speech dataset demonstrate that our method significantly outperforms prior approaches. Evaluated on off-the-shelf mobile devices, we are able to reduce the size of original model by eight times with real-time model inference and negligible accuracy loss.

cs.LG↗

arXiv2018

Boundary-sensitive Network for Portrait Segmentation

Compared to the general semantic segmentation problem, portrait segmentation has higher precision requirement on boundary area. However, this problem has not been well studied in previous works. In this paper, we propose a boundary-sensitive deep neural network (BSN) for portrait segmentation. BSN introduces three novel techniques. First, an individual boundary-sensitive kernel is proposed by dilating the contour line and assigning the boundary pixels with multi-class labels. Second, a global boundary-sensitive kernel is employed as a position sensitive prior to further constrain the overall shape of the segmentation map. Third, we train a boundary-sensitive attribute classifier jointly with the segmentation network to reinforce the network with semantic boundary shape information. We have evaluated BSN on the current largest public portrait segmentation dataset, i.e, the PFCN dataset, as well as the portrait images collected from other three popular image segmentation datasets: COCO, COCO-Stuff, and PASCAL VOC. Our method achieves the superior quantitative and qualitative performance over state-of-the-arts on all the datasets, especially on the boundary area.

cs.CV↗

arXiv2018

LCANet: End-to-End Lipreading with Cascaded Attention-CTC

Machine lipreading is a special type of automatic speech recognition (ASR) which transcribes human speech by visually interpreting the movement of related face regions including lips, face, and tongue. Recently, deep neural network based lipreading methods show great potential and have exceeded the accuracy of experienced human lipreaders in some benchmark datasets. However, lipreading is still far from being solved, and existing methods tend to have high error rates on the wild data. In this paper, we propose LCANet, an end-to-end deep neural network based lipreading system. LCANet encodes input video frames using a stacked 3D convolutional neural network (CNN), highway network and bidirectional GRU network. The encoder effectively captures both short-term and long-term spatio-temporal information. More importantly, LCANet incorporates a cascaded attention-CTC decoder to generate output texts. By cascading CTC with attention, it partially eliminates the defect of the conditional independence assumption of CTC within the hidden neural layers, and this yields notably performance improvement as well as faster convergence. The experimental results show the proposed system achieves a 1.3% CER and 3.0% WER on the GRID corpus database, leading to a 12.3% improvement compared to the state-of-the-art methods.

cs.CV↗

arXiv2018

DeepRebirth: Accelerating Deep Neural Network Execution on Mobile Devices

Deploying deep neural networks on mobile devices is a challenging task. Current model compression methods such as matrix decomposition effectively reduce the deployed model size, but still cannot satisfy real-time processing requirement. This paper first discovers that the major obstacle is the excessive execution time of non-tensor layers such as pooling and normalization without tensor-like trainable parameters. This motivates us to design a novel acceleration framework: DeepRebirth through "slimming" existing consecutive and parallel non-tensor and tensor layers. The layer slimming is executed at different substructures: (a) streamline slimming by merging the consecutive non-tensor and tensor layer vertically; (b) branch slimming by merging non-tensor and tensor branches horizontally. The proposed optimization operations significantly accelerate the model execution and also greatly reduce the run-time memory cost since the slimmed model architecture contains less hidden layers. To maximally avoid accuracy loss, the parameters in new generated layers are learned with layer-wise fine-tuning based on both theoretical analysis and empirical verification. As observed in the experiment, DeepRebirth achieves more than 3x speed-up and 2.5x run-time memory saving on GoogLeNet with only 0.4% drop of top-5 accuracy on ImageNet. Furthermore, by combining with other model compression techniques, DeepRebirth offers an average of 65ms inference time on the CPU of Samsung Galaxy S6 with 86.5% top-5 accuracy, 14% faster than SqueezeNet which only has a top-5 accuracy of 80.5%.

cs.CV↗

arXiv2017

Large-scale pedestrian flow experiments under high-density conditions

Despite the vast amount of studies on pedestrian flow, the data concerning high densities are still very inadequate. We organize one large-scale pedestrian flow experiment on a ring corridor. With 278 participants, the density as high as 9 m^(-2) is reached. In the uni-directional flow, four different states are observed, including the free flow, congested state, over-congested state and hyper-congested state. The features of the hyper-congested state are similar to the "crowd turbulence" reported in the empirical data of Helbing et al., and the transition between the stopped state and the moving state can be found. The flow rates in the over-congested state are nearly constant, due to the downstream propagation of pedestrian clusters. In the bi-directional flow, three different types of lane formations are observed in the experiment: (1) three lanes are directly formed ; (2) two lanes are directly formed; (3) firstly three lanes are formed, and then they transit into two lanes. After the lane formation, some interesting phenomena have been observed, including the inhomogeneous distribution of pedestrians across the lanes, and the formation and dissipation of localized crowd. Our study is expected to help for better understanding and modeling the dynamics of high density pedestrian flow.

physics.soc-ph↗

arXiv2017

Performance Study of the First 2D Prototype of Vertically Integrated Pattern Recognition Associative Memory (VIPRAM)

Extremely fast pattern recognition capabilities are necessary to find and fit billions of tracks at the hardware trigger level produced every second anticipated at high luminosity LHC (HL-LHC) running conditions. Associative Memory (AM) based approaches for fast pattern recognition have been proposed as a potential solution to the tracking trigger. However, at the HL-LHC, there is much less time available and speed performance must be improved over previous systems while maintaining a comparable number of patterns. The Vertically Integrated Pattern Recognition Associative Memory (VIPRAM) Project aims to achieve the target pattern density and performance goal using 3DIC technology. The first step taken in the VIPRAM work was the development of a 2D prototype (protoVIPRAM00) in which the associative memory building blocks were designed to be compatible with the 3D integration. In this paper, we present the results from extensive performance studies of the protoVIPRAM00 chip in both realistic HL-LHC and extreme conditions. Results indicate that the chip operates at the design frequency of 100 MHz with perfect correctness in realistic conditions and conclude that the building blocks are ready for 3D stacking. We also present performance boundary characterization of the chip under extreme conditions.

physics.ins-det↗

arXiv2016

Evaluation at the Remote Site for Ultra-stable Radio Frequency Dissemination via Fiber Links

We demonstrate a method which can directly evaluate the radio frequency transfer quality via fiber links at the remote site. Coherent signals are first transferred to the same remote site via two stabilized fiber links. The two signals at the remote site are compared with each other. The relative phase difference can represent transfer stability loss. This evaluation method at the remote site has been compared with the traditional one with which the signal is evaluated at the local site. The two results match perfectly. It indicates that the method is available to evaluate the transfer performance of radio frequency (RF) dissemination in such applications as antenna array systems.

physics.ins-det↗

arXiv2016

Efficient laser noise reduction by locking to an actively stabilized fiber interferometer with 10 km arm imbalance

We report a laser noise reduction method by locking it to an actively stabilized fiber-based Mach Zehnder interferometer with 10 km optical fiber to achieve large arm imbalance. An acousto optic modulator is used for interferometer stabilization and heterodyne detection. The out-of-loop frequency noise is reduced by more than 90 dB for Fourier frequency at 1 Hz. This structure presents an efficient laser noise reduction method both at high Fourier frequency and low Fourier frequency. The signal of stabilized laser is transferred via a 10 km fiber link with a fractional frequency stability of 1.12 times 10-16 at 1 s. Compared with the fractional frequency stability of that when the interferometer is not stabilized, more than one order of magnitude is improved.

physics.optics↗

arXiv2016

Sensitivity of the CUORE detector to $14.4$ keV solar axions emitted by the M1 nuclear transition of$~^{57}$Fe

In this paper we present a calculation of the sensitivity of the CUORE detector to the monoenergetic $14.4$ keV solar axions emitted by the M1 nuclear transition of$~^{57}$Fe in the Sun and detected by inverse coherent Bragg-Primakoff conversion in single-crystal $TeO_2$ bolometers. The expected counting rate is calculated using density functional theory for the electron charge density of $TeO_2$ and realistic background and energy resolution of CUORE. Monte Carlo simulations for $5$ y $\times$ $741$ kg=$3705-$kg$\cdot$y of exposure are analyzed using time correlation of individual events with the theoretical time-dependent counting rate. We find an expected model-independent limit on the product of the axion-photon coupling and the axion-nucleon coupling $g_{aγγ}g_{aN}^{\text{eff}}<1.105\times 10^{-16}$ /GeV for axion masses less than 500 eV with $95\%$ confidence level.

astro-ph.CO↗