arXiv subjects

Wei Wu

Publications and source records attributed to Wei Wu.

At least 469 records · Page 26Linked to original sources

arXiv2019

IRLAS: Inverse Reinforcement Learning for Architecture Search

In this paper, we propose an inverse reinforcement learning method for architecture search (IRLAS), which trains an agent to learn to search network structures that are topologically inspired by human-designed network. Most existing architecture search approaches totally neglect the topological characteristics of architectures, which results in complicated architecture with a high inference latency. Motivated by the fact that human-designed networks are elegant in topology with a fast inference speed, we propose a mirror stimuli function inspired by biological cognition theory to extract the abstract topological knowledge of an expert human-design network (ResNeXt). To avoid raising a too strong prior over the search space, we introduce inverse reinforcement learning to train the mirror stimuli function and exploit it as a heuristic guidance for architecture search, easily generalized to different architecture search algorithms. On CIFAR-10, the best architecture searched by our proposed IRLAS achieves 2.60% error rate. For ImageNet mobile setting, our model achieves a state-of-the-art top-1 accuracy 75.28%, while being 2~4x faster than most auto-generated architectures. A fast version of this model achieves 10% faster than MobileNetV2, while maintaining a higher accuracy.

cs.CV↗

arXiv2019

Energy-Efficient Resource Allocation for Secure NOMA-Enabled Mobile Edge Computing Networks

Mobile edge computing (MEC) has been envisaged as a promising technique in the next-generation wireless networks. In order to improve the security of computation tasks offloading and enhance user connectivity, physical layer security and non-orthogonal multiple access (NOMA) are studied in MEC-aware networks. The secrecy outage probability is adopted to measure the secrecy performance of computation offloading by considering a practically passive eavesdropping scenario. The weighted sum-energy consumption minimization problem is firstly investigated subject to the secrecy offloading rate constraints, the computation latency constraints and the secrecy outage probability constraints. The semi-closed form expression for the optimal solution is derived. We then investigate the secrecy outage probability minimization problem by taking the priority of two users into account, and characterize the optimal secrecy offloading rates and power allocations with closed-form expressions. Numerical results demonstrate that the performance of our proposed design are better than those of the alternative benchmark schemes.

cs.IT↗

arXiv2019

Improving One-shot NAS by Suppressing the Posterior Fading

There is a growing interest in automated neural architecture search (NAS). To improve the efficiency of NAS, previous approaches adopt weight sharing method to force all models share the same set of weights. However, it has been observed that a model performing better with shared weights does not necessarily perform better when trained alone. In this paper, we analyse existing weight sharing one-shot NAS approaches from a Bayesian point of view and identify the posterior fading problem, which compromises the effectiveness of shared weights. To alleviate this problem, we present a practical approach to guide the parameter posterior towards its true distribution. Moreover, a hard latency constraint is introduced during the search so that the desired latency can be achieved. The resulted method, namely Posterior Convergent NAS (PC-NAS), achieves state-of-the-art performance under standard GPU latency constraint on ImageNet. In our small search space, our model PC-NAS-S attains 76.8 % top-1 accuracy, 2.1% higher than MobileNetV2 (1.4x) with the same latency. When adopted to the large search space, PC-NAS-L achieves 78.1 % top-1 accuracy within 11ms. The discovered architecture also transfers well to other computer vision applications such as object detection and person re-identification.

cs.CV↗

arXiv2019

A Deep Learning System That Generates Quantitative CT Reports for Diagnosing Pulmonary Tuberculosis

We developed a deep learning model-based system to automatically generate a quantitative Computed Tomography (CT) diagnostic report for Pulmonary Tuberculosis (PTB) cases.501 CT imaging datasets from 223 patients with active PTB were collected, and another 501 cases from a healthy population served as negative samples.2884 lesions of PTB were carefully labeled and classified manually by professional radiologists.Three state-of-the-art 3D convolution neural network (CNN) models were trained and evaluated in the inspection of PTB CT images. Transfer learning method was also utilized during this process. The best model was selected to annotate the spatial location of lesions and classify them into miliary, infiltrative, caseous, tuberculoma and cavitary types simultaneously.Then the Noisy-Or Bayesian function was used to generate an overall infection probability.Finally, a quantitative diagnostic report was exported.The results showed that the recall and precision rates, from the perspective of a single lesion region of PTB, were 85.9% and 89.2% respectively. The overall recall and precision rates,from the perspective of one PTB case, were 98.7% and 93.7%, respectively. Moreover, the precision rate of the PTB lesion type classification was 90.9%.The new method might serve as an effective reference for decision making by clinical doctors.

eess.IV↗

arXiv2019

Explaining a black-box using Deep Variational Information Bottleneck Approach

Interpretable machine learning has gained much attention recently. Briefness and comprehensiveness are necessary in order to provide a large amount of information concisely when explaining a black-box decision system. However, existing interpretable machine learning methods fail to consider briefness and comprehensiveness simultaneously, leading to redundant explanations. We propose the variational information bottleneck for interpretation, VIBI, a system-agnostic interpretable method that provides a brief but comprehensive explanation. VIBI adopts an information theoretic principle, information bottleneck principle, as a criterion for finding such explanations. For each instance, VIBI selects key features that are maximally compressed about an input (briefness), and informative about a decision made by a black-box system on that input (comprehensive). We evaluate VIBI on three datasets and compare with state-of-the-art interpretable machine learning methods in terms of both interpretability and fidelity evaluated by human and quantitative metrics

cs.LG↗

arXiv2019

Read, Attend and Comment: A Deep Architecture for Automatic News Comment Generation

Automatic news comment generation is a new testbed for techniques of natural language generation. In this paper, we propose a "read-attend-comment" procedure for news comment generation and formalize the procedure with a reading network and a generation network. The reading network comprehends a news article and distills some important points from it, then the generation network creates a comment by attending to the extracted discrete points and the news title. We optimize the model in an end-to-end manner by maximizing a variational lower bound of the true objective using the back-propagation algorithm. Experimental results on two datasets indicate that our model can significantly outperform existing methods in terms of both automatic evaluation and human judgment.

cs.CL↗

arXiv2019

$C^2$ regularity of the surface tension for the $\nabla ϕ$ interface model

We consider the $\nabla ϕ$ interface model with a uniformly convex interaction potential possessing Hölder continuous second derivatives. Combining ideas of Naddaf and Spencer with methods from quantitative homogenization, we show that the surface tension (or free energy) associated to the model is at least $C^{2,β}$ for some $β>0$. We also prove a fluctuation-dissipation relation by identifying its Hessian with the covariance matrix characterizing the scaling limit of the model. Finally, we obtain a quantitative rate of convergence for the Hessian of the finite-volume surface tension to that of its infinite-volume limit.

math-ph↗

arXiv2019

DSReg: Using Distant Supervision as a Regularizer

In this paper, we aim at tackling a general issue in NLP tasks where some of the negative examples are highly similar to the positive examples, i.e., hard-negative examples. We propose the distant supervision as a regularizer (DSReg) approach to tackle this issue. The original task is converted to a multi-task learning problem, in which distant supervision is used to retrieve hard-negative examples. The obtained hard-negative examples are then used as a regularizer. The original target objective of distinguishing positive examples from negative examples is jointly optimized with the auxiliary task objective of distinguishing softened positive (i.e., hard-negative examples plus positive examples) from easy-negative examples. In the neural context, this can be done by outputting the same representation from the last neural layer to different $softmax$ functions. Using this strategy, we can improve the performance of baseline models in a range of different NLP tasks, including text classification, sequence labeling and reading comprehension.

cs.CL↗

arXiv2019

Model-based Statistical Depth with Applications to Functional Data

Statistical depth, a commonly used analytic tool in non-parametric statistics, has been extensively studied for multivariate and functional observations over the past few decades. Although various forms of depth were introduced, they are mainly procedure-based whose definitions are independent of the generative model for observations. To address this problem, we introduce a generative model-based approach to define statistical depth for both multivariate and functional data. The proposed model-based depth framework permits simple computation via Monte Carlo sampling and improves the depth estimation accuracy. When applied to functional data, the proposed depth can capture important features such as continuity, smoothness, or phase variability, depending on the defining criteria. Specifically, we view functional data as realizations from a second-order stochastic process, and define their depths through the eigensystem of the covariance operator. These new definitions are given through a proper metric related to the reproducing kernel Hilbert space of the covariance operator. We propose efficient algorithms to compute the proposed depths and establish estimation consistency. Through simulations and real data, we demonstrate that the proposed functional depths reveal important statistical information such as those captured by the median and quantiles, and detect outliers.

stat.ME↗

arXiv2019

Intensity Estimation for Poisson Process with Compositional Noise

Intensity estimation for Poisson processes is a classical problem and has been extensively studied over the past few decades. Practical observations, however, often contain compositional noise, i.e. a nonlinear shift along the time axis, which makes standard methods not directly applicable. The key challenge is that these observations are not "aligned", and registration procedures are required for successful estimation. In this paper, we propose an alignment-based framework for positive intensity estimation. We first show that the intensity function is area-preserved with respect to compositional noise. Such a property implies that the time warping is only encoded in the normalized intensity, or density, function. Then, we decompose the estimation of the intensity by the product of the estimated total intensity and estimated density. The estimation of the density relies on a metric which measures the phase difference between two density functions. An asymptotic study shows that the proposed estimation algorithm provides a consistent estimator for the normalized intensity. We then extend the framework to estimating non-negative intensity functions. The success of the proposed estimation algorithms is illustrated using two simulations. Finally, we apply the new framework in a real data set of neural spike trains, and find that the newly estimated intensities provide better classification accuracy than previous methods.

stat.ME↗

arXiv2019

Dirichlet Depths for Point Process

Statistical depths have been well studied for multivariate and functional data over the past few decades, but remain under-explored for point processes. A first attempt on the notion of point process depth was conducted recently where the depth was defined as a weighted product of two terms: (1) the probability of the number of events in each process and (2) the depth of the event times conditioned on the number of events by using a Mahalanobis depth. We point out that multivariate depths such as the Mahalanobis depth cannot be directly used because they often neglect the important ordered property in the point process events. To deal with this problem, we propose a model-based approach for point processes systematically. In particular, we develop a Dirichlet-distribution-based framework on the conditional depth term, where the new methods are referred to as Dirichlet depths. We examine the mathematical properties of the new depths and conduct the asymptotic analysis. In addition, we illustrate the new methods using various simulated and real experiment data. It is found that the proposed framework provides a proper center-outward rank and the new methods have superior decoding performance to previous methods in two neural spike train datasets.

stat.ME↗

arXiv2019

Robust Multiple Kernel k-means Clustering using Min-Max Optimization

Multiple kernel learning is a type of multiview learning that combines different data modalities by capturing view-specific patterns using kernels. Although supervised multiple kernel learning has been extensively studied, until recently, only a few unsupervised approaches have been proposed. In the meanwhile, adversarial learning has recently received much attention. Many works have been proposed to defend against adversarial examples. However, little is known about the effect of adversarial perturbation in the context of multiview learning, and even less in the unsupervised case. In this study, we show that adversarial features added to a view can make the existing approaches with the min-max formulation in multiple kernel clustering yield unfavorable clusters. To address this problem and inspired by recent works in adversarial learning, we propose a multiple kernel clustering method with the min-max framework that aims to be robust to such adversarial perturbation. We evaluate the robustness of our method on simulation data under different types of adversarial perturbations and show that it outperforms several compared existing methods. In the real data analysis, We demonstrate the utility of our method on a real-world problem.

cs.LG↗

arXiv2019

Versatile surface ion trap for effective cooling and large-scale trapping of ions

Scaling up and effective cooling of ions in surface ion trap are central challenges in quantum computing and quantum simulation with trapped ions. In this theoretical study, we propose a versatile surface ion trap. In the manipulation zone of our trap, a symmetric seven-wire geometry enables innate principle-axes rotation of two parallel linear ion chains, which facilitates the cooling of ions along all principle trap axes. To alleviate contaminating the manipulation zone during ion loading, a symmetric five-wire geometry is designed as the loading zone. And a "fork junction" connects the loading and manipulation zones, which also enables the shuttling and reordering of ions. A multi-objective optimization procedure suitable for arbitrary junction designs is described in detail, and we present the corresponding optimal results for the key components of our trap. The proposed versatile trap can be used in the construction of large-scale ion quantum processors. The trap also can be used as the multi-ion-mixer or the efficient ion beam splitter, which has the potential applications in quantum simulation and quantum computing, the research of 2D dimensional ion crystals and the guides of quantum microscope, like an electron beam splitter used for quantum matter-wave optics experiments. Interesting topics involving the spin-spin interactions between two ion chains can also be simulated in our trap.

physics.atom-ph↗

arXiv2019

STM: SpatioTemporal and Motion Encoding for Action Recognition

Spatiotemporal and motion features are two complementary and crucial information for video action recognition. Recent state-of-the-art methods adopt a 3D CNN stream to learn spatiotemporal features and another flow stream to learn motion features. In this work, we aim to efficiently encode these two features in a unified 2D framework. To this end, we first propose an STM block, which contains a Channel-wise SpatioTemporal Module (CSTM) to present the spatiotemporal features and a Channel-wise Motion Module (CMM) to efficiently encode motion features. We then replace original residual blocks in the ResNet architecture with STM blcoks to form a simple yet effective STM network by introducing very limited extra computation cost. Extensive experiments demonstrate that the proposed STM network outperforms the state-of-the-art methods on both temporal-related datasets (i.e., Something-Something v1 & v2 and Jester) and scene-related datasets (i.e., Kinetics-400, UCF-101, and HMDB-51) with the help of encoding spatiotemporal and motion features together.

cs.CV↗

arXiv2019

Dynamic Curriculum Learning for Imbalanced Data Classification

Human attribute analysis is a challenging task in the field of computer vision, since the data is largely imbalance-distributed. Common techniques such as re-sampling and cost-sensitive learning require prior-knowledge to train the system. To address this problem, we propose a unified framework called Dynamic Curriculum Learning (DCL) to online adaptively adjust the sampling strategy and loss learning in single batch, which resulting in better generalization and discrimination. Inspired by the curriculum learning, DCL consists of two level curriculum schedulers: (1) sampling scheduler not only manages the data distribution from imbalanced to balanced but also from easy to hard; (2) loss scheduler controls the learning importance between classification and metric learning loss. Learning from these two schedulers, we demonstrate our DCL framework with the new state-of-the-art performance on the widely used face attribute dataset CelebA and pedestrian attribute dataset RAP.

cs.CV↗

arXiv2019

Online Hyper-parameter Learning for Auto-Augmentation Strategy

Data augmentation is critical to the success of modern deep learning techniques. In this paper, we propose Online Hyper-parameter Learning for Auto-Augmentation (OHL-Auto-Aug), an economical solution that learns the augmentation policy distribution along with network training. Unlike previous methods on auto-augmentation that search augmentation strategies in an offline manner, our method formulates the augmentation policy as a parameterized probability distribution, thus allowing its parameters to be optimized jointly with network parameters. Our proposed OHL-Auto-Aug eliminates the need of re-training and dramatically reduces the cost of the overall search process, while establishes significantly accuracy improvements over baseline models. On both CIFAR-10 and ImageNet, our method achieves remarkable on search accuracy, 60x faster on CIFAR-10 and 24x faster on ImageNet, while maintaining competitive accuracies.

cs.CV↗

arXiv2019

AM-LFS: AutoML for Loss Function Search

Designing an effective loss function plays an important role in visual analysis. Most existing loss function designs rely on hand-crafted heuristics that require domain experts to explore the large design space, which is usually sub-optimal and time-consuming. In this paper, we propose AutoML for Loss Function Search (AM-LFS) which leverages REINFORCE to search loss functions during the training process. The key contribution of this work is the design of search space which can guarantee the generalization and transferability on different vision tasks by including a bunch of existing prevailing loss functions in a unified formulation. We also propose an efficient optimization framework which can dynamically optimize the parameters of loss function's distribution during training. Extensive experimental results on four benchmark datasets show that, without any tricks, our method outperforms existing hand-crafted loss functions in various computer vision tasks.

cs.CV↗

arXiv2019

Feedback Network for Image Super-Resolution

Recent advances in image super-resolution (SR) explored the power of deep learning to achieve a better reconstruction performance. However, the feedback mechanism, which commonly exists in human visual system, has not been fully exploited in existing deep learning based image SR methods. In this paper, we propose an image super-resolution feedback network (SRFBN) to refine low-level representations with high-level information. Specifically, we use hidden states in an RNN with constraints to achieve such feedback manner. A feedback block is designed to handle the feedback connections and to generate powerful high-level representations. The proposed SRFBN comes with a strong early reconstruction ability and can create the final high-resolution image step by step. In addition, we introduce a curriculum learning strategy to make the network well suitable for more complicated tasks, where the low-resolution images are corrupted by multiple types of degradation. Extensive experimental results demonstrate the superiority of the proposed SRFBN in comparison with the state-of-the-art methods. Code is avaliable at https://github.com/Paper99/SRFBN_CVPR19.

cs.CV↗