arXiv subjects

Wei Wu

Publications and source records attributed to Wei Wu.

At least 505 records · Page 28Linked to original sources

arXiv2018

Synaptic Strength For Convolutional Neural Network

Convolutional Neural Networks(CNNs) are both computation and memory intensive which hindered their deployment in mobile devices. Inspired by the relevant concept in neural science literature, we propose Synaptic Pruning: a data-driven method to prune connections between input and output feature maps with a newly proposed class of parameters called Synaptic Strength. Synaptic Strength is designed to capture the importance of a connection based on the amount of information it transports. Experiment results show the effectiveness of our approach. On CIFAR-10, we prune connections for various CNN models with up to 96% , which results in significant size reduction and computation saving. Further evaluation on ImageNet demonstrates that synaptic pruning is able to discover efficient models which is competitive to state-of-the-art compact CNNs such as MobileNet-V2 and NasNet-Mobile. Our contribution is summarized as following: (1) We introduce Synaptic Strength, a new class of parameters for CNNs to indicate the importance of each connections. (2) Our approach can prune various CNNs with high compression without compromising accuracy. (3) Further investigation shows, the proposed Synaptic Strength is a better indicator for kernel pruning compared with the previous approach in both empirical result and theoretical analysis.

cs.LG↗

arXiv2018

PointCNN: Convolution On $\mathcal{X}$-Transformed Points

We present a simple and general framework for feature learning from point clouds. The key to the success of CNNs is the convolution operator that is capable of leveraging spatially-local correlation in data represented densely in grids (e.g. images). However, point clouds are irregular and unordered, thus directly convolving kernels against features associated with the points, will result in desertion of shape information and variance to point ordering. To address these problems, we propose to learn an $\mathcal{X}$-transformation from the input points, to simultaneously promote two causes. The first is the weighting of the input features associated with the points, and the second is the permutation of the points into a latent and potentially canonical order. Element-wise product and sum operations of the typical convolution operator are subsequently applied on the $\mathcal{X}$-transformed features. The proposed method is a generalization of typical CNNs to feature learning from point clouds, thus we call it PointCNN. Experiments show that PointCNN achieves on par or better performance than state-of-the-art methods on multiple challenging benchmark datasets and tasks.

cs.CV↗

arXiv2018

Lithium Niobate Metasurfaces

Lithium niobate is a multi-functional material, which has been regarded as one of the most promising platform for the multi-purpose optical components and photonic circuits. Targeting at the miniature optical components and systems, lithium niobate microstructures with feature sizes of several to hundreds of micrometers have been demonstrated, such as waveguides, photonic crystals, micro-cavities, and modulators, et al. In this paper, we presented subwavelength nanograting metasurfaces fabricated in a crystalline lithium niobate film, which hold the possibilities towards further shrinking the footprint of the photonic devices with new optical functionalities. Due to the collective lattice interactions between isolated ridge resonances, distinct transmission spectral resonances were observed, which could be tunable by varying the structural parameters. Furthermore, our metasurfaces are capable to show high efficiency transmission structural colors as a result of structural resonances and intrinsic high transparency of lithium niobate in visible spectral range. Our results would pave the way for the new types of ultracompact photonic devices based on lithium niobate.

physics.optics↗

arXiv2018

Recursive Geman-McClure method for implementing second-order Volterra filter

The second-order Volterra (SOV) filter is a powerful tool for modeling the nonlinear system. The Geman-McClure estimator, whose loss function is non-convex and has been proven to be a robust and efficient optimization criterion for learning system. In this paper, we present a SOV filter, named SOV recursive Geman-McClure, which is an adaptive recursive Volterra algorithm based on the Geman-McClure estimator. The mean stability and mean-square stability (steady-state excess mean square error (EMSE)) of the proposed algorithm is analyzed in detail. Simulation results support the analytical findings and show the improved performance of the proposed new SOV filter as compared with existing algorithms in both Gaussian and impulsive noise environments.

eess.SY↗

arXiv2018

Four dimensional loop-erased random walk

The loop-erased random walk (LERW) in $\mathbb{Z}^4$ is the process obtained by erasing loops chronologically for simple random walk. We prove that the escape probability of the LERW renormalized by $(\log n)^{\frac{1}{3}}$ converges almost surely and in $L^{p}$ for all $p>0$. Along the way, we extend previous results by the first author building on slowly recurrent sets. We provide two applications for the escape probability. We construct the two-sided LERW, and we construct a $\pm 1$ spin model coupled with the wired spanning forests on $\mathbb{Z}^4$ with the bi-Laplacian Gaussian field on $\mathbb{R}^{4}$ as its scaling limit.

math.PR↗

arXiv2018

Lee-Yang Property and Gaussian multiplicative chaos

The Lee-Yang property of certain moment generating functions having only pure imaginary zeros is valid for Ising type models with one-component spins and XY models with two-component spins. Villain models and complex Gaussian multiplicative chaos are two-component systems analogous to XY models and related to Gaussian free fields. Although the Lee-Yang property is known to be valid generally in the first case, we show that is not so in the second. Our proof is based on two theorems of general interest relating the Lee-Yang property to distribution tail behavior.

math-ph↗

arXiv2018

Distractor-aware Siamese Networks for Visual Object Tracking

Recently, Siamese networks have drawn great attention in visual tracking community because of their balanced accuracy and speed. However, features used in most Siamese tracking approaches can only discriminate foreground from the non-semantic backgrounds. The semantic backgrounds are always considered as distractors, which hinders the robustness of Siamese trackers. In this paper, we focus on learning distractor-aware Siamese networks for accurate and long-term tracking. To this end, features used in traditional Siamese trackers are analyzed at first. We observe that the imbalanced distribution of training data makes the learned features less discriminative. During the off-line training phase, an effective sampling strategy is introduced to control this distribution and make the model focus on the semantic distractors. During inference, a novel distractor-aware module is designed to perform incremental learning, which can effectively transfer the general embedding to the current video domain. In addition, we extend the proposed approach for long-term tracking by introducing a simple yet effective local-to-global search region strategy. Extensive experiments on benchmarks show that our approach significantly outperforms the state-of-the-arts, yielding 9.6% relative gain in VOT2016 dataset and 35.9% relative gain in UAV20L dataset. The proposed tracker can perform at 160 FPS on short-term benchmarks and 110 FPS on long-term benchmarks.

cs.CV↗

arXiv2018

BlockQNN: Efficient Block-wise Neural Network Architecture Generation

Convolutional neural networks have gained a remarkable success in computer vision. However, most usable network architectures are hand-crafted and usually require expertise and elaborate design. In this paper, we provide a block-wise network generation pipeline called BlockQNN which automatically builds high-performance networks using the Q-Learning paradigm with epsilon-greedy exploration strategy. The optimal network block is constructed by the learning agent which is trained to choose component layers sequentially. We stack the block to construct the whole auto-generated network. To accelerate the generation process, we also propose a distributed asynchronous framework and an early stop strategy. The block-wise generation brings unique advantages: (1) it yields state-of-the-art results in comparison to the hand-crafted networks on image classification, particularly, the best network generated by BlockQNN achieves 2.35% top-1 error rate on CIFAR-10. (2) it offers tremendous reduction of the search space in designing networks, spending only 3 days with 32 GPUs. A faster version can yield a comparable result with only 1 GPU in 20 hours. (3) it has strong generalizability in that the network built on CIFAR also performs well on the larger-scale dataset. The best network achieves very competitive accuracy of 82.0% top-1 and 96.0% top-5 on ImageNet.

cs.CV↗

arXiv2018

Stochastic decoupling approach to the quantum dissipative dynamics: Perturbative and non-perturbative treatments

We develop a hierarchical functional derivative method to investigate the reduced dynamics of a quantum dissipative system within the framework of a stochastic decoupling description. Keeping only the lowest order truncation of the hierarchical functional derivatives, one can recover the second order Nakajima-Zwanzig quantum master equation. Taking into account the higher-order corrections, our method can be implemented as a highly efficient numerical scheme for a general quantum dissipative system beyond the usual Markovian, rotating-wave, and weak-coupling approximations.

quant-ph↗

arXiv2018

Towards Explainable and Controllable Open Domain Dialogue Generation with Dialogue Acts

We study open domain dialogue generation with dialogue acts designed to explain how people engage in social chat. To imitate human behavior, we propose managing the flow of human-machine interactions with the dialogue acts as policies. The policies and response generation are jointly learned from human-human conversations, and the former is further optimized with a reinforcement learning approach. With the dialogue acts, we achieve significant improvement over state-of-the-art methods on response quality for given contexts and dialogue length in both machine-machine simulation and human-machine conversation.

cs.CL↗

arXiv2018

Realization of hierarchical equations of motion from stochastic perspectives

The hierarchical equations of motion (HEOM) for a generalized quantum dissipative system is rigorously constructed in the frameworks of two different stochastic dynamical descriptions, i.e., the non-Markovian quantum state diffusion approach as well as the stochastic decoupling scheme. We demonstrated that the HEOMs obtained by these two different stochastic dynamical methods are identical. Moreover, we present some numerical examples to verify the feasibility of our formalism.

quant-ph↗

arXiv2018

Prolonged mixed phase induced by high pressure in MnRuP

Hexagonally structured MnRuP was studied under high pressure up to 35 GPa from 5 to 300 K using synchrotron X-ray diffraction. We observed that a partial phase transition from hexagonal to orthorhombic symmetry started at 11 GPa. The new and denser orthorhombic phase coexisted with its parent phase for an unusually long pressure range, ΔP ~ 50 GPa. We attribute this structural transformation to a magnetic origin, where a decisive criterion for the boundary of the mixed phase lays in the different distances between the Mn-Mn atoms. In addition, our theoretical study shows that the orthorhombic phase of MnRuP remains steady even at very high pressures up to ~ 250 GPa, when it should transform to a new tetragonal phase.

cond-mat.mtrl-sci↗

arXiv2018

SGM: Sequence Generation Model for Multi-label Classification

Multi-label classification is an important yet challenging task in natural language processing. It is more complex than single-label classification in that the labels tend to be correlated. Existing methods tend to ignore the correlations between labels. Besides, different parts of the text can contribute differently for predicting different labels, which is not considered by existing models. In this paper, we propose to view the multi-label classification task as a sequence generation problem, and apply a sequence generation model with a novel decoder structure to solve it. Extensive experimental results show that our proposed methods outperform previous work by a substantial margin. Further analysis of experimental results demonstrates that the proposed methods not only capture the correlations between labels, but also select the most informative words automatically when predicting different labels.

cs.CL↗

arXiv2018

Practical Block-wise Neural Network Architecture Generation

Convolutional neural networks have gained a remarkable success in computer vision. However, most usable network architectures are hand-crafted and usually require expertise and elaborate design. In this paper, we provide a block-wise network generation pipeline called BlockQNN which automatically builds high-performance networks using the Q-Learning paradigm with epsilon-greedy exploration strategy. The optimal network block is constructed by the learning agent which is trained sequentially to choose component layers. We stack the block to construct the whole auto-generated network. To accelerate the generation process, we also propose a distributed asynchronous framework and an early stop strategy. The block-wise generation brings unique advantages: (1) it performs competitive results in comparison to the hand-crafted state-of-the-art networks on image classification, additionally, the best network generated by BlockQNN achieves 3.54% top-1 error rate on CIFAR-10 which beats all existing auto-generate networks. (2) in the meanwhile, it offers tremendous reduction of the search space in designing networks which only spends 3 days with 32 GPUs, and (3) moreover, it has strong generalizability that the network built on CIFAR also performs well on a larger-scale ImageNet dataset.

cs.CV↗

arXiv2018

Learning Matching Models with Weak Supervision for Response Selection in Retrieval-based Chatbots

We propose a method that can leverage unlabeled data to learn a matching model for response selection in retrieval-based chatbots. The method employs a sequence-to-sequence architecture (Seq2Seq) model as a weak annotator to judge the matching degree of unlabeled pairs, and then performs learning with both the weak signals and the unlabeled data. Experimental results on two public data sets indicate that matching models get significant improvements when they are learned with the proposed method.

cs.CL↗

arXiv2018

Reply to "Comment on 'Modified quantum-speed-limit bounds for open quantum dynamics in quantum channels'"

The authors of the comment[Phys. Rev. A 97, 046101 (2018)] raise that the inconsistency in calculating some common quantum-speed-limit (QSL) bounds, which is presented in our paper [Phys. Rev. A 95, 052118 (2017)], does not exist in their paper [Phys. Rev. A 94, 052125 (2016)]. Therefore they insist that their criticism to some QSL bounds is still valid. We demonstrate all the QSL bounds mentioned in the comment are similar in essence. We also show the inconsistency which is presented in their original paper can not be used to support their criticism. Furthermore, we exhibit although the inconsistency presented in our paper is not a same one to which is presented by the authors of the comment, it exists and be unavoidable in their example. Accordingly, we believe the result in our paper is usable in numerical calculations.

quant-ph↗

arXiv2018

Wave-Field Shaping in Cavities by Tunable Metasurfaces

Cavities, because they trap waves for long times due to their reflecting walls, are used in a vast number of scientific domains. Indeed, in these closed media and due to interferences, the free space continuum of solutions becomes a discrete set of stationary eigenmodes. These enhanced stationary fields are commonly used in fundamental physics to increase wave-matter interactions. The eigenmodes and associated eigenfrequencies of a cavity are imposed by its geometrical properties through the boundary conditions. In this paper, we show that one can control the wave fields created by point sources inside cavities by tailoring only the boundaries of the cavities. This is achieved through the use of a tunable reflecting metasurface, which is part of the frontiers of the cavity, and can switch its boundary conditions from Dirichlet to Neumann. Based on the use of arrays of subwavelength resonators, a mathematical modeling of the physical mechanism underlying the concept of tunable metasurfaces is provided.

math.AP↗

arXiv2018

Topological order in the pseudogap metal

We compute the electronic Green's function of the topologically ordered Higgs phase of a SU(2) gauge theory of fluctuating antiferromagnetism on the square lattice. The results are compared with cluster extensions of dynamical mean field theory, and quantum Monte Carlo calculations, on the pseudogap phase of the strongly interacting hole-doped Hubbard model. Good agreement is found in the momentum, frequency, hopping, and doping dependencies of the spectral function and electronic self-energy. We show that lines of (approximate) zeros of the zero-frequency electronic Green's function are signs of the underlying topological order of the gauge theory, and describe how these lines of zeros appear in our theory of the Hubbard model. We also derive a modified, non-perturbative version of the Luttinger theorem that holds in the Higgs phase.

cond-mat.str-el↗