arXiv subjects

Zheng Zhang

Publications and source records attributed to Zheng Zhang.

At least 613 records · Page 34Linked to original sources

arXiv2019

An Empirical Study of Spatial Attention Mechanisms in Deep Networks

Attention mechanisms have become a popular component in deep neural networks, yet there has been little examination of how different influencing factors and methods for computing attention from these factors affect performance. Toward a better general understanding of attention mechanisms, we present an empirical study that ablates various spatial attention elements within a generalized attention formulation, encompassing the dominant Transformer attention as well as the prevalent deformable convolution and dynamic convolution modules. Conducted on a variety of applications, the study yields significant findings about spatial attention in deep networks, some of which run counter to conventional understanding. For example, we find that the query and key content comparison in Transformer attention is negligible for self-attention, but vital for encoder-decoder attention. A proper combination of deformable convolution with key content only saliency achieves the best accuracy-efficiency tradeoff in self-attention. Our results suggest that there exists much room for improvement in the design of attention mechanisms.

cs.CV↗

arXiv2019

Effects of transmutation elements in tungsten

Tungsten (W) is widely considered as the most promising plasma facing material (PFM), which will be used in nuclear fusion devices. Due to the transmutation reaction caused by the fusion neutron irradiation, transmutation elements (such as Re, Os, and Ta) are generated in the W-based PFM during the operation of nuclear fusion devices. In this paper, transmutation elements effects on mechanical properties of the W and the behavior of hydrogen/helium (H/He) atom in the W were investigated by using the first-principles calculation method. The results show that the ductility of the W is enhanced by transmutation elements if dislocation and other defects were ignored, while mechanical properties of the W incompletely depend on transmutation elements content. Compared with the pure W, the formation energy of the H/He in the W is reduced by transmutation elements, but the most favorable sites of the H/He in the W is not changed. Except for a repulsion between Ta and He in the W, the H/He in the W is attracted by transmutation elements. In addition, transmutation elements can change the best diffusion path of the H/He in the W and increase the diffusion rate of the H/He in W. This study provides a theoretical basis for the Tungsten (W) application, and further study on the effects of transmutation elements in the W will be needed.

cond-mat.mtrl-sci↗

arXiv2019

Explainable Text-Driven Neural Network for Stock Prediction

It has been shown that financial news leads to the fluctuation of stock prices. However, previous work on news-driven financial market prediction focused only on predicting stock price movement without providing an explanation. In this paper, we propose a dual-layer attention-based neural network to address this issue. In the initial stage, we introduce a knowledge-based method to adaptively extract relevant financial news. Then, we use input attention to pay more attention to the more influential news and concatenate the day embeddings with the output of the news representation. Finally, we use an output attention mechanism to allocate different weights to different days in terms of their contribution to stock price movement. Thorough empirical studies based upon historical prices of several individual stocks demonstrate the superiority of our proposed method in stock price prediction compared to state-of-the-art methods.

cs.CL↗

arXiv2019

Bilinear Supervised Hashing Based on 2D Image Features

Hashing has been recognized as an efficient representation learning method to effectively handle big data due to its low computational complexity and memory cost. Most of the existing hashing methods focus on learning the low-dimensional vectorized binary features based on the high-dimensional raw vectorized features. However, studies on how to obtain preferable binary codes from the original 2D image features for retrieval is very limited. This paper proposes a bilinear supervised discrete hashing (BSDH) method based on 2D image features which utilizes bilinear projections to binarize the image matrix features such that the intrinsic characteristics in the 2D image space are preserved in the learned binary codes. Meanwhile, the bilinear projection approximation and vectorization binary codes regression are seamlessly integrated together to formulate the final robust learning framework. Furthermore, a discrete optimization strategy is developed to alternatively update each variable for obtaining the high-quality binary codes. In addition, two 2D image features, traditional SURF-based FVLAD feature and CNN-based AlexConv5 feature are designed for further improving the performance of the proposed BSDH method. Results of extensive experiments conducted on four benchmark datasets show that the proposed BSDH method almost outperforms all competing hashing methods with different input features by different evaluation protocols.

cs.CV↗

arXiv2019

Prediction of multi-dimensional spatial variation data via Bayesian tensor completion

This paper presents a multi-dimensional computational method to predict the spatial variation data inside and across multiple dies of a wafer. This technique is based on tensor computation. A tensor is a high-dimensional generalization of a matrix or a vector. By exploiting the hidden low-rank property of a high-dimensional data array, the large amount of unknown variation testing data may be predicted from a few random measurement samples. The tensor rank, which decides the complexity of a tensor representation, is decided by an available variational Bayesian approach. Our approach is validated by a practical chip testing data set, and it can be easily generalized to characterize the process variations of multiple wafers. Our approach is more efficient than the previous virtual probe techniques in terms of memory and computational cost when handling high-dimensional chip testing data.

cs.LG↗

arXiv2019

Adaptive Locality Preserving Regression

This paper proposes a novel discriminative regression method, called adaptive locality preserving regression (ALPR) for classification. In particular, ALPR aims to learn a more flexible and discriminative projection that not only preserves the intrinsic structure of data, but also possesses the properties of feature selection and interpretability. To this end, we introduce a target learning technique to adaptively learn a more discriminative and flexible target matrix rather than the pre-defined strict zero-one label matrix for regression. Then a locality preserving constraint regularized by the adaptive learned weights is further introduced to guide the projection learning, which is beneficial to learn a more discriminative projection and avoid overfitting. Moreover, we replace the conventional `Frobenius norm' with the special l21 norm to constrain the projection, which enables the method to adaptively select the most important features from the original high-dimensional data for feature extraction. In this way, the negative influence of the redundant features and noises residing in the original data can be greatly eliminated. Besides, the proposed method has good interpretability for features owing to the row-sparsity property of the l21 norm. Extensive experiments conducted on the synthetic database with manifold structure and many real-world databases prove the effectiveness of the proposed method.

cs.CV↗

arXiv2018

Variational Bayesian Inference for Robust Streaming Tensor Factorization and Completion

Streaming tensor factorization is a powerful tool for processing high-volume and multi-way temporal data in Internet networks, recommender systems and image/video data analysis. Existing streaming tensor factorization algorithms rely on least-squares data fitting and they do not possess a mechanism for tensor rank determination. This leaves them susceptible to outliers and vulnerable to over-fitting. This paper presents a Bayesian robust streaming tensor factorization model to identify sparse outliers, automatically determine the underlying tensor rank and accurately fit low-rank structure. We implement our model in Matlab and compare it with existing algorithms on tensor datasets generated from dynamic MRI and Internet traffic.

stat.ML↗

arXiv2018

Stochastic Collocation with Non-Gaussian Correlated Process Variations: Theory, Algorithms and Applications

Stochastic spectral methods have achieved great success in the uncertainty quantification of many engineering problems, including electronic and photonic integrated circuits influenced by fabrication process variations. Existing techniques employ a generalized polynomial-chaos expansion, and they almost always assume that all random parameters are mutually independent or Gaussian correlated. However, this assumption is rarely true in real applications. How to handle non-Gaussian correlated random parameters is a long-standing and fundamental challenge. A main bottleneck is the lack of theory and computational methods to perform a projection step in a correlated uncertain parameter space. This paper presents an optimization-based approach to automatically determinate the quadrature nodes and weights required in a projection step, and develops an efficient stochastic collocation algorithm for systems with non-Gaussian correlated parameters. We also provide some theoretical proofs for the complexity and error bound of our proposed method. Numerical experiments on synthetic, electronic and photonic integrated circuit examples show the nearly exponential convergence rate and excellent efficiency of our proposed approach. Many other challenging uncertainty-related problems can be further solved based on this work.

math.NA↗

arXiv2018

Integrated Object Detection and Tracking with Tracklet-Conditioned Detection

Accurate detection and tracking of objects is vital for effective video understanding. In previous work, the two tasks have been combined in a way that tracking is based heavily on detection, but the detection benefits marginally from the tracking. To increase synergy, we propose to more tightly integrate the tasks by conditioning the object detection in the current frame on tracklets computed in prior frames. With this approach, the object detection results not only have high detection responses, but also improved coherence with the existing tracklets. This greater coherence leads to estimated object trajectories that are smoother and more stable than the jittered paths obtained without tracklet-conditioned detection. Over extensive experiments, this approach is shown to achieve state-of-the-art performance in terms of both detection and tracking accuracy, as well as noticeable improvements in tracking stability.

cs.CV↗

arXiv2018

Melodic Phrase Segmentation By Deep Neural Networks

Automated melodic phrase detection and segmentation is a classical task in content-based music information retrieval and also the key towards automated music structure analysis. However, traditional methods still cannot satisfy practical requirements. In this paper, we explore and adapt various neural network architectures to see if they can be generalized to work with the symbolic representation of music and produce satisfactory melodic phrase segmentation. The main issue of applying deep-learning methods to phrase detection is the sparse labeling problem of training sets. We proposed two tailored label engineering with corresponding training techniques for different neural networks in order to make decisions at a sequential level. Experiment results show that the CNN-CRF architecture performs the best, being able to offer finer segmentation and faster to train, while CNN, Bi-LSTM-CNN and Bi-LSTM-CRF are acceptable alternatives.

cs.LG↗

arXiv2018

On the moduli space of pairs consisting of a cubic threefold and a hyperplane

We study the moduli space of pairs $(X,H)$ consisting of a cubic threefold $X$ and a hyperplane $H$ in $\mathbb P^4$. The interest in this moduli comes from two sources: the study of certain weighted hypersurfaces whose middle cohomology admit Hodge structures of $K3$ type and, on the other hand, the study of the singularity $O_{16}$ (the cone over a cubic surface). In this paper, we give a Hodge theoretic construction of the moduli space of cubic pairs by relating $(X,H)$ to certain "lattice polarized" cubic fourfolds $Y$. A period map for the pairs $(X,H)$ is then defined using the periods of the cubic fourfolds $Y$. The main result is that the period map induces an isomorphism between a GIT model for the pairs $(X,H)$ and the Baily-Borel compactification of some locally symmetric domain of type IV.

math.AG↗

arXiv2018

Loss Functions for Multiset Prediction

We study the problem of multiset prediction. The goal of multiset prediction is to train a predictor that maps an input to a multiset consisting of multiple items. Unlike existing problems in supervised learning, such as classification, ranking and sequence generation, there is no known order among items in a target multiset, and each item in the multiset may appear more than once, making this problem extremely challenging. In this paper, we propose a novel multiset loss function by viewing this problem from the perspective of sequential decision making. The proposed multiset loss function is empirically evaluated on two families of datasets, one synthetic and the other real, with varying levels of difficulty, against various baseline loss functions including reinforcement learning, sequence, and aggregated distribution matching loss functions. The experiments reveal the effectiveness of the proposed loss function over the others.

cs.LG↗

arXiv2018

Rare-earth chalcogenides: A large family of triangular lattice spin liquid candidate

Frustrated quantum magnets are expected to host many exotic quantum spin states like quantum spin liquid (QSL), and have attracted numerous interest in modern condensed matter physics. The discovery of the triangular lattice spin liquid candidate YbMgGaO$_4$ stimulated an increasing attention on the rare-earth-based frustrated magnets with strong spin-orbit coupling. Here we report the synthesis and characterization of a large family of rare-earth chalcogenides AReCh$_2$ (A = alkali or monovalent ions, Re = rare earth, Ch = O, S, Se). The family compounds share the same structure (R$\bar{3}$m) as YbMgGaO$_4$, and antiferromagnetically coupled rare-earth ions form perfect triangular layers that are well separated along the $c$-axis. Specific heat and magnetic susceptibility measurements on NaYbO$_2$, NaYbS$_2$ and NaYbSe$_2$ single crystals and polycrystals, reveal no structural or magnetic transition down to 50mK. The family, having the simplest structure and chemical formula among the known QSL candidates, removes the issue on possible exchange disorders in YbMgGaO$_4$. More excitingly, the rich diversity of the family members allows tunable charge gaps, variable exchange coupling, and many other advantages. This makes the family an ideal platform for fundamental research of QSLs and its promising applications.

cond-mat.str-el↗

arXiv2018

Highly-Economized Multi-View Binary Compression for Scalable Image Clustering

How to economically cluster large-scale multi-view images is a long-standing problem in computer vision. To tackle this challenge, we introduce a novel approach named Highly-economized Scalable Image Clustering (HSIC) that radically surpasses conventional image clustering methods via binary compression. We intuitively unify the binary representation learning and efficient binary cluster structure learning into a joint framework. In particular, common binary representations are learned by exploiting both sharable and individual information across multiple views to capture their underlying correlations. Meanwhile, cluster assignment with robust binary centroids is also performed via effective discrete optimization under L21-norm constraint. By this means, heavy continuous-valued Euclidean distance computations can be successfully reduced by efficient binary XOR operations during the clustering procedure. To our best knowledge, HSIC is the first binary clustering work specifically designed for scalable multi-view image clustering. Extensive experimental results on four large-scale image datasets show that HSIC consistently outperforms the state-of-the-art approaches, whilst significantly reducing computational time and memory footprint.

cs.CV↗

arXiv2018

Incomplete Multi-view Clustering via Graph Regularized Matrix Factorization

Clustering with incomplete views is a challenge in multi-view clustering. In this paper, we provide a novel and simple method to address this issue. Specifically, the proposed method simultaneously exploits the local information of each view and the complementary information among views to learn the common latent representation for all samples, which can greatly improve the compactness and discriminability of the obtained representation. Compared with the conventional graph embedding methods, the proposed method does not introduce any extra regularization term and corresponding penalty parameter to preserve the local structure of data, and thus does not increase the burden of extra parameter selection. By imposing the orthogonal constraint on the basis matrix of each view, the proposed method is able to handle the out-of-sample. Moreover, the proposed method can be viewed as a unified framework for multi-view learning since it can handle both incomplete and complete multi-view clustering and classification tasks. Extensive experiments conducted on several multi-view datasets prove that the proposed method can significantly improve the clustering performance.

cs.CV↗

arXiv2018

Robust Factorization and Completion of Streaming Tensor Data via Variational Bayesian Inference

Streaming tensor factorization is a powerful tool for processing high-volume and multi-way temporal data in Internet networks, recommender systems and image/video data analysis. In many applications the full tensor is not known, but instead received in a slice-by-slice manner over time. Streaming factorizations aim to take advantage of inherent temporal relationships in data analytics. Existing streaming tensor factorization algorithms rely on least-squares data fitting and they do not possess a mechanism for tensor rank determination. This leaves them susceptible to outliers and vulnerable to over-fitting. This paper presents the first Bayesian robust streaming tensor factorization model. Our model successfully identifies sparse outliers, automatically determines the underlying tensor rank and accurately fits low-rank structure. We implement our model in Matlab and compare it to existing algorithms. Our algorithm is applied to factorize and complete various streaming tensors including synthetic data, dynamic MRI, video sequences, and Internet traffic data.

eess.SP↗

arXiv2018

Stochastic Collocation with Non-Gaussian Correlated Parameters via a New Quadrature Rule

This paper generalizes stochastic collocation methods to handle correlated non-Gaussian random parameters. The key challenge is to perform a multivariate numerical integration in a correlated parameter space when computing the coefficient of each basis function via a projection step. We propose an optimization model and a block coordinate descent solver to compute the required quadrature samples. Our method is verified with a CMOS ring oscillator and an optical ring resonator, showing 3000x speedup over Monte Carlo.

math.NA↗

arXiv2018

A Unified Framework for Efficient Estimation of General Treatment Models

This paper presents a weighted optimization framework that unifies the binary,multi-valued, continuous, as well as mixture of discrete and continuous treatment, under the unconfounded treatment assignment. With a general loss function, the framework includes the average, quantile and asymmetric least squares causal effect of treatment as special cases. For this general framework, we first derive the semiparametric efficiency bound for the causal effect of treatment, extending the existing bound results to a wider class of models. We then propose a generalized optimization estimation for the causal effect with weights estimated by solving an expanding set of equations. Under some sufficient conditions, we establish consistency and asymptotic normality of the proposed estimator of the causal effect and show that the estimator attains our semiparametric efficiency bound, thereby extending the existing literature on efficient estimation of causal effect to a wider class of applications. Finally, we discuss etimation of some causal effect functionals such as the treatment effect curve and the average outcome. To evaluate the finite sample performance of the proposed procedure, we conduct a small scale simulation study and find that the proposed estimation has practical value. To illustrate the applicability of the procedure, we revisit the literature on campaign advertise and campaign contributions. Unlike the existing procedures which produce mixed results, we find no evidence of campaign advertise on campaign contribution.

econ.EM↗