arXiv subjects

Zheng Zhang

Publications and source records attributed to Zheng Zhang.

At least 577 records · Page 32Linked to original sources

arXiv2020

Learning Goal-oriented Dialogue Policy with Opposite Agent Awareness

Most existing approaches for goal-oriented dialogue policy learning used reinforcement learning, which focuses on the target agent policy and simply treat the opposite agent policy as part of the environment. While in real-world scenarios, the behavior of an opposite agent often exhibits certain patterns or underlies hidden policies, which can be inferred and utilized by the target agent to facilitate its own decision making. This strategy is common in human mental simulation by first imaging a specific action and the probable results before really acting it. We therefore propose an opposite behavior aware framework for policy learning in goal-oriented dialogues. We estimate the opposite agent's policy from its behavior and use this estimation to improve the target agent by regarding it as part of the target policy. We evaluate our model on both cooperative and competitive dialogue tasks, showing superior performance over state-of-the-art baselines.

cs.CL↗

arXiv2020

DGL-KE: Training Knowledge Graph Embeddings at Scale

Knowledge graphs have emerged as a key abstraction for organizing information in diverse domains and their embeddings are increasingly used to harness their information in various information retrieval and machine learning tasks. However, the ever growing size of knowledge graphs requires computationally efficient algorithms capable of scaling to graphs with millions of nodes and billions of edges. This paper presents DGL-KE, an open-source package to efficiently compute knowledge graph embeddings. DGL-KE introduces various novel optimizations that accelerate training on knowledge graphs with millions of nodes and billions of edges using multi-processing, multi-GPU, and distributed parallelism. These optimizations are designed to increase data locality, reduce communication overhead, overlap computations with memory accesses, and achieve high operation efficiency. Experiments on knowledge graphs consisting of over 86M nodes and 338M edges show that DGL-KE can compute embeddings in 100 minutes on an EC2 instance with 8 GPUs and 30 minutes on an EC2 cluster with 4 machines with 48 cores/machine. These results represent a 2x~5x speedup over the best competing approaches. DGL-KE is available on https://github.com/awslabs/dgl-ke.

cs.DC↗

arXiv2020

Negative Margin Matters: Understanding Margin in Few-shot Classification

This paper introduces a negative margin loss to metric learning based few-shot learning methods. The negative margin loss significantly outperforms regular softmax loss, and achieves state-of-the-art accuracy on three standard few-shot classification benchmarks with few bells and whistles. These results are contrary to the common practice in the metric learning field, that the margin is zero or positive. To understand why the negative margin loss performs well for the few-shot classification, we analyze the discriminability of learned features w.r.t different margins for training and novel classes, both empirically and theoretically. We find that although negative margin reduces the feature discriminability for training classes, it may also avoid falsely mapping samples of the same novel class to multiple peaks or clusters, and thus benefit the discrimination of novel classes. Code is available at https://github.com/bl0/negative-margin.few-shot.

cs.CV↗

arXiv2020

Pressure induced metallization and possible unconventional superconductivity in spin liquid $NaYbSe_{2}$

Beyond the conventional electron pairing mediated by phonons, high-temperature superconductivity in cuprates is believed to stem from quantum spin liquid (QSL). The unconventional superconductivity by doping a spin liquid/Mott insulator, is a long-sought goal but a principal challenge in condensed matter physics because of the lack of an ideal QSL platform. Here we report the pressure induced metallization and possible unconventional superconductivity in $NaYbSe_{2}$, which belongs to a large and ideal family of triangular lattice spin liquid we revealed recently and is evidenced to possess a QSL ground state. The charge gap of NaYbSe2 is gradually reduced by applying pressures, and at ~20 GPa the crystal jumps into a superconducting (SC) phase with Tc ~ 5.8 K even before the insulating gap is completely closed. The metallization is confirmed by further high-pressure experiments but the sign of superconductivity is not well repeated. No symmetry breaking accompanies the SC transition, as indicated by X-ray diffraction and low-temperature Raman experiments under high pressures. This intrinsically connects QSL and SC phases, and suggests an unconventional superconductivity developed from QSL. We further observed the magnetic-field-tuned superconductor-insulator transition which is analogous to that found in the underdoped cuprate superconductor $La_{2-x}Sr_{x}CuO_{4}$. The study is expected to inspire interest in exploring new types of superconductors and sheds light into the intriguing physics from a spin liquid/Mott insulator to a superconductor.

cond-mat.supr-con↗

arXiv2020

CrossWOZ: A Large-Scale Chinese Cross-Domain Task-Oriented Dialogue Dataset

To advance multi-domain (cross-domain) dialogue modeling as well as alleviate the shortage of Chinese task-oriented datasets, we propose CrossWOZ, the first large-scale Chinese Cross-Domain Wizard-of-Oz task-oriented dataset. It contains 6K dialogue sessions and 102K utterances for 5 domains, including hotel, restaurant, attraction, metro, and taxi. Moreover, the corpus contains rich annotation of dialogue states and dialogue acts at both user and system sides. About 60% of the dialogues have cross-domain user goals that favor inter-domain dependency and encourage natural transition across domains in conversation. We also provide a user simulator and several benchmark models for pipelined task-oriented dialogue systems, which will facilitate researchers to compare and evaluate their models on this corpus. The large size and rich annotation of CrossWOZ make it suitable to investigate a variety of tasks in cross-domain dialogue modeling, such as dialogue state tracking, policy learning, user simulation, etc.

cs.CL↗

arXiv2020

Crystalline Electric-Field Excitations in Quantum Spin Liquids Candidate $NaYbSe_{2}$

Very recently we revealed a large family of triangular lattice quantum spin liquid candidates named rare-earth chalcogenides, which features a high-symmetry structure without structural/charge disorders and spin impurities, and may serve as an ideal platform exploring spin liquid physics. The knowledge of crystalline electric-field (CEF) excitations is an essential step to explore the fundamental magnetism of rare-earth spin systems. Here we employed inelastic neutron scattering (INS) and Raman scattering (RS) to carry out a comprehensive CFE investigation on $NaYbSe_{2}$, a promising representative of the family. By comparison with its nonmagnetic compound $NaLuSe_{2}$, we are able to identify the CEF excitations at 15.8, 24.3 and 30.5 meV at 5K. The selected cuts of the INS spectra are well re-produced with a large anisotropy of $g$ factors ($g_{ab}:g_{c}\sim3:1$). Further, the CEF excitations are explained well by our calculations based on the point charge model. Interestingly, $NaYbSe_{2}$ exhibits an unusual CEF shift to higher energies with increasing temperatures, and the Raman mode close to the first CEF excitation shows an anomalously large softening with decreasing temperatures. The absence of the anomalies in $NaLuSe_{2}$ clearly demonstrates a CEF-phonon coupling not reported in the family. It can be understood in term of the weaker electronegativity of Se. The fact that the smallest first CEF excitation in the sub-family of $NaYbCh_{2}$ is $\sim$ 180K (Ch=O, S, Se), guarantees that the sub-family can be strictly described with an effective S=1/2 picture at sufficiently low temperatures. Interestingly the CEF-phonon coupling revealed here may present alternative possibilities to manipulate the spin systems.

cond-mat.str-el↗

arXiv2020

Transformer on a Diet

Transformer has been widely used thanks to its ability to capture sequence information in an efficient way. However, recent developments, such as BERT and GPT-2, deliver only heavy architectures with a focus on effectiveness. In this paper, we explore three carefully-designed light Transformer architectures to figure out whether the Transformer with less computations could produce competitive results. Experimental results on language model benchmark datasets hint that such trade-off is promising, and the light Transformer reduces 70% parameters at best, while obtains competitive perplexity compared to standard Transformer. The source code is publicly available.

cs.CL↗

arXiv2020

Nambu-Jona-Lasinio model in a sphere

We study the chiral phase transition of the two flavor Nambu-Jona-Lasinio (NJL) model in a sphere with the MIT boundary condition. We find that the MIT boundary condition results in much stronger finite size effects than the antiperiodic boundary condition. Our work may be helpful to study the finite size effects in heavy-ion collision in a more realistic way.

hep-ph↗

arXiv2020

Race, Gender and Beauty: The Effect of Information Provision on Online Hiring Biases

We conduct a study of hiring bias on a simulation platform where we ask Amazon MTurk participants to make hiring decisions for a mathematically intensive task. Our findings suggest hiring biases against Black workers and less attractive workers and preferences towards Asian workers female workers and more attractive workers. We also show that certain UI designs including provision of candidates information at the individual level and reducing the number of choices can significantly reduce discrimination. However provision of candidates information at the subgroup level can increase discrimination. The results have practical implications for designing better online freelance marketplaces.

cs.CY↗

arXiv2020

Fully-Convolutional Intensive Feature Flow Neural Network for Text Recognition

The Deep Convolutional Neural Networks (CNNs) have obtained a great success for pattern recognition, such as recognizing the texts in images. But existing CNNs based frameworks still have several drawbacks: 1) the traditaional pooling operation may lose important feature information and is unlearnable; 2) the tradi-tional convolution operation optimizes slowly and the hierar-chical features from different layers are not fully utilized. In this work, we address these problems by developing a novel deep network model called Fully-Convolutional Intensive Feature Flow Neural Network (IntensiveNet). Specifically, we design a further dense block called intensive block to extract the feature information, where the original inputs and two dense blocks are connected tightly. To encode data appropriately, we present the concepts of dense fusion block and further dense fusion opera-tions for our new intensive block. By adding short connections to different layers, the feature flow and coupling between layers are enhanced. We also replace the traditional convolution by depthwise separable convolution to make the operation efficient. To prevent important feature information being lost to a certain extent, we use a convolution operation with stride 2 to replace the original pooling operation in the customary transition layers. The recognition results on large-scale Chinese string and MNIST datasets show that our IntensiveNet can deliver enhanced recog-nition results, compared with other related deep models.

cs.CV↗

arXiv2019

Deep Self-representative Concept Factorization Network for Representation Learning

In this paper, we investigate the unsupervised deep representation learning issue and technically propose a novel framework called Deep Self-representative Concept Factorization Network (DSCF-Net), for clustering deep features. To improve the representation and clustering abilities, DSCF-Net explicitly considers discovering hidden deep semantic features, enhancing the robustness proper-ties of the deep factorization to noise and preserving the local man-ifold structures of deep features. Specifically, DSCF-Net seamlessly integrates the robust deep concept factorization, deep self-expressive representation and adaptive locality preserving feature learning into a unified framework. To discover hidden deep repre-sentations, DSCF-Net designs a hierarchical factorization architec-ture using multiple layers of linear transformations, where the hierarchical representation is performed by formulating the prob-lem as optimizing the basis concepts in each layer to improve the representation indirectly. DSCF-Net also improves the robustness by subspace recovery for sparse error correction firstly and then performs the deep factorization in the recovered visual subspace. To obtain locality-preserving representations, we also present an adaptive deep self-representative weighting strategy by using the coefficient matrix as the adaptive reconstruction weights to keep the locality of representations. Extensive comparison results with several other related models show that DSCF-Net delivers state-of-the-art performance on several public databases.

cs.LG↗

arXiv2019

Multi-Scale Self-Attention for Text Classification

In this paper, we introduce the prior knowledge, multi-scale structure, into self-attention modules. We propose a Multi-Scale Transformer which uses multi-scale multi-head self-attention to capture features from different scales. Based on the linguistic perspective and the analysis of pre-trained Transformer (BERT) on a huge corpus, we further design a strategy to control the scale distribution for each layer. Results of three different kinds of tasks (21 datasets) show our Multi-Scale Transformer outperforms the standard Transformer consistently and significantly on small and moderate size datasets.

cs.CL↗

arXiv2019

Discriminative Local Sparse Representation by Robust Adaptive Dictionary Pair Learning

In this paper, we propose a structured Robust Adaptive Dic-tionary Pair Learning (RA-DPL) framework for the discrim-inative sparse representation learning. To achieve powerful representation ability of the available samples, the setting of RA-DPL seamlessly integrates the robust projective dictionary pair learning, locality-adaptive sparse representations and discriminative coding coefficients learning into a unified learning framework. Specifically, RA-DPL improves existing projective dictionary pair learning in four perspectives. First, it applies a sparse l2,1-norm based metric to encode the recon-struction error to deliver the robust projective dictionary pairs, and the l2,1-norm has the potential to minimize the error. Sec-ond, it imposes the robust l2,1-norm clearly on the analysis dictionary to ensure the sparse property of the coding coeffi-cients rather than using the costly l0/l1-norm. As such, the robustness of the data representation and the efficiency of the learning process are jointly considered to guarantee the effi-cacy of our RA-DPL. Third, RA-DPL conceives a structured reconstruction weight learning paradigm to preserve the local structures of the coding coefficients within each class clearly in an adaptive manner, which encourages to produce the locality preserving representations. Fourth, it also considers improving the discriminating ability of coding coefficients and dictionary by incorporating a discriminating function, which can ensure high intra-class compactness and inter-class separation in the code space. Extensive experiments show that our RA-DPL can obtain superior performance over other state-of-the-arts.

cs.CV↗

arXiv2019

Learning from the Past: Continual Meta-Learning via Bayesian Graph Modeling

Meta-learning for few-shot learning allows a machine to leverage previously acquired knowledge as a prior, thus improving the performance on novel tasks with only small amounts of data. However, most mainstream models suffer from catastrophic forgetting and insufficient robustness issues, thereby failing to fully retain or exploit long-term knowledge while being prone to cause severe error accumulation. In this paper, we propose a novel Continual Meta-Learning approach with Bayesian Graph Neural Networks (CML-BGNN) that mathematically formulates meta-learning as continual learning of a sequence of tasks. With each task forming as a graph, the intra- and inter-task correlations can be well preserved via message-passing and history transition. To remedy topological uncertainty from graph initialization, we utilize Bayes by Backprop strategy that approximates the posterior distribution of task-specific parameters with amortized inference networks, which are seamlessly integrated into the end-to-end edge learning. Extensive experiments conducted on the miniImageNet and tieredImageNet datasets demonstrate the effectiveness and efficiency of the proposed method, improving the performance by 42.8% compared with state-of-the-art on the miniImageNet 5-way 1-shot classification task.

cs.LG↗

arXiv2019

BP-Transformer: Modelling Long-Range Context via Binary Partitioning

The Transformer model is widely successful on many natural language processing tasks. However, the quadratic complexity of self-attention limit its application on long text. In this paper, adopting a fine-to-coarse attention mechanism on multi-scale spans via binary partitioning (BP), we propose BP-Transformer (BPT for short). BPT yields $O(k\cdot n\log (n/k))$ connections where $k$ is a hyperparameter to control the density of attention. BPT has a good balance between computation complexity and model capacity. A series of experiments on text classification, machine translation and language modeling shows BPT has a superior performance for long text than previous self-attention models. Our code, hyperparameters and CUDA kernels for sparse attention are available in PyTorch.

cs.CL↗

arXiv2019

Deep Collaborative Discrete Hashing with Semantic-Invariant Structure

Existing deep hashing approaches fail to fully explore semantic correlations and neglect the effect of linguistic context on visual attention learning, leading to inferior performance. This paper proposes a dual-stream learning framework, dubbed Deep Collaborative Discrete Hashing (DCDH), which constructs a discriminative common discrete space by collaboratively incorporating the shared and individual semantics deduced from visual features and semantic labels. Specifically, the context-aware representations are generated by employing the outer product of visual embeddings and semantic encodings. Moreover, we reconstruct the labels and introduce the focal loss to take advantage of frequent and rare concepts. The common binary code space is built on the joint learning of the visual representations attended by language, the semantic-invariant structure construction and the label distribution correction. Extensive experiments demonstrate the superiority of our method.

cs.IR↗

arXiv2019

Efficient Uncertainty Modeling for System Design via Mixed Integer Programming

The post-Moore era casts a shadow of uncertainty on many aspects of computer system design. Managing that uncertainty requires new algorithmic tools to make quantitative assessments. While prior uncertainty quantification methods, such as generalized polynomial chaos (gPC), show how to work precisely under the uncertainty inherent to physical devices, these approaches focus solely on variables from a continuous domain. However, as one moves up the system stack to the architecture level many parameters are constrained to a discrete (integer) domain. This paper proposes an efficient and accurate uncertainty modeling technique, named mixed generalized polynomial chaos (M-gPC), for architectural uncertainty analysis. The M-gPC technique extends the generalized polynomial chaos (gPC) theory originally developed in the uncertainty quantification community, such that it can efficiently handle the mixed-type (i.e., both continuous and discrete) uncertainties in computer architecture design. Specifically, we employ some stochastic basis functions to capture the architecture-level impact caused by uncertain parameters in a simulator. We also develop a novel mixed-integer programming method to select a small number of uncertain parameter samples for detailed simulations. With a few highly informative simulation samples, an accurate surrogate model is constructed in place of cycle-level simulators for various architectural uncertainty analysis. In the chip-multiprocessor (CMP) model, we are able to estimate the propagated uncertainties with only 95 samples whereas Monte Carlo requires 5*10^4 samples to achieve the similar accuracy. We also demonstrate the efficiency and effectiveness of our method on a detailed DRAM subsystem.

eess.SP↗

arXiv2019

Fundamental Limits of the Dew-Harvesting Technology

Dew-harvesting technology radiatively cools a condenser below the dewpoint to achieve condensation of the water vapor from the atmosphere. Due to its passive nature, this technology has attracted a broad interest, in particular in the context of the worldwide drinking-water scarcity. However, the fundamental limit of its performance has not yet been clarified. Moreover, the existing applications have been limited to humid areas. Here, we point out the upper bound of the performance of this technology by carefully considering the spectral directional atmospheric transmittance in a wide range of parameters such as the ambient temperature, the relative humidity, and the convection coefficient. Moreover, we highlight the potential of a condenser consisting of a selective emitter, which is capable of condensing water vapor under significantly more arid conditions as compared with the use of a blackbody emitter. For example, a near-ideal emitter could achieve a dew-harvesting mass flux of 13 gram per square metre per hour even at ambient temperature of 20 degrees Celsius with relative humidity of 40%, whereas the black emitter cannot operate. We provide a numerical design of such a selective emitter, consisting of six layers, optimized for dew-harvesting purposes.

physics.app-ph↗