arXiv subjects

Peng Xu

Publications and source records attributed to Peng Xu.

At least 325 records · Page 18Linked to original sources

arXiv2020

Cross-lingual Spoken Language Understanding with Regularized Representation Alignment

Despite the promising results of current cross-lingual models for spoken language understanding systems, they still suffer from imperfect cross-lingual representation alignments between the source and target languages, which makes the performance sub-optimal. To cope with this issue, we propose a regularization approach to further align word-level and sentence-level representations across languages without any external resource. First, we regularize the representation of user utterances based on their corresponding labels. Second, we regularize the latent variable model (Liu et al., 2019) by leveraging adversarial training to disentangle the latent variables. Experiments on the cross-lingual spoken language understanding task show that our model outperforms current state-of-the-art methods in both few-shot and zero-shot scenarios, and our model, trained on a few-shot setting with only 3\% of the target language training data, achieves comparable performance to the supervised training with all the training data.

cs.CL↗

arXiv2020

Improve Transformer Models with Better Relative Position Embeddings

Transformer architectures rely on explicit position encodings in order to preserve a notion of word order. In this paper, we argue that existing work does not fully utilize position information. For example, the initial proposal of a sinusoid embedding is fixed and not learnable. In this paper, we first review absolute position embeddings and existing methods for relative position embeddings. We then propose new techniques that encourage increased interaction between query, key and relative position embeddings in the self-attention mechanism. Our most promising approach is a generalization of the absolute position embedding, improving results on SQuAD1.1 compared to previous position embeddings approaches. In addition, we address the inductive property of whether a position embedding can be robust enough to handle long sequences. We demonstrate empirically that our relative position embedding method is reasonably generalized and robust from the inductive perspective. Finally, we show that our proposed method can be adopted as a near drop-in replacement for improving the accuracy of large models with a small computational budget.

cs.CL↗

arXiv2020

Embedding-based Zero-shot Retrieval through Query Generation

Passage retrieval addresses the problem of locating relevant passages, usually from a large corpus, given a query. In practice, lexical term-matching algorithms like BM25 are popular choices for retrieval owing to their efficiency. However, term-based matching algorithms often miss relevant passages that have no lexical overlap with the query and cannot be finetuned to downstream datasets. In this work, we consider the embedding-based two-tower architecture as our neural retrieval model. Since labeled data can be scarce and because neural retrieval models require vast amounts of data to train, we propose a novel method for generating synthetic training data for retrieval. Our system produces remarkable results, significantly outperforming BM25 on 5 out of 6 datasets tested, by an average of 2.45 points for Recall@1. In some cases, our model trained on synthetic data can even outperform the same model trained on real data

cs.IR↗

arXiv2020

EmoGraph: Capturing Emotion Correlations using Graph Networks

Most emotion recognition methods tackle the emotion understanding task by considering individual emotion independently while ignoring their fuzziness nature and the interconnections among them. In this paper, we explore how emotion correlations can be captured and help different classification tasks. We propose EmoGraph that captures the dependencies among different emotions through graph networks. These graphs are constructed by leveraging the co-occurrence statistics among different emotion categories. Empirical results on two multi-label classification datasets demonstrate that EmoGraph outperforms strong baselines, especially for macro-F1. An additional experiment illustrates the captured emotion correlations can also benefit a single-label classification task.

cs.CL↗

arXiv2020

Extension of causal decomposition in the mutual complex dynamic process

Causal decomposition depicts a cause-effect relationship that is not based on the concept of prediction, but based on the phase dependence of time series. It has been validated in both stochastic and deterministic systems and is now anticipated for its application in the complex dynamic process. Here, we present an extension of causal decomposition in the mutual complex dynamic process: cause and effect of time series are inherited in the decomposition of intrinsic components in a similar time scale. Furthermore, we illustrate comparative studies with predominate methods used in neuroscience, and show the applicability of the method particularly to physiological time series in brain-muscle interactions, implying the potential to the causality analysis in the complex physiological process.

eess.SP↗

arXiv2020

Reconfigurable Intelligent Surfaces Assisted Communications with Discrete Phase Shifts: How Many Quantization Levels are Required to Achieve Full Diversity?

Due to hardware limitations, the phase shifts of the reflecting elements of reconfigurable intelligent surfaces (RISs) need to be quantized into discrete values. This letter aims to unveil the minimum required number of phase quantization levels $L$ in order to achieve the full diversity order in RIS-assisted wireless communication systems. With the aid of an upper bound of the outage probability, we first prove that the full diversity order is achievable provided that $L$ is not less than three. If $L=2$, on the other hand, we prove that the achievable diversity order cannot exceed $(N+1)/2$, where $N$ is the number of reflecting elements. This is obtained with the aid of a lower bound of the outage probability. Therefore, we prove that the minimum required value of $L$ to achieve the full diversity order is $L=3$. Simulation results verify the theoretical analysis and the impact of phase quantization levels on RIS-assisted communication systems.

cs.IT↗

arXiv2020

On Variational Learning of Controllable Representations for Text without Supervision

The variational autoencoder (VAE) can learn the manifold of natural images on certain datasets, as evidenced by meaningful interpolating or extrapolating in the continuous latent space. However, on discrete data such as text, it is unclear if unsupervised learning can discover similar latent space that allows controllable manipulation. In this work, we find that sequence VAEs trained on text fail to properly decode when the latent codes are manipulated, because the modified codes often land in holes or vacant regions in the aggregated posterior latent space, where the decoding network fails to generalize. Both as a validation of the explanation and as a fix to the problem, we propose to constrain the posterior mean to a learned probability simplex, and performs manipulation within this simplex. Our proposed method mitigates the latent vacancy problem and achieves the first success in unsupervised learning of controllable representations for text. Empirically, our method outperforms unsupervised baselines and strong supervised approaches on text style transfer, and is capable of performing more flexible fine-grained control over text generation than existing methods.

cs.CL↗

arXiv2020

Density Dependent Spin-Orbit Coupling in Degenerate Quantum Gases

In this letter we propose a method to realize a kind of spin-orbit coupling in ultracold Bose and Fermi gases whose format and strength depend on density of atoms. Our method combines two-photon Raman transition and periodical modulation of spin-dependent interaction, which gives rise to the direct Raman process and the interaction assisted Raman process, and the latter depends on density of atoms. These two processes have opposite effects in term of spin-momentum locking and compete with each other. As the interaction modulation increases, the system undergoes a crossover from the direct Raman process dominated regime to the interaction assisted Raman process dominated regime. For this crossover, we show that for bosons, both the condensate momentum and the chirality of condensate wave function change sign, and for fermions, the Fermi surface distortion is inverted. We highlight that there exists an emergent spatial reflection symmetry in the crossover regime, which can manifest itself universally in both Bose and Fermi gases. Our method paves a way to novel phenomena in a non-abelian gauge field with intrinsic dynamics.

cond-mat.quant-gas↗

arXiv2020

On Learning Semantic Representations for Million-Scale Free-Hand Sketches

In this paper, we study learning semantic representations for million-scale free-hand sketches. This is highly challenging due to the domain-unique traits of sketches, e.g., diverse, sparse, abstract, noisy. We propose a dual-branch CNNRNN network architecture to represent sketches, which simultaneously encodes both the static and temporal patterns of sketch strokes. Based on this architecture, we further explore learning the sketch-oriented semantic representations in two challenging yet practical settings, i.e., hashing retrieval and zero-shot recognition on million-scale sketches. Specifically, we use our dual-branch architecture as a universal representation framework to design two sketch-specific deep models: (i) We propose a deep hashing model for sketch retrieval, where a novel hashing loss is specifically designed to accommodate both the abstract and messy traits of sketches. (ii) We propose a deep embedding model for sketch zero-shot recognition, via collecting a large-scale edge-map dataset and proposing to extract a set of semantic vectors from edge-maps as the semantic knowledge for sketch zero-shot domain alignment. Both deep models are evaluated by comprehensive experiments on million-scale sketches and outperform the state-of-the-art competitors.

cs.CV↗

arXiv2020

From Weak Antilocalization to Kondo Scattering in a Magnetic Complex Oxide Interface

Quantum corrections to electrical resistance can serve as sensitive probes of the magnetic landscape of a material. For example, interference between time-reversed electron paths gives rise to weak localization effects, which can provide information about the coupling between spins and orbital motion, while the Kondo effect is sensitive to the presence of spin impurities. Here we use low-temperature magnetotransport measurements to reveal a transition from weak antilocalization (WAL) to Kondo scattering in the quasi-two-dimensional electron gas formed at the interface between SrTiO$_3$ and the Mott insulator NdTiO$_3$. This transition occurs as the thickness of the NdTiO$_3$ layer is increased. Analysis of the Kondo scattering and WAL points to the presence of atomic-scale magnetic impurities coexisting with extended magnetic regions that affect transport via a strong magnetic exchange interaction. This leads to distinct magnetoresistance behaviors that can serve as a sensitive probe of magnetic properties in two dimensions.

cond-mat.mtrl-sci↗

arXiv2020

Fault Detection and Identification using Bayesian Recurrent Neural Networks

In processing and manufacturing industries, there has been a large push to produce higher quality products and ensure maximum efficiency of processes. This requires approaches to effectively detect and resolve disturbances to ensure optimal operations. While the control system can compensate for many types of disturbances, there are changes to the process which it still cannot handle adequately. It is therefore important to further develop monitoring systems to effectively detect and identify those faults such that they can be quickly resolved by operators. In this paper, a novel probabilistic fault detection and identification method is proposed which adopts a newly developed deep learning approach using Bayesian recurrent neural networks~(BRNNs) with variational dropout. The BRNN model is general and can model complex nonlinear dynamics. Moreover, compared to traditional statistic-based data-driven fault detection and identification methods, the proposed BRNN-based method yields uncertainty estimates which allow for simultaneous fault detection of chemical processes, direct fault identification, and fault propagation analysis. The outstanding performance of this method is demonstrated and contrasted to (dynamic) principal component analysis, which are widely applied in the industry, in the benchmark Tennessee Eastman process~(TEP) and a real chemical manufacturing dataset.

stat.ML↗

arXiv2020

Fault Tolerant Free Gait and Footstep Planning for Hexapod Robot Based on Monte-Carlo Tree

Legged robots can pass through complex field environments by selecting gaits and discrete footholds carefully. Traditional methods plan gait and foothold separately and treat them as the single-step optimal process. However, such processing causes its poor passability in a sparse foothold environment. This paper novelly proposes a coordinative planning method for hexapod robots that regards the planning of gait and foothold as a sequence optimization problem with the consideration of dealing with the harshness of the environment as leg fault. The Monte Carlo tree search algorithm(MCTS) is used to optimize the entire sequence. Two methods, FastMCTS, and SlidingMCTS are proposed to solve some defeats of the standard MCTS applicating in the field of legged robot planning. The proposed planning algorithm combines the fault-tolerant gait method to improve the passability of the algorithm. Finally, compared with other planning methods, experiments on terrains with different densities of footholds and artificially-designed challenging terrain are carried out to verify our methods. All results show that the proposed method dramatically improves the hexapod robot's ability to pass through sparse footholds environment.

cs.RO↗

arXiv2020

Meta-Transfer Learning for Code-Switched Speech Recognition

An increasing number of people in the world today speak a mixed-language as a result of being multilingual. However, building a speech recognition system for code-switching remains difficult due to the availability of limited resources and the expense and significant effort required to collect mixed-language data. We therefore propose a new learning method, meta-transfer learning, to transfer learn on a code-switched speech recognition system in a low-resource setting by judiciously extracting information from high-resource monolingual datasets. Our model learns to recognize individual languages, and transfer them so as to better recognize mixed-language speech by conditioning the optimization on the code-switching data. Based on experimental results, our model outperforms existing baselines on speech recognition and language modeling tasks, and is faster to converge.

cs.CL↗

arXiv2020

CAiRE: An Empathetic Neural Chatbot

In this paper, we present an end-to-end empathetic conversation agent CAiRE. Our system adapts TransferTransfo (Wolf et al., 2019) learning approach that fine-tunes a large-scale pre-trained language model with multi-task objectives: response language modeling, response prediction and dialogue emotion detection. We evaluate our model on the recently proposed empathetic-dialogues dataset (Rashkin et al., 2019), the experiment results show that CAiRE achieves state-of-the-art performance on dialogue emotion detection and empathetic response generation.

cs.CL↗

arXiv2020

Coach: A Coarse-to-Fine Approach for Cross-domain Slot Filling

As an essential task in task-oriented dialog systems, slot filling requires extensive training data in a certain domain. However, such data are not always available. Hence, cross-domain slot filling has naturally arisen to cope with this data scarcity problem. In this paper, we propose a Coarse-to-fine approach (Coach) for cross-domain slot filling. Our model first learns the general pattern of slot entities by detecting whether the tokens are slot entities or not. It then predicts the specific types for the slot entities. In addition, we propose a template regularization approach to improve the adaptation robustness by regularizing the representation of utterances based on utterance templates. Experimental results show that our model significantly outperforms state-of-the-art approaches in slot filling. Furthermore, our model can also be applied to the cross-domain named entity recognition task, and it achieves better adaptation performance than other existing baselines. The code is available at https://github.com/zliucr/coach.

cs.CL↗

arXiv2020

On certain Iwahori--Hecke modules of $GL_3$ in characteristic $p$

In this note, we show that the natural analogue of certain finiteness result of Barthel--Livn$\acute{\text{e}}$ on $GL_2$ fails for $GL_3$. More precisely, within the pro-$p$-Iwahori invariants of a maximal compact induction of $GL_3$, we show there exist non-zero Iwahori--Hecke submodules of \emph{infinite} codimension.

math.RT↗

arXiv2020

LUDA: Boost LSM Key Value Store Compactions with GPUs

Log-Structured-Merge (LSM) tree-based key value stores are facing critical challenges of fully leveraging the dramatic performance improvements of the underlying storage devices, which makes the compaction operations of LSM key value stores become CPU-bound, and slow compactions significantly degrade key value store performance. To address this issue, we propose LUDA, an LSM key value store with CUDA, which uses a GPU to accelerate compaction operations of LSM key value stores. How to efficiently parallelize compaction procedures as well as accommodate the optimal performance contract of the GPU architecture challenge LUDA. Specifically, LUDA overcomes these challenges by exploiting the data independence between compaction procedures and using cooperative sort mechanism and judicious data movements. Running on a commodity GPU under different levels of CPU overhead, evaluation results show that LUDA provides up to 2x higher throughput and 2x data processing speed, and achieves more stable 99th percentile latencies than LevelDB and RocksDB.

cs.DC↗

arXiv2020

Variational Transformers for Diverse Response Generation

Despite the great promise of Transformers in many sequence modeling tasks (e.g., machine translation), their deterministic nature hinders them from generalizing to high entropy tasks such as dialogue response generation. Previous work proposes to capture the variability of dialogue responses with a recurrent neural network (RNN)-based conditional variational autoencoder (CVAE). However, the autoregressive computation of the RNN limits the training efficiency. Therefore, we propose the Variational Transformer (VT), a variational self-attentive feed-forward sequence model. The VT combines the parallelizability and global receptive field of the Transformer with the variational nature of the CVAE by incorporating stochastic latent variables into Transformers. We explore two types of the VT: 1) modeling the discourse-level diversity with a global latent variable; and 2) augmenting the Transformer decoder with a sequence of fine-grained latent variables. Then, the proposed models are evaluated on three conversational datasets with both automatic metric and human evaluation. The experimental results show that our models improve standard Transformers and other baselines in terms of diversity, semantic relevance, and human judgment.

cs.CL↗