arXiv subjects

Liang Huang

Publications and source records attributed to Liang Huang.

At least 109 records · Page 6Linked to original sources

arXiv2018

Ensemble Sequence Level Training for Multimodal MT: OSU-Baidu WMT18 Multimodal Machine Translation System Report

This paper describes multimodal machine translation systems developed jointly by Oregon State University and Baidu Research for WMT 2018 Shared Task on multimodal translation. In this paper, we introduce a simple approach to incorporate image information by feeding image features to the decoder side. We also explore different sequence level training methods including scheduled sampling and reinforcement learning which lead to substantial improvements. Our systems ensemble several models using different architectures and training methods and achieve the best performance for three subtasks: En-De and En-Cs in task 1 and (En+De+Fr)-Cs task 1B.

cs.CL↗

arXiv2018

When to Finish? Optimal Beam Search for Neural Text Generation (modulo beam size)

In neural text generation such as neural machine translation, summarization, and image captioning, beam search is widely used to improve the output text quality. However, in the neural generation setting, hypotheses can finish in different steps, which makes it difficult to decide when to end beam search to ensure optimality. We propose a provably optimal beam search algorithm that will always return the optimal-score complete hypothesis (modulo beam size), and finish as soon as the optimality is established (finishing no later than the baseline). To counter neural generation's tendency for shorter hypotheses, we also introduce a bounded length reward mechanism which allows a modified version of our beam search algorithm to remain optimal. Experiments on neural machine translation demonstrate that our principled beam search algorithm leads to improvement in BLEU score over previously proposed alternatives.

cs.CL↗

arXiv2018

Multi-Reference Training with Pseudo-References for Neural Translation and Text Generation

Neural text generation, including neural machine translation, image captioning, and summarization, has been quite successful recently. However, during training time, typically only one reference is considered for each example, even though there are often multiple references available, e.g., 4 references in NIST MT evaluations, and 5 references in image captioning data. We first investigate several different ways of utilizing multiple human references during training. But more importantly, we then propose an algorithm to generate exponentially many pseudo-references by first compressing existing human references into lattices and then traversing them to generate new pseudo-references. These approaches lead to substantial improvements over strong baselines in both machine translation (+1.5 BLEU) and image captioning (+3.1 BLEU / +11.7 CIDEr).

cs.CL↗

arXiv2018

Large Margin Neural Language Model

We propose a large margin criterion for training neural language models. Conventionally, neural language models are trained by minimizing perplexity (PPL) on grammatical sentences. However, we demonstrate that PPL may not be the best metric to optimize in some tasks, and further propose a large margin formulation. The proposed method aims to enlarge the margin between the "good" and "bad" sentences in a task-specific sense. It is trained end-to-end and can be widely applied to tasks that involve re-scoring of generated text. Compared with minimum-PPL training, our method gains up to 1.1 WER reduction for speech recognition and 1.0 BLEU increase for machine translation.

cs.CL↗

arXiv2018

Linear-Time Constituency Parsing with RNNs and Dynamic Programming

Recently, span-based constituency parsing has achieved competitive accuracies with extremely simple models by using bidirectional RNNs to model "spans". However, the minimal span parser of Stern et al (2017a) which holds the current state of the art accuracy is a chart parser running in cubic time, $O(n^3)$, which is too slow for longer sentences and for applications beyond sentence boundaries such as end-to-end discourse parsing and joint sentence boundary detection and parsing. We propose a linear-time constituency parser with RNNs and dynamic programming using graph-structured stack and beam search, which runs in time $O(n b^2)$ where $b$ is the beam size. We further speed this up to $O(n b\log b)$ by integrating cube pruning. Compared with chart parsing baselines, this linear-time parser is substantially faster for long sentences on the Penn Treebank and orders of magnitude faster for discourse parsing, and achieves the highest F1 accuracy on the Penn Treebank among single model end-to-end systems.

cs.CL↗

arXiv2018

Chaos in Dirac electron optics: Emergence of a relativistic quantum chimera

We uncover a remarkable quantum scattering phenomenon in two-dimensional Dirac material systems where the manifestations of both classically integrable and chaotic dynamics emerge simultaneously and are electrically controllable. The distinct relativistic quantum fingerprints associated with different electron spin states are due to a physical mechanism analogous to chiroptical effect in the presence of degeneracy breaking. The phenomenon mimics a chimera state in classical complex dynamical systems but here in a relativistic quantum setting - henceforth the term "Dirac quantum chimera," associated with which are physical phenomena with potentially significant applications such as enhancement of spin polarization, unusual coexisting quasibound states for distinct spin configurations, and spin selective caustics. Experimental observations of these phenomena are possible through, e.g., optical realizations of ballistic Dirac fermion systems.

quant-ph↗

arXiv2018

Event Nugget Detection with Forward-Backward Recurrent Neural Networks

Traditional event detection methods heavily rely on manually engineered rich features. Recent deep learning approaches alleviate this problem by automatic feature engineering. But such efforts, like tradition methods, have so far only focused on single-token event mentions, whereas in practice events can also be a phrase. We instead use forward-backward recurrent neural networks (FBRNNs) to detect events that can be either words or phrases. To the best our knowledge, this is one of the first efforts to handle multi-word events and also the first attempt to use RNNs for event detection. Experimental results demonstrate that FBRNN is competitive with the state-of-the-art methods on the ACE 2005 and the Rich ERE 2015 event detection tasks.

cs.CL↗

arXiv2018

Artificial intelligence meets minority game: toward optimal resource allocation

Resource allocation systems provide the fundamental support for the normal functioning and well being of the modern society, and can be modeled as minority games. A ubiquitous dynamical phenomenon is the emergence of herding, where a vast majority of the users concentrate on a small number of resources, leading to a low efficiency in resource allocation. To devise strategies to prevent herding is thus of high interest. Previous works focused on control strategies that rely on external interventions, such as pinning control where a fraction of users are forced to choose a certain action. Is it possible to eliminate herding without any external control? The main point of this paper is to provide an affirmative answer through exploiting artificial intelligence (AI). In particular, we demonstrate that, when agents are empowered with reinforced learning in that they get familiar with the unknown game environment gradually and attempt to deliver the optimal actions to maximize the payoff, herding can effectively be eliminated. Computations reveal the striking phenomenon that, regardless of the initial state, the system evolves persistently and relentlessly toward the optimal state in which all resources are used efficiently. However, the evolution process is not without interruptions: there are large fluctuations that occur but only intermittently in time. The statistical distribution of the time between two successive fluctuating events is found to depend on the parity of the evolution, i.e., whether the number of time steps in between is odd or even. We develop a physical analysis and derive mean-field equations to gain an understanding of these phenomena. As minority game dynamics and the phenomenon of herding are common in social, economic, and political systems, and since AI is becoming increasingly widespread, we expect our AI empowered minority game system to have broad applications.

physics.soc-ph↗

arXiv2017

OSU Multimodal Machine Translation System Report

This paper describes Oregon State University's submissions to the shared WMT'17 task "multimodal translation task I". In this task, all the sentence pairs are image captions in different languages. The key difference between this task and conventional machine translation is that we have corresponding images as additional information for each sentence pair. In this paper, we introduce a simple but effective system which takes an image shared between different languages, feeding it into the both encoding and decoding side. We report our system's performance for English-French and English-German with Flickr30K (in-domain) and MSCOCO (out-of-domain) datasets. Our system achieves the best performance in TER for English-German for MSCOCO dataset.

cs.CL↗

arXiv2017

Group Sparse CNNs for Question Classification with Answer Sets

Question classification is an important task with wide applications. However, traditional techniques treat questions as general sentences, ignoring the corresponding answer data. In order to consider answer information into question modeling, we first introduce novel group sparse autoencoders which refine question representation by utilizing group information in the answer set. We then propose novel group sparse CNNs which naturally learn question representation with respect to their answers by implanting group sparse autoencoders into traditional CNNs. The proposed model significantly outperform strong baselines on four datasets.

cs.CL↗

arXiv2017

Jointly Trained Sequential Labeling and Classification by Sparse Attention Neural Networks

Sentence-level classification and sequential labeling are two fundamental tasks in language understanding. While these two tasks are usually modeled separately, in reality, they are often correlated, for example in intent classification and slot filling, or in topic classification and named-entity recognition. In order to utilize the potential benefits from their correlations, we propose a jointly trained model for learning the two tasks simultaneously via Long Short-Term Memory (LSTM) networks. This model predicts the sentence-level category and the word-level label sequence from the stepwise output hidden representations of LSTM. We also introduce a novel mechanism of "sparse attention" to weigh words differently based on their semantic relevance to sentence-level classification. The proposed method outperforms baseline models on ATIS and TREC datasets.

cs.CL↗

arXiv2017

Non-equilibrium transport in the pseudospin-1 Dirac-Weyl system

Solid state materials hosting pseudospin-1 quasiparticles have attracted a great deal of recent attention. In these materials, the energy band contains of a pair of Dirac cones and a flat band through the connecting point of the cones. As the "caging" of carriers with a zero group velocity, the flat band itself has zero conductivity. However, in a non-equilibrium situation where a constant electric field is suddenly switched on, the flat band can enhance the resulting current in both the linear and nonlinear response regimes through distinct physical mechanisms. Using the ($2+1$) dimensional pseudospin-$1$ Dirac-Weyl system as a concrete setting, we demonstrate that, in the weak field regime, the interband current is about twice larger than that for pseudospin-1/2 system due to the interplay between the flat band and the negative band, with the scaling behavior determined by the Kubo formula. In the strong field regime, the intraband current is $\sqrt{2}$ times larger than that in the pseudospin-1/2 system, due to the additional contribution from particles residing in the flat band. In this case, the current and field follows the scaling law associated with Landau-Zener tunneling. These results provide a better understanding of the role of the flat band in non-equilibrium transport and are experimentally testable using electronic or photonic systems.

cond-mat.mes-hall↗

arXiv2017

Fast(er) Exact Decoding and Global Training for Transition-Based Dependency Parsing via a Minimal Feature Set

We first present a minimal feature set for transition-based dependency parsing, continuing a recent trend started by Kiperwasser and Goldberg (2016a) and Cross and Huang (2016a) of using bi-directional LSTM features. We plug our minimal feature set into the dynamic-programming framework of Huang and Sagae (2010) and Kuhlmann et al. (2011) to produce the first implementation of worst-case O(n^3) exact decoders for arc-hybrid and arc-eager transition systems. With our minimal features, we also present O(n^3) global training methods. Finally, using ensembles including our new parsers, we achieve the best unlabeled attachment score reported (to our knowledge) on the Chinese Treebank and the "second-best-in-class" result on the English Penn Treebank.

cs.CL↗

arXiv2017

Joint Syntacto-Discourse Parsing and the Syntacto-Discourse Treebank

Discourse parsing has long been treated as a stand-alone problem independent from constituency or dependency parsing. Most attempts at this problem are pipelined rather than end-to-end, sophisticated, and not self-contained: they assume gold-standard text segmentations (Elementary Discourse Units), and use external parsers for syntactic features. In this paper we propose the first end-to-end discourse parser that jointly parses in both syntax and discourse levels, as well as the first syntacto-discourse treebank by integrating the Penn Treebank with the RST Treebank. Built upon our recent span-based constituency parser, this joint syntacto-discourse parser requires no preprocessing whatsoever (such as segmentation or feature extraction), achieves the state-of-the-art end-to-end discourse parsing accuracy.

cs.CL↗

arXiv2017

Scars in Dirac fermion systems: the influence of an Aharonov--Bohm flux

Time-reversal ($\mathcal{T}$-) symmetry is fundamental to many physical processes. Typically, $\mathcal{T}$-breaking for microscopic processes requires the presence of magnetic field. However, for 2D massless Dirac billiards, $\mathcal{T}$-symmetry is broken automatically by the mass confinement, leading to chiral quantum scars. In this paper, we investigate the mechanism of $\mathcal{T}$-breaking by analyzing the local current of the scarring eigenstates and their magnetic response to an Aharonov--Bohm flux. Our results unveil the complete understanding of the subtle $\mathcal{T}$-breaking phenomena from both the semiclassical formula of chiral scars and the microscopic current and spin reflection at the boundaries, leading to a controlling scheme to change the chirality of the relativistic quantum scars. Our findings not only have significant implications on the transport behavior and spin textures of the relativistic pseudoparticles, but also add basic knowledge to relativistic quantum chaos.

quant-ph↗

arXiv2017

Low Power SI Class E Power Amplifier and RF Switch For Health Care

This research was to design a 2.4 GHz class E Power Amplifier (PA) for health care, with 0.18um Semiconductor Manufacturing International Corporation CMOS technology by using Cadence software. And also RF switch was designed at cadence software with power Jazz 180nm SOI process. The ultimate goal for such application is to reach high performance and low cost, and between high performance and low power consumption design. This paper introduces the design of a 2.4GHz class E power amplifier and RF switch design. PA consists of cascade stage with negative capacitance. This power amplifier can transmit 16dBm output power to a 50Ω load. The performance of the power amplifier and switch meet the specification requirements of the desired.

cs.OH↗

arXiv2017

Textual Entailment with Structured Attentions and Composition

Deep learning techniques are increasingly popular in the textual entailment task, overcoming the fragility of traditional discrete models with hard alignments and logics. In particular, the recently proposed attention models (Rocktäschel et al., 2015; Wang and Jiang, 2015) achieves state-of-the-art accuracy by computing soft word alignments between the premise and hypothesis sentences. However, there remains a major limitation: this line of work completely ignores syntax and recursion, which is helpful in many traditional efforts. We show that it is beneficial to extend the attention model to tree nodes between premise and hypothesis. More importantly, this subtree-level attention reveals information about entailment relation. We study the recursive composition of this subtree-level entailment relation, which can be viewed as a soft version of the Natural Logic framework (MacCartney and Manning, 2009). Experiments show that our structured attention and entailment composition model can correctly identify and infer entailment relations from the bottom up, and bring significant improvements in accuracy.

cs.CL↗

arXiv2016

Detecting and characterizing high frequency oscillations in epilepsy - A case study of big data analysis

We develop a framework to uncover and analyze dynamical anomalies from massive, nonlinear and non-stationary time series data. The framework consists of three steps: preprocessing of massive data sets to eliminate erroneous data segments, application of the empirical mode decomposition and Hilbert transform paradigm to obtain the fundamental components embedded in the time series at distinct time scales, and statistical/scaling analysis of the components. As a case study, we apply our framework to detecting and characterizing high frequency oscillations (HFOs) from a big database of rat EEG recordings. We find a striking phenomenon: HFOs exhibit on-off intermittency that can be quantified by algebraic scaling laws. Our framework can be generalized to big data-related problems in other fields such as large-scale sensor data and seismic data analysis.

physics.med-ph↗