arXiv subjects

Hang Li

Publications and source records attributed to Hang Li.

At least 343 records · Page 19Linked to original sources

arXiv2016

Neural Enquirer: Learning to Query Tables with Natural Language

We proposed Neural Enquirer as a neural network architecture to execute a natural language (NL) query on a knowledge-base (KB) for answers. Basically, Neural Enquirer finds the distributed representation of a query and then executes it on knowledge-base tables to obtain the answer as one of the values in the tables. Unlike similar efforts in end-to-end training of semantic parsers, Neural Enquirer is fully "neuralized": it not only gives distributional representation of the query and the knowledge-base, but also realizes the execution of compositional queries as a series of differentiable operations, with intermediate results (consisting of annotations of the tables at different levels) saved on multiple layers of memory. Neural Enquirer can be trained with gradient descent, with which not only the parameters of the controlling components and semantic parsing component, but also the embeddings of the tables and query words can be learned from scratch. The training can be done in an end-to-end fashion, but it can take stronger guidance, e.g., the step-by-step supervision for complicated queries, and benefit from it. Neural Enquirer is one step towards building neural network systems which seek to understand language by executing it on real-world. Our experiments show that Neural Enquirer can learn to execute fairly complicated NL queries on tables with rich structures.

cs.AI↗

arXiv2016

A Deep Memory-based Architecture for Sequence-to-Sequence Learning

We propose DEEPMEMORY, a novel deep architecture for sequence-to-sequence learning, which performs the task through a series of nonlinear transformations from the representation of the input sequence (e.g., a Chinese sentence) to the final output sequence (e.g., translation to English). Inspired by the recently proposed Neural Turing Machine (Graves et al., 2014), we store the intermediate representations in stacked layers of memories, and use read-write operations on the memories to realize the nonlinear transformations between the representations. The types of transformations are designed in advance but the parameters are learned from data. Through layer-by-layer transformations, DEEPMEMORY can model complicated relations between sequences necessary for applications such as machine translation between distant languages. The architecture can be trained with normal back-propagation on sequenceto-sequence data, and the learning can be easily scaled up to a large corpus. DEEPMEMORY is broad enough to subsume the state-of-the-art neural translation model in (Bahdanau et al., 2015) as its special case, while significantly improving upon the model with its deeper architecture. Remarkably, DEEPMEMORY, being purely neural network-based, can achieve performance comparable to the traditional phrase-based machine translation system Moses with a small vocabulary and a modest parameter size.

cs.CL↗

arXiv2015

Performance Analysis for Energy Harvesting Communication Systems: From Throughput to Energy Diversity

Energy harvesting (EH) based communication has raised great research interests due to its wide application and the feasibility of commercialization. In this paper, we consider wireless communications with EH constraints at the transmitter. First, for delay-tolerant traffic, we investigate the long-term average throughput maximization problem and analytically compare the throughput performance against that of a system supported by conventional power supplies. Second, for delay-sensitive traffic, we analyze the outage probability by studying its asymptotic behavior in the high energy arrival rate regime, where the new concept of energy diversity is formally introduced. Moreover, we show that the speed of outage probability approaching zero, termed energy diversity gain, varies under different power supply models.

cs.IT↗

arXiv2015

Learning to Answer Questions From Image Using Convolutional Neural Network

In this paper, we propose to employ the convolutional neural network (CNN) for the image question answering (QA). Our proposed CNN provides an end-to-end framework with convolutional architectures for learning not only the image and question representations, but also their inter-modal interactions to produce the answer. More specifically, our model consists of three CNNs: one image CNN to encode the image content, one sentence CNN to compose the words of the question, and one multimodal convolution layer to learn their joint representation for the classification in the space of candidate answer words. We demonstrate the efficacy of our proposed model on the DAQUAR and COCO-QA datasets, which are two benchmark datasets for the image QA, with the performances significantly outperforming the state-of-the-art.

cs.CL↗

arXiv2015

Valley-dependent spin-orbit torques in two dimensional hexagonal crystals

We study spin-orbit torques in two dimensional hexagonal crystals such as graphene, silicene, germanene and stanene. The torque possesses two components, a field-like term due to inverse spin galvanic effect and an antidamping torque originating from Berry curvature in mixed spin-$k$ space. In the presence of staggered potential and exchange field, the valley degeneracy can be lifted and we obtain a valley-dependent Berry curvature, leading to a tunable antidamping torque by controlling the valley degree of freedom. The valley imbalance can be as high as 100\% by tuning the bias voltage or magnetization angle. These findings open new venues for the development of current-driven spin-orbit torques by structural design.

cond-mat.mes-hall↗

arXiv2015

Multimodal Convolutional Neural Networks for Matching Image and Sentence

In this paper, we propose multimodal convolutional neural networks (m-CNNs) for matching image and sentence. Our m-CNN provides an end-to-end framework with convolutional architectures to exploit image representation, word composition, and the matching relations between the two modalities. More specifically, it consists of one image CNN encoding the image content, and one matching CNN learning the joint representation of image and sentence. The matching CNN composes words to different semantic fragments and learns the inter-modal relations between image and the composed fragments at different levels, thus fully exploit the matching relations between image and sentence. Experimental results on benchmark databases of bidirectional image and sentence retrieval demonstrate that the proposed m-CNNs can effectively capture the information necessary for image and sentence matching. Specifically, our proposed m-CNNs for bidirectional image and sentence retrieval on Flickr30K and Microsoft COCO databases achieve the state-of-the-art performances.

cs.CV↗

arXiv2015

Towards Neural Network-based Reasoning

We propose Neural Reasoner, a framework for neural network-based reasoning over natural language sentences. Given a question, Neural Reasoner can infer over multiple supporting facts and find an answer to the question in specific forms. Neural Reasoner has 1) a specific interaction-pooling mechanism, allowing it to examine multiple facts, and 2) a deep architecture, allowing it to model the complicated logical relations in reasoning tasks. Assuming no particular structure exists in the question and facts, Neural Reasoner is able to accommodate different types of reasoning and different forms of language expressions. Despite the model complexity, Neural Reasoner can still be trained effectively in an end-to-end manner. Our empirical studies show that Neural Reasoner can outperform existing neural reasoning systems with remarkable margins on two difficult artificial tasks (Positional Reasoning and Path Finding) proposed in [8]. For example, it improves the accuracy on Path Finding(10K) from 33.4% [6] to over 98%.

cs.AI↗

arXiv2015

Context-Dependent Translation Selection Using Convolutional Neural Network

We propose a novel method for translation selection in statistical machine translation, in which a convolutional neural network is employed to judge the similarity between a phrase pair in two languages. The specifically designed convolutional architecture encodes not only the semantic similarity of the translation pair, but also the context containing the phrase in the source language. Therefore, our approach is able to capture context-dependent semantic similarities of translation pairs. We adopt a curriculum learning strategy to train the model: we classify the training examples into easy, medium, and difficult categories, and gradually build the ability of representing phrase and sentence level context by using training examples from easy to difficult. Experimental results show that our approach significantly outperforms the baseline system by up to 1.4 BLEU points.

cs.CL↗

arXiv2015

Syntax-based Deep Matching of Short Texts

Many tasks in natural language processing, ranging from machine translation to question answering, can be reduced to the problem of matching two sentences or more generally two short texts. We propose a new approach to the problem, called Deep Match Tree (DeepMatch$_{tree}$), under a general setting. The approach consists of two components, 1) a mining algorithm to discover patterns for matching two short-texts, defined in the product space of dependency trees, and 2) a deep neural network for matching short texts using the mined patterns, as well as a learning algorithm to build the network having a sparse structure. We test our algorithm on the problem of matching a tweet and a response in social media, a hard matching problem proposed in [Wang et al., 2013], and show that DeepMatch$_{tree}$ can outperform a number of competitor models including one without using dependency trees and one based on word-embedding, all with large margins

cs.CL↗

arXiv2015

Encoding Source Language with Convolutional Neural Network for Machine Translation

The recently proposed neural network joint model (NNJM) (Devlin et al., 2014) augments the n-gram target language model with a heuristically chosen source context window, achieving state-of-the-art performance in SMT. In this paper, we give a more systematic treatment by summarizing the relevant source information through a convolutional architecture guided by the target information. With different guiding signals during decoding, our specifically designed convolution+gating architectures can pinpoint the parts of a source sentence that are relevant to predicting a target word, and fuse them with the context of entire source sentence to form a unified representation. This representation, together with target language words, are fed to a deep neural network (DNN) to form a stronger NNJM. Experiments on two NIST Chinese-English translation tasks show that the proposed model can achieve significant improvements over the previous NNJM by up to +1.08 BLEU points on average

cs.CL↗

arXiv2015

Opportunistic Multi-Channel Access in Heterogeneous 5G Network with Renewable Energy Supplies

A heterogeneous system, where small networks (e.g., small cell or WiFi) boost the system throughput under the umbrella of a large network (e.g., large cell), is a promising architecture for the 5G wireless communication networks, where green and sustainable communication is also a key aspect. Renewable energy based communication via energy harvesting (EH) devices is one of such green technology candidates. In this paper, we study an uplink transmission scenario under a heterogeneous network hierarchy, where each mobile user (MU) is powered by a sustainable energy supply, capable of both deterministic access to the large network via one private channel, and dynamic access to a small network with certain probability via one common channel shared by multiple MUs. Considering a general EH model, i.e., energy arrivals are time-correlated, we study an opportunistic transmission scheme and aim to maximize the average throughput for each MU, which jointly exploits the statistics and current states of the private channel, common channel, battery level, and EH rate. Applying a simple yet efficient "save-then-transmit" scheme, the throughput maximization problem is cast as a "rate-of-return" optimal stopping problem. The optimal stopping rule is proved to has a time-dependent threshold-based structure for the case with general Markovian system dynamics, and degrades to a pure threshold policy for the case with independent and identically distributed system dynamics. As performance benchmarks, the optimal power allocation scheme with conventional power supplies is also examined. Finally, numerical results are presented, and a new concept of "EH diversity" is discussed.

cs.IT↗

arXiv2015

Distributed Opportunistic Scheduling for Energy Harvesting Based Wireless Networks: A Two-Stage Probing Approach

This paper considers a heterogeneous ad hoc network with multiple transmitter-receiver pairs, in which all transmitters are capable of harvesting renewable energy from the environment and compete for one shared channel by random access. In particular, we focus on two different scenarios: the constant energy harvesting (EH) rate model where the EH rate remains constant within the time of interest and the i.i.d. EH rate model where the EH rates are independent and identically distributed across different contention slots. To quantify the roles of both the energy state information (ESI) and the channel state information (CSI), a distributed opportunistic scheduling (DOS) framework with two-stage probing and save-then-transmit energy utilization is proposed. Then, the optimal throughput and the optimal scheduling strategy are obtained via one-dimension search, i.e., an iterative algorithm consisting of the following two steps in each iteration: First, assuming that the stored energy level at each transmitter is stationary with a given distribution, the expected throughput maximization problem is formulated as an optimal stopping problem, whose solution is proved to exist and then derived for both models; second, for a fixed stopping rule, the energy level at each transmitter is shown to be stationary and an efficient iterative algorithm is proposed to compute its steady-state distribution. Finally, we validate our analysis by numerical results and quantify the throughput gain compared with the best-effort delivery scheme.

cs.IT↗

arXiv2015

Reader-Aware Multi-Document Summarization via Sparse Coding

We propose a new MDS paradigm called reader-aware multi-document summarization (RA-MDS). Specifically, a set of reader comments associated with the news reports are also collected. The generated summaries from the reports for the event should be salient according to not only the reports but also the reader comments. To tackle this RA-MDS problem, we propose a sparse-coding-based method that is able to calculate the salience of the text units by jointly considering news reports and reader comments. Another reader-aware characteristic of our framework is to improve linguistic quality via entity rewriting. The rewriting consideration is jointly assessed together with other summarization requirements under a unified optimization model. To support the generation of compressive summaries via optimization, we explore a finer syntactic unit, namely, noun/verb phrase. In this work, we also generate a data set for conducting RA-MDS. Extensive experiments on this data set and some classical data sets demonstrate the effectiveness of our proposed approach.

cs.CL↗

arXiv2015

Neural Responding Machine for Short-Text Conversation

We propose Neural Responding Machine (NRM), a neural network-based response generator for Short-Text Conversation. NRM takes the general encoder-decoder framework: it formalizes the generation of response as a decoding process based on the latent representation of the input text, while both encoding and decoding are realized with recurrent neural networks (RNN). The NRM is trained with a large amount of one-round conversation data collected from a microblogging service. Empirical study shows that NRM can generate grammatically correct and content-wise appropriate responses to over 75% of the input text, outperforming state-of-the-arts in the same setting, including retrieval-based and SMT-based models.

cs.CL↗

arXiv2015

$gen$CNN: A Convolutional Architecture for Word Sequence Prediction

We propose a novel convolutional architecture, named $gen$CNN, for word sequence prediction. Different from previous work on neural network-based language modeling and generation (e.g., RNN or LSTM), we choose not to greedily summarize the history of words as a fixed length vector. Instead, we use a convolutional neural network to predict the next word with the history of words of variable length. Also different from the existing feedforward networks for language modeling, our model can effectively fuse the local correlation and global correlation in the word sequence, with a convolution-gating strategy specifically designed for the task. We argue that our model can give adequate representation of the history, and therefore can naturally exploit both the short and long range dependencies. Our model is fast, easy to train, and readily parallelized. Our extensive experiments on text generation and $n$-best re-ranking in machine translation show that $gen$CNN outperforms the state-of-the-arts with big margins.

cs.CL↗

arXiv2015

Convolutional Neural Network Architectures for Matching Natural Language Sentences

Semantic matching is of central importance to many natural language tasks \cite{bordes2014semantic,RetrievalQA}. A successful matching algorithm needs to adequately model the internal structures of language objects and the interaction between them. As a step toward this goal, we propose convolutional neural network models for matching two sentences, by adapting the convolutional strategy in vision and speech. The proposed models not only nicely represent the hierarchical structures of sentences with their layer-by-layer composition and pooling, but also capture the rich matching patterns at different levels. Our models are rather generic, requiring no prior knowledge on language, and can hence be applied to matching tasks of different nature and in different languages. The empirical study on a variety of matching tasks demonstrates the efficacy of the proposed model on a variety of matching tasks and its superiority to competitor models.

cs.CL↗

arXiv2015

Intraband and interband spin-orbit torques in non-centrosymmetric ferromagnets

Intraband and interband contributions to the current-driven spin-orbit torque in magnetic materials lacking inversion symmetry are theoretically studied using Kubo formula. In addition to the current-driven field-like torque ${\bf T}_{\rm FL}= τ_{\rm FL}{\bf m}\times{\bf u}_{\rm so}$ (${\bf u}_{\rm so}$ being a unit vector determined by the symmetry of the spin-orbit coupling), we explore the intrinsic contribution arising from impurity-independent interband transitions and producing an anti-damping-like torque of the form ${\bf T}_{\rm DL}= τ_{\rm DL}{\bf m}\times({\bf u}_{\rm so}\times{\bf m})$. Analytical expressions are obtained in the model case of a magnetic Rashba two-dimensional electron gas, while numerical calculations have been performed on a dilute magnetic semiconductor (Ga,Mn)As modeled by the Kohn-Luttinger Hamiltonian exchanged coupled to the Mn moments. Parametric dependences of the different torque components and similarities to the analytical results of the Rashba two-dimensional electron gas in the weak disorder limit are described.

cond-mat.mes-hall↗

arXiv2015

Collective magnetization dynamics in ferromagnetic (Ga,Mn)As mediated by photo-excited carriers

We present a study of photo-excited magnetization dynamics in ferromagnetic (Ga,Mn)As films observed by time-resolved magneto-optical measurements. The magnetization precession triggered by linearly polarized optical pulses in the absence of an external field shows a strong dependence on photon frequency when the photo-excitation energy approaches the band-edge of (Ga,Mn)As. This can be understood in terms of magnetic anisotropy modulation by both laser heating of the sample and by hole-induced non-thermal paths. Our findings provide a means for identifying the transition of laser-triggered magnetization dynamics from thermal to non-thermal mechanisms, a result that is of importance for ultrafast optical spin manipulation in ferromagnetic materials via non-thermal paths.

cond-mat.mes-hall↗