arXiv subjects

Hua Wu

Publications and source records attributed to Hua Wu.

At least 217 records · Page 12Linked to original sources

arXiv2019

End-to-End Speech Translation with Knowledge Distillation

End-to-end speech translation (ST), which directly translates from source language speech into target language text, has attracted intensive attentions in recent years. Compared to conventional pipeline systems, end-to-end ST models have advantages of lower latency, smaller model size and less error propagation. However, the combination of speech recognition and text translation in one model is more difficult than each of these two tasks. In this paper, we propose a knowledge distillation approach to improve ST model by transferring the knowledge from text translation model. Specifically, we first train a text translation model, regarded as a teacher model, and then ST model is trained to learn output probabilities from teacher model through knowledge distillation. Experiments on English- French Augmented LibriSpeech and English-Chinese TED corpus show that end-to-end ST is possible to implement on both similar and dissimilar language pairs. In addition, with the instruction of teacher model, end-to-end ST model can gain significant improvements by over 3.5 BLEU points.

cs.CL↗

arXiv2018

Modeling Coherence for Discourse Neural Machine Translation

Discourse coherence plays an important role in the translation of one text. However, the previous reported models most focus on improving performance over individual sentence while ignoring cross-sentence links and dependencies, which affects the coherence of the text. In this paper, we propose to use discourse context and reward to refine the translation quality from the discourse perspective. In particular, we generate the translation of individual sentences at first. Next, we deliberate the preliminary produced translations, and train the model to learn the policy that produces discourse coherent text by a reward teacher. Practical results on multiple discourse test datasets indicate that our model significantly improves the translation quality over the state-of-the-art baseline system by +1.23 BLEU score. Moreover, our model generates more discourse coherent text and obtains +2.2 BLEU improvements when evaluated by discourse metrics.

cs.CL↗

arXiv2018

Possible High TC Layered Ferromagnetic Insulator Sr2NiRuO4: An ab initio Study

Magnetic insulators are often antiferromagnetic (AFM) and layered AFM compounds usually show low ordering temperature. On the other hand, layered ferromagnetic (FM) insulators with high-TC are very rare although they could be quite useful for spintronic applications. Here, using crystal field level analysis in combination with density functional theory calculations as well as Monte Carlo simulations, we predict that the layered insulator Sr2NiRuO4 would have a strong FM coupling with TC as high as 240 K. The tetragonal crystal field in the Ni-O-Ru square plane stabilizes the S=1/2 Ni+ and S=3/2 Ru3+ states. The unique level ordering and occupation optimize the FM Ni-Ru superexchange interactions in the checkerboard arrangement, thus suggesting Sr2NiRuO4 as an unusual high-TC layered FM insulator. This work highlights the potential of charge-spin-orbital degrees of freedom for stabilizing strong FM coupling in layered oxides.

cond-mat.mtrl-sci↗

arXiv2018

Familia: A Configurable Topic Modeling Framework for Industrial Text Engineering

In the last decade, a variety of topic models have been proposed for text engineering. However, except Probabilistic Latent Semantic Analysis (PLSA) and Latent Dirichlet Allocation (LDA), most of existing topic models are seldom applied or considered in industrial scenarios. This phenomenon is caused by the fact that there are very few convenient tools to support these topic models so far. Intimidated by the demanding expertise and labor of designing and implementing parameter inference algorithms, software engineers are prone to simply resort to PLSA/LDA, without considering whether it is proper for their problem at hand or not. In this paper, we propose a configurable topic modeling framework named Familia, in order to bridge the huge gap between academic research fruits and current industrial practice. Familia supports an important line of topic models that are widely applicable in text engineering scenarios. In order to relieve burdens of software engineers without knowledge of Bayesian networks, Familia is able to conduct automatic parameter inference for a variety of topic models. Simply through changing the data organization of Familia, software engineers are able to easily explore a broad spectrum of existing topic models or even design their own topic models, and find the one that best suits the problem at hand. With its superior extendability, Familia has a novel sampling mechanism that strikes balance between effectiveness and efficiency of parameter inference. Furthermore, Familia is essentially a big topic modeling framework that supports parallel parameter inference and distributed parameter storage. The utilities and necessity of Familia are demonstrated in real-life industrial applications. Familia would significantly enlarge software engineers' arsenal of topic models and pave the way for utilizing highly customized topic models in real-life problems.

cs.CL↗

arXiv2018

DuReader: a Chinese Machine Reading Comprehension Dataset from Real-world Applications

This paper introduces DuReader, a new large-scale, open-domain Chinese ma- chine reading comprehension (MRC) dataset, designed to address real-world MRC. DuReader has three advantages over previous MRC datasets: (1) data sources: questions and documents are based on Baidu Search and Baidu Zhidao; answers are manually generated. (2) question types: it provides rich annotations for more question types, especially yes-no and opinion questions, that leaves more opportunity for the research community. (3) scale: it contains 200K questions, 420K answers and 1M documents; it is the largest Chinese MRC dataset so far. Experiments show that human performance is well above current state-of-the-art baseline systems, leaving plenty of room for the community to make improvements. To help the community make these improvements, both DuReader and baseline systems have been posted online. We also organize a shared competition to encourage the exploration of more models. Since the release of the task, there are significant improvements over the baselines.

cs.CL↗

arXiv2018

Multi-Passage Machine Reading Comprehension with Cross-Passage Answer Verification

Machine reading comprehension (MRC) on real web data usually requires the machine to answer a question by analyzing multiple passages retrieved by search engine. Compared with MRC on a single passage, multi-passage MRC is more challenging, since we are likely to get multiple confusing answer candidates from different passages. To address this problem, we propose an end-to-end neural model that enables those answer candidates from different passages to verify each other based on their content representations. Specifically, we jointly train three modules that can predict the final answer based on three factors: the answer boundary, the answer content and the cross-passage answer verification. The experimental results show that our method outperforms the baseline by a large margin and achieves the state-of-the-art performance on the English MS-MARCO dataset and the Chinese DuReader dataset, both of which are designed for MRC in real-world settings.

cs.CL↗

arXiv2018

Unusual layered order and charge disproportionation in double perovskite Ca2FeMnO6

While double perovskites A2BB'O6, if ordered, usually form a rock-salt-type structure with a checkerboard B/B' ordering, it is surprising that Ca2FeMnO6 has alternate FeO2 and MnO2 layers in its perovskite structure. Here we demonstrate, using density functional calculations, that this unusual layered ordering facilitates, and is largely helped by, the Fe3+-Fe5+ charge disproportionation (CD) of the formal Fe4+ ions, which would otherwise be frustrated in the common rock salt structure. To further verify the important role of the CD for stabilization of this layered ordering, we carry out a comparative study for the isostructural Ca2TiMnO6 which has a simple Ti4+-Mn4+ state free of the CD. Our calculations indicate that Ca2TiMnO6 instead prefers the standard rock salt structure to the layered one. Thus our study shows a nontrivial interplay between the CD and the type of ion ordering, and proves that the CD is strongly involved in stabilizing the unusual layered order of Ca2FeMnO6.

cond-mat.str-el↗

arXiv2017

Multi-channel Encoder for Neural Machine Translation

Attention-based Encoder-Decoder has the effective architecture for neural machine translation (NMT), which typically relies on recurrent neural networks (RNN) to build the blocks that will be lately called by attentive reader during the decoding process. This design of encoder yields relatively uniform composition on source sentence, despite the gating mechanism employed in encoding RNN. On the other hand, we often hope the decoder to take pieces of source sentence at varying levels suiting its own linguistic structure: for example, we may want to take the entity name in its raw form while taking an idiom as a perfectly composed unit. Motivated by this demand, we propose Multi-channel Encoder (MCE), which enhances encoding components with different levels of composition. More specifically, in addition to the hidden state of encoding RNN, MCE takes 1) the original word embedding for raw encoding with no composition, and 2) a particular design of external memory in Neural Turing Machine (NTM) for more complex composition, while all three encoding strategies are properly blended during decoding. Empirical study on Chinese-English translation shows that our model can improve by 6.52 BLEU points upon a strong open source NMT system: DL4MT1. On the WMT14 English- French task, our single shallow system achieves BLEU=38.8, comparable with the state-of-the-art deep models.

cs.CL↗

arXiv2017

Largely tunable band structures of few-layer InSe by uniaxial strain

Due to the strong quantum confinement effect, few-layer γ-InSe exhibits a layer-dependent bandgap, spanning the visible and near infrared regions, and thus recently draws tremendous attention. As a two-dimensional material, the mechanical flexibility provides an additional tuning knob for the electronic structure. Here, for the first time, we engineer the band structures of few-layer and bulk-like InSe by uniaxial tensile strain, and observe salient shift of photoluminescence (PL) peaks. The shift rate of the optical gap is approximately 90-100 meV per 1% strain for 4- to 8-layer samples, which is much larger than that for the widely studied MoS2 monolayer. Density functional calculations well reproduce the observed layer-dependent bandgaps and the strain effect, and reveal that the shift rate decreases with increasing layer number for few-layer InSe. Our study demonstrates that InSe is a very versatile 2D electronic and optoelectronic material, which is suitable for tunable light emitters, photo-detectors and other optoelectronic devices.

cond-mat.mes-hall↗

arXiv2017

The $c$-axis dimer and its electronic break-up: the insulator-to-metal transition in Ti$_2$O$_3$

We report on our investigation of the electronic structure of Ti$_2$O$_3$ using (hard) x-ray photoelectron and soft x-ray absorption spectroscopy. From the distinct satellite structures in the spectra we have been able to establish unambiguously that the Ti-Ti $c$-axis dimer in the corundum crystal structure is electronically present and forms an $a_{1g}a_{1g}$ molecular singlet in the low temperature insulating phase. Upon heating we observed a considerable spectral weight transfer to lower energies with orbital reconstruction. The insulator-metal transition may be viewed as a transition from a solid of isolated Ti-Ti molecules into a solid of electronically partially broken dimers where the Ti ions acquire additional hopping in the $a$-$b$ plane via the $e_g^π$ channel, the opening of which requires the consideration of the multiplet structure of the on-site Coulomb interaction.

cond-mat.str-el↗

arXiv2016

Semi-Supervised Learning for Neural Machine Translation

While end-to-end neural machine translation (NMT) has made remarkable progress recently, NMT systems only rely on parallel corpora for parameter estimation. Since parallel corpora are usually limited in quantity, quality, and coverage, especially for low-resource languages, it is appealing to exploit monolingual corpora to improve NMT. We propose a semi-supervised approach for training NMT models on the concatenation of labeled (parallel corpora) and unlabeled (monolingual corpora) data. The central idea is to reconstruct the monolingual corpora using an autoencoder, in which the source-to-target and target-to-source translation models serve as the encoder and decoder, respectively. Our approach can not only exploit the monolingual corpora of the target language, but also of the source language. Experiments on the Chinese-English dataset show that our approach achieves significant improvements over state-of-the-art SMT and NMT systems.

cs.CL↗

arXiv2016

Chinese Poetry Generation with Planning based Neural Network

Chinese poetry generation is a very challenging task in natural language processing. In this paper, we propose a novel two-stage poetry generating method which first plans the sub-topics of the poem according to the user's writing intent, and then generates each line of the poem sequentially, using a modified recurrent neural network encoder-decoder framework. The proposed planning-based method can ensure that the generated poem is coherent and semantically consistent with the user's intent. A comprehensive evaluation with human judgments demonstrates that our proposed approach outperforms the state-of-the-art poetry generating methods and the poem quality is somehow comparable to human poets.

cs.CL↗

arXiv2016

Minimum Risk Training for Neural Machine Translation

We propose minimum risk training for end-to-end neural machine translation. Unlike conventional maximum likelihood estimation, minimum risk training is capable of optimizing model parameters directly with respect to arbitrary evaluation metrics, which are not necessarily differentiable. Experiments show that our approach achieves significant improvements over maximum likelihood estimation on a state-of-the-art neural machine translation system across various languages pairs. Transparent to architectures, our approach can be applied to more neural networks and potentially benefit more NLP tasks.

cs.CL↗

arXiv2016

Question Answering over Knowledge Base with Neural Attention Combining Global Knowledge Information

With the rapid growth of knowledge bases (KBs) on the web, how to take full advantage of them becomes increasingly important. Knowledge base-based question answering (KB-QA) is one of the most promising approaches to access the substantial knowledge. Meantime, as the neural network-based (NN-based) methods develop, NN-based KB-QA has already achieved impressive results. However, previous work did not put emphasis on question representation, and the question is converted into a fixed vector regardless of its candidate answers. This simple representation strategy is unable to express the proper information of the question. Hence, we present a neural attention-based model to represent the questions dynamically according to the different focuses of various candidate answer aspects. In addition, we leverage the global knowledge inside the underlying KB, aiming at integrating the rich KB information into the representation of the answers. And it also alleviates the out of vocabulary (OOV) problem, which helps the attention model to represent the question more precisely. The experimental results on WEBQUESTIONS demonstrate the effectiveness of the proposed approach.

cs.IR↗

arXiv2016

Agreement-based Joint Training for Bidirectional Attention-based Neural Machine Translation

The attentional mechanism has proven to be effective in improving end-to-end neural machine translation. However, due to the intricate structural divergence between natural languages, unidirectional attention-based models might only capture partial aspects of attentional regularities. We propose agreement-based joint training for bidirectional attention-based end-to-end neural machine translation. Instead of training source-to-target and target-to-source translation models independently,our approach encourages the two complementary models to agree on word alignment matrices on the same training data. Experiments on Chinese-English and English-French translation tasks show that agreement-based joint training significantly improves both alignment and translation quality over independent training.

cs.CL↗

arXiv2015

Giant magnetic anisotropy of Co, Ru, and Os adatoms on MgO (001) surface

Large magnetic anisotropy energy (MAE) is desirable and critical for nanoscale magnetic devices. Here, using ligand-field level diagrams and density functional calculations, we well explain the very recent discovery [I. G. Rau et al., Science 344, 988 (2014)] that an individual Co adatom on a MgO (001) surface has a large MAE of more than 60 meV. More importantly, we predict that a giant MAE up to 110 meV could be realized for Ru adatoms on MgO (001), and even more for the Os adatoms (208 meV). This is a joint effect of the special ligand field, orbital multiplet, and significant spin-orbit interaction, in the intermediate-spin state of the Ru or Os adatoms on top of the surface oxygens. The giant MAE could provide a route to atomic scale memory.

cond-mat.mtrl-sci↗

arXiv2014

Long-range magnetic interaction and frustration in double perovskite Sr$_{2}$NiIrO$_{6}$

Sr$_{2}$NiIrO$_{6}$ would be a ferromagnetic (FM) insulator in terms of the common superexchange mechanism between the first nearest neighboring (1NN) magnetic ions Ni$^{2+}$ ($t_{2g}^{6}e_{g}^{2}$) and Ir$^{6+}$ ($t_{2g}^{3}$). However, the observed antiferromagnetic (AF) order questions this viewpoint. In this work, we present first-principles calculations and find that while the 1NN Ni$^{2+}$-Ir$^{6+}$ exchange is indeed FM, the 2NN and 3NN couplings in the fcc Ir (and Ni) sublattice are AF. Moreover, the 2NN AF Ir-Ir coupling turns out to be even stronger than the 1NN FM Ni-Ir coupling, thus giving rise to a magnetic frustration. Sr$_{2}$NiIrO$_{6}$ hence becomes a distorted low-temperature antiferromagnet. Naturally, a very similar magnetic property in Sr$_{2}$ZnIrO$_{6}$ can be explained by the frustrated AF coupling in the fcc Ir$^{6+}$ sublattice. This work highlights the long-range magnetic interaction of the delocalized $5d$ electrons, and also addresses why the spin-orbit coupling is ineffective here.

cond-mat.str-el↗

arXiv2014

Ferrimagnetism in the double perovskite Ca2FeOsO6: a density functional study

Using density functional calculations, we find that the newly synthesized Ca$_2$FeOsO$_6$ has the high-spin Fe$^{3+}$ ($3d^5$)-Os$^{5+}$ ($5d^3$) state. The octahedral Os$^{5+}$ ion has a large intrinsic exchange splitting, and its $t_{2g\uparrow}^3$ configuration makes the spin-orbit coupling ineffective. Moreover, there is a strong antiferromagnetic (AF) coupling between the neighboring Fe$^{3+}$ ($S$ = 5/2) and Os$^{5+}$ ($S$ = -3/2), but the AF couplings within both the fcc Fe$^{3+}$ and Os$^{5+}$ sublattices are one order of magnitude weaker. Therefore, a magnetic frustration is suppressed and a stable ferrimagnetic (FiM) ground state appears. This FiM order is due to the virtual hopping of the $t_{2g}$ electrons from Os$^{5+}$ ($t_{2g\downarrow}^3$) to Fe$^{3+}$ ($t_{2g\uparrow}^3e_{g\uparrow}^2$). However, if the experimental bended Fe$^{3+}$-O$^{2-}$-Os$^{5+}$ exchange path gets straight, the $e_g$ hopping from Fe$^{3+}$ ($t_{2g\uparrow}^3e_{g\uparrow}^2$) to Os$^{5+}$ ($t_{2g\uparrow}^3$) would be facilitated and then a ferromagnetic (FM) coupling would occur.

cond-mat.str-el↗