arXiv subjects

Hua Wu

Publications and source records attributed to Hua Wu.

At least 163 records · Page 9Linked to original sources

arXiv2021

CrSbSe3: a pseudo one-dimensional ferromagnetic semiconductor

Low-dimensional magnetic materials have attracted much attention due to their novel properties and high potential for spintronic applications. In this work, we study the electronic structure and magnetic properties of the pseudo one-dimensional compound CrSbSe$_3$, using density functional calculations, superexchange model analyses, and Monte Carlo simulations. We find that CrSbSe$_3$ is a ferromagnetic (FM) semiconductor with a band gap of about 0.65 eV. The FM couplings within each zig-zag spin chain are due to the Cr-Se-Cr superexchange with the near-90$^\circ$ bonds, and the inter-chain FM couplings are one order of magnitude weaker. By inclusion of the spin-orbit coupling (SOC) effects, our calculations reproduce the experimental observation of the easy magnetization $a$ axis and the hard $b$ axis (the spin-chain direction), with the calculated moderate magnetic anisotropy of 0.19 meV/Cr. Moreover, we identify the nearly equal contributions from the single ion anisotropy of Cr, and from the exchange anisotropy due to the strong SOC of the heavy elements Sb and Se and their couplings with Cr. Using the parameters of the magnetic coupling and anisotropy from the above calculations, our Monte Carlo simulations yield the Curie temperature $T_{\rm{C}}$ of 108 K.

cond-mat.mtrl-sci↗

arXiv2021

Docking-based Virtual Screening with Multi-Task Learning

Machine learning shows great potential in virtual screening for drug discovery. Current efforts on accelerating docking-based virtual screening do not consider using existing data of other previously developed targets. To make use of the knowledge of the other targets and take advantage of the existing data, in this work, we apply multi-task learning to the problem of docking-based virtual screening. With two large docking datasets, the results of extensive experiments show that multi-task learning can achieve better performances on docking score prediction. By learning knowledge across multiple targets, the model trained by multi-task learning shows a better ability to adapt to a new target. Additional empirical study shows that other problems in drug discovery, such as the experimental drug-target affinity prediction, may also benefit from multi-task learning. Our results demonstrate that multi-task learning is a promising machine learning approach for docking-based virtual screening and accelerating the process of drug discovery.

cs.LG↗

arXiv2021

Amendable Generation for Dialogue State Tracking

In task-oriented dialogue systems, recent dialogue state tracking methods tend to perform one-pass generation of the dialogue state based on the previous dialogue state. The mistakes of these models made at the current turn are prone to be carried over to the next turn, causing error propagation. In this paper, we propose a novel Amendable Generation for Dialogue State Tracking (AG-DST), which contains a two-pass generation process: (1) generating a primitive dialogue state based on the dialogue of the current turn and the previous dialogue state, and (2) amending the primitive dialogue state from the first pass. With the additional amending generation pass, our model is tasked to learn more robust dialogue state tracking by amending the errors that still exist in the primitive dialogue state, which plays the role of reviser in the double-checking process and alleviates unnecessary error propagation. Experimental results show that AG-DST significantly outperforms previous works in two active DST datasets (MultiWOZ 2.2 and WOZ 2.0), achieving new state-of-the-art performances.

cs.CL↗

arXiv2021

SgSum: Transforming Multi-document Summarization into Sub-graph Selection

Most of existing extractive multi-document summarization (MDS) methods score each sentence individually and extract salient sentences one by one to compose a summary, which have two main drawbacks: (1) neglecting both the intra and cross-document relations between sentences; (2) neglecting the coherence and conciseness of the whole summary. In this paper, we propose a novel MDS framework (SgSum) to formulate the MDS task as a sub-graph selection problem, in which source documents are regarded as a relation graph of sentences (e.g., similarity graph or discourse graph) and the candidate summaries are its sub-graphs. Instead of selecting salient sentences, SgSum selects a salient sub-graph from the relation graph as the summary. Comparing with traditional methods, our method has two main advantages: (1) the relations between sentences are captured by modeling both the graph structure of the whole document set and the candidate sub-graphs; (2) directly outputs an integrate summary in the form of sub-graph which is more informative and coherent. Extensive experiments on MultiNews and DUC datasets show that our proposed method brings substantial improvements over several strong baselines. Human evaluation results also demonstrate that our model can produce significantly more coherent and informative summaries compared with traditional MDS methods. Moreover, the proposed architecture has strong transfer ability from single to multi-document input, which can reduce the resource bottleneck in MDS tasks. Our code and results are available at: \url{https://github.com/PaddlePaddle/Research/tree/master/NLP/EMNLP2021-SgSum}.

cs.CL↗

arXiv2021

Triaxial magnetic anisotropy in the two-dimensional ferromagnetic semiconductor CrSBr

Two-dimensional (2D) ferromagnets have recently drawn extensive attention, and here we study the electronic structure and magnetic properties of the bulk and monolayer of CrSBr, using first-principles calculations and Monte Carlo simulations. Our results show that bulk CrSBr is a magnetic semiconductor and has the easy magnetization b-axis, hard c-axis, and intermediate a-axis. Thus, the experimental triaxial magnetic anisotropy (MA) is well reproduced here, and it is identified to be the joint effects of spin-orbit coupling (SOC) and magnetic dipole-dipole interaction. We find that bulk CrSBr has a strong ferromagnetic (FM) intralayer coupling but a marginal interlayer one. We also study CrSBr monolayer in detail and find that the intralayer FM exchange persists and the shape anisotropy has a more pronounced contribution to the MA. Using the parameters of the FM exchange and the triaxial MA, our Monte Carlo simulations show that CrSBr monolayer has Curie temperature Tc = 175 K. Moreover, we find that a uniaxial tensile (compressive) strain along the a (b) axis would further increase the Tc.

cond-mat.mtrl-sci↗

arXiv2021

DuRecDial 2.0: A Bilingual Parallel Corpus for Conversational Recommendation

In this paper, we provide a bilingual parallel human-to-human recommendation dialog dataset (DuRecDial 2.0) to enable researchers to explore a challenging task of multilingual and cross-lingual conversational recommendation. The difference between DuRecDial 2.0 and existing conversational recommendation datasets is that the data item (Profile, Goal, Knowledge, Context, Response) in DuRecDial 2.0 is annotated in two languages, both English and Chinese, while other datasets are built with the setting of a single language. We collect 8.2k dialogs aligned across English and Chinese languages (16.5k dialogs and 255k utterances in total) that are annotated by crowdsourced workers with strict quality control procedure. We then build monolingual, multilingual, and cross-lingual conversational recommendation baselines on DuRecDial 2.0. Experiment results show that the use of additional English data can bring performance improvement for Chinese conversational recommendation, indicating the benefits of DuRecDial 2.0. Finally, this dataset provides a challenging testbed for future studies of monolingual, multilingual, and cross-lingual conversational recommendation.

cs.CL↗

arXiv2021

ERNIE-M: Enhanced Multilingual Representation by Aligning Cross-lingual Semantics with Monolingual Corpora

Recent studies have demonstrated that pre-trained cross-lingual models achieve impressive performance in downstream cross-lingual tasks. This improvement benefits from learning a large amount of monolingual and parallel corpora. Although it is generally acknowledged that parallel corpora are critical for improving the model performance, existing methods are often constrained by the size of parallel corpora, especially for low-resource languages. In this paper, we propose ERNIE-M, a new training method that encourages the model to align the representation of multiple languages with monolingual corpora, to overcome the constraint that the parallel corpus size places on the model performance. Our key insight is to integrate back-translation into the pre-training process. We generate pseudo-parallel sentence pairs on a monolingual corpus to enable the learning of semantic alignments between different languages, thereby enhancing the semantic modeling of cross-lingual models. Experimental results show that ERNIE-M outperforms existing cross-lingual models and delivers new state-of-the-art results in various cross-lingual downstream tasks.

cs.CL↗

arXiv2021

A Multimodal Sentiment Dataset for Video Recommendation

Recently, multimodal sentiment analysis has seen remarkable advance and a lot of datasets are proposed for its development. In general, current multimodal sentiment analysis datasets usually follow the traditional system of sentiment/emotion, such as positive, negative and so on. However, when applied in the scenario of video recommendation, the traditional sentiment/emotion system is hard to be leveraged to represent different contents of videos in the perspective of visual senses and language understanding. Based on this, we propose a multimodal sentiment analysis dataset, named baiDu Video Sentiment dataset (DuVideoSenti), and introduce a new sentiment system which is designed to describe the sentimental style of a video on recommendation scenery. Specifically, DuVideoSenti consists of 5,630 videos which displayed on Baidu, each video is manually annotated with a sentimental style label which describes the user's real feeling of a video. Furthermore, we propose UNIMO as our baseline for DuVideoSenti. Experimental results show that DuVideoSenti brings new challenges to multimodal sentiment analysis, and could be used as a new benchmark for evaluating approaches designed for video understanding and multimodal fusion. We also expect our proposed DuVideoSenti could further improve the development of multimodal sentiment analysis and its application to video recommendations.

cs.CL↗

arXiv2021

Mixup Decoding for Diverse Machine Translation

Diverse machine translation aims at generating various target language translations for a given source language sentence. Leveraging the linear relationship in the sentence latent space introduced by the mixup training, we propose a novel method, MixDiversity, to generate different translations for the input sentence by linearly interpolating it with different sentence pairs sampled from the training corpus when decoding. To further improve the faithfulness and diversity of the translations, we propose two simple but effective approaches to select diverse sentence pairs in the training corpus and adjust the interpolation weight for each pair correspondingly. Moreover, by controlling the interpolation weight, our method can achieve the trade-off between faithfulness and diversity without any additional training, which is required in most of the previous methods. Experiments on WMT'16 en-ro, WMT'14 en-de, and WMT'17 zh-en are conducted to show that our method substantially outperforms all previous diverse machine translation methods.

cs.CL↗

arXiv2021

Fine-grained Entity Typing via Label Reasoning

Conventional entity typing approaches are based on independent classification paradigms, which make them difficult to recognize inter-dependent, long-tailed and fine-grained entity types. In this paper, we argue that the implicitly entailed extrinsic and intrinsic dependencies between labels can provide critical knowledge to tackle the above challenges. To this end, we propose \emph{Label Reasoning Network(LRN)}, which sequentially reasons fine-grained entity labels by discovering and exploiting label dependencies knowledge entailed in the data. Specifically, LRN utilizes an auto-regressive network to conduct deductive reasoning and a bipartite attribute graph to conduct inductive reasoning between labels, which can effectively model, learn and reason complex label dependencies in a sequence-to-set, end-to-end manner. Experiments show that LRN achieves the state-of-the-art performance on standard ultra fine-grained entity typing benchmarks, and can also resolve the long tail label problem effectively.

cs.CL↗

arXiv2021

DuTrust: A Sentiment Analysis Dataset for Trustworthiness Evaluation

While deep learning models have greatly improved the performance of most artificial intelligence tasks, they are often criticized to be untrustworthy due to the black-box problem. Consequently, many works have been proposed to study the trustworthiness of deep learning. However, as most open datasets are designed for evaluating the accuracy of model outputs, there is still a lack of appropriate datasets for evaluating the inner workings of neural networks. The lack of datasets obviously hinders the development of trustworthiness research. Therefore, in order to systematically evaluate the factors for building trustworthy systems, we propose a novel and well-annotated sentiment analysis dataset to evaluate robustness and interpretability. To evaluate these factors, our dataset contains diverse annotations about the challenging distribution of instances, manual adversarial instances and sentiment explanations. Several evaluation metrics are further proposed for interpretability and robustness. Based on the dataset and metrics, we conduct comprehensive comparisons for the trustworthiness of three typical models, and also study the relations between accuracy, robustness and interpretability. We release this trustworthiness evaluation dataset at \url{https://github/xyz} and hope our work can facilitate the progress on building more trustworthy systems for real-world applications.

cs.CL↗

arXiv2021

DuReader_robust: A Chinese Dataset Towards Evaluating Robustness and Generalization of Machine Reading Comprehension in Real-World Applications

Machine reading comprehension (MRC) is a crucial task in natural language processing and has achieved remarkable advancements. However, most of the neural MRC models are still far from robust and fail to generalize well in real-world applications. In order to comprehensively verify the robustness and generalization of MRC models, we introduce a real-world Chinese dataset -- DuReader_robust. It is designed to evaluate the MRC models from three aspects: over-sensitivity, over-stability and generalization. Comparing to previous work, the instances in DuReader_robust are natural texts, rather than the altered unnatural texts. It presents the challenges when applying MRC models to real-world applications. The experimental results show that MRC models do not perform well on the challenge test set. Moreover, we analyze the behavior of existing models on the challenge test set, which may provide suggestions for future model development. The dataset and codes are publicly available at https://github.com/baidu/DuReader.

cs.CL↗

arXiv2021

Magnetic order and its interplay with structure phase transition in van der Waals ferromagnet VI$_3$

Van der Waals magnet VI$_3$ demonstrates intriguing magnetic properties that render it great for use in various applications. However, its microscopic magnetic structure has not been determined yet. Here, we report neutron diffraction and susceptibility measurements in VI$_3$ that revealed a ferromagnetic order with the moment direction tilted from the $c$-axis by ~36° at 4 K. A spin reorientation accompanied by a structure distortion within the honeycomb plane is observed at a temperature of ~27 K, before the magnetic order completely disappears at $T_C$ = 50 K. The refined magnetic moment of ~1.3 $μ_B$ at 4 K is considerably lower than the fully ordered spin moment of 2 $μ_B$/ V$^{3+}$, suggesting the presence of a considerable orbital moment antiparallel to the spin moment and strong spin-orbit coupling in VI$_3$. This results in strong magnetoelastic interactions that make the magnetic properties of VI$_3$ easily tunable via strain and pressure.

cond-mat.str-el↗

arXiv2021

ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation

Pre-trained models have achieved state-of-the-art results in various Natural Language Processing (NLP) tasks. Recent works such as T5 and GPT-3 have shown that scaling up pre-trained language models can improve their generalization abilities. Particularly, the GPT-3 model with 175 billion parameters shows its strong task-agnostic zero-shot/few-shot learning capabilities. Despite their success, these large-scale models are trained on plain texts without introducing knowledge such as linguistic knowledge and world knowledge. In addition, most large-scale models are trained in an auto-regressive way. As a result, this kind of traditional fine-tuning approach demonstrates relatively weak performance when solving downstream language understanding tasks. In order to solve the above problems, we propose a unified framework named ERNIE 3.0 for pre-training large-scale knowledge enhanced models. It fuses auto-regressive network and auto-encoding network, so that the trained model can be easily tailored for both natural language understanding and generation tasks with zero-shot learning, few-shot learning or fine-tuning. We trained the model with 10 billion parameters on a 4TB corpus consisting of plain texts and a large-scale knowledge graph. Empirical results show that the model outperforms the state-of-the-art models on 54 Chinese NLP tasks, and its English version achieves the first place on the SuperGLUE benchmark (July 3, 2021), surpassing the human performance by +0.8% (90.6% vs. 89.8%).

cs.CL↗

arXiv2021

2D hybrid CrCl2(N2C4H4)2 with tunable ferromagnetic half-metallicity

Two-dimensional ferromagnetic (2D FM) half-metal holds great potential for quantum magnetoelectronics and spintronic devices. Here, using density functional calculations and magnetic pictures, we study the electronic structure and magnetic properties of the novel van der Waals (vdW) metal-organic framework (MOF), CrCl2(N2C4H4)2, i.e. CrCl2(pyrazine)2. Our results show that CrCl2(pyrazine)2 is a 2D FM half-metal, having a strong intralayer FM coupling but a much weak interlayer one due to the vdW spacing. Its spin-polarized conduction bands are formed by the pyrazine molecular orbitals and are polarized by the robust Cr3+ local spin = 3/2. These results agree with the recent experiments [Pedersen et al., Nature Chemistry, 2018, 10, 1056]. More interestingly, CrCl2(pyrazine)2 monolayer has a strong doping tunability of the FM half-metallicity, and the FM coupling would be significantly enhanced by electron doping. Our work highlights a vital role of the organic ligand and suggests that vdW MOF is also worth exploration for new 2D magnetic materials.

cond-mat.mtrl-sci↗

arXiv2021

Ferromagnetic half-metallicity in YBaCo2O6 and spin-states driven metal-insulator transition

Cobaltates have rich spin-states and diverse properties. Using spin-state pictures and firstprinciples calculations, here we study the electronic structure and magnetism of the mixed-valent double perovskite YBaCo2O6. We find that YBaCo2O6 is in the formal intermediate-spin (IS) Co3+/low-spin (LS) Co4+ ground state. The hopping of eg electron from IS-Co3+ to LS-Co4+ via double exchange gives rise to a ferromagnetic half-metallicity, which well accounts for the recent experiments. The reduction of both magnetization and Curie temperature by oxygen vacancies is discussed, aided with Monte Carlo simulations. We also explore several other possible spin-states and their interesting electronic/magnetic properties. Moreover, we predict that a volume expansion more than 3% would tune YBaCo2O6 into the high-spin (HS) Co3+/LS Co4+ ferromagnetic state and simultaneously drive a metal-insulator transition. Therefore, spin-states are a useful parameter for tuning the material properties of cobaltates.

cond-mat.mtrl-sci↗

arXiv2021

ERNIE-Tiny : A Progressive Distillation Framework for Pretrained Transformer Compression

Pretrained language models (PLMs) such as BERT adopt a training paradigm which first pretrain the model in general data and then finetune the model on task-specific data, and have recently achieved great success. However, PLMs are notorious for their enormous parameters and hard to be deployed on real-life applications. Knowledge distillation has been prevailing to address this problem by transferring knowledge from a large teacher to a much smaller student over a set of data. We argue that the selection of thee three key components, namely teacher, training data, and learning objective, is crucial to the effectiveness of distillation. We, therefore, propose a four-stage progressive distillation framework ERNIE-Tiny to compress PLM, which varies the three components gradually from general level to task-specific level. Specifically, the first stage, General Distillation, performs distillation with guidance from pretrained teacher, gerenal data and latent distillation loss. Then, General-Enhanced Distillation changes teacher model from pretrained teacher to finetuned teacher. After that, Task-Adaptive Distillation shifts training data from general data to task-specific data. In the end, Task-Specific Distillation, adds two additional losses, namely Soft-Label and Hard-Label loss onto the last stage. Empirical results demonstrate the effectiveness of our framework and generalization gain brought by ERNIE-Tiny.In particular, experiments show that a 4-layer ERNIE-Tiny maintains over 98.0%performance of its 12-layer teacher BERT base on GLUE benchmark, surpassing state-of-the-art (SOTA) by 1.0% GLUE score with the same amount of parameters. Moreover, ERNIE-Tiny achieves a new compression SOTA on five Chinese NLP tasks, outperforming BERT base by 0.4% accuracy with 7.5x fewer parameters and9.4x faster inference speed.

cs.CL↗

arXiv2021

PLATO-2: Towards Building an Open-Domain Chatbot via Curriculum Learning

To build a high-quality open-domain chatbot, we introduce the effective training process of PLATO-2 via curriculum learning. There are two stages involved in the learning process. In the first stage, a coarse-grained generation model is trained to learn response generation under the simplified framework of one-to-one mapping. In the second stage, a fine-grained generative model augmented with latent variables and an evaluation model are further trained to generate diverse responses and to select the best response, respectively. PLATO-2 was trained on both Chinese and English data, whose effectiveness and superiority are verified through comprehensive evaluations, achieving new state-of-the-art results.

cs.CL↗