arXiv subjects

Chiyu Zhang

Publications and source records attributed to Chiyu Zhang.

45 records · Page 3Linked to original sources

arXiv2020

AraNet: A Deep Learning Toolkit for Arabic Social Media

We describe AraNet, a collection of deep learning Arabic social media processing tools. Namely, we exploit an extensive host of publicly available and novel social media datasets to train bidirectional encoders from transformer models (BERT) to predict age, dialect, gender, emotion, irony, and sentiment. AraNet delivers state-of-the-art performance on a number of the cited tasks and competitively on others. In addition, AraNet has the advantage of being exclusively based on a deep learning framework and hence feature engineering free. To the best of our knowledge, AraNet is the first to performs predictions across such a wide range of tasks for Arabic NLP and thus meets a critical needs. We publicly release AraNet to accelerate research and facilitate comparisons across the different tasks.

cs.CL↗

arXiv2020

Wrapper Feature Selection Algorithm for the Optimization of an Indicator System of Patent Value Assessment

Effective patent value assessment provides decision support for patent transection and promotes the practical application of patent technology. The limitations of previous research on patent value assessment were analyzed in this work, and a wrapper-mode feature selection algorithm that is based on classifier prediction accuracy was developed. Verification experiments on multiple UCI standard datasets indicated that the algorithm effectively reduced the size of the feature set and significantly enhanced the prediction accuracy of the classifier. When the algorithm was utilized to establish an indicator system of patent value assessment, the size of the system was reduced, and the generalization performance of the classifier was enhanced. Sequential forward selection was applied to further reduce the size of the indicator set and generate an optimal indicator system of patent value assessment.

cs.LG↗

arXiv2019

Sentence-Level BERT and Multi-Task Learning of Age and Gender in Social Media

Social media currently provide a window on our lives, making it possible to learn how people from different places, with different backgrounds, ages, and genders use language. In this work we exploit a newly-created Arabic dataset with ground truth age and gender labels to learn these attributes both individually and in a multi-task setting at the sentence level. Our models are based on variations of deep bidirectional neural networks. More specifically, we build models with gated recurrent units and bidirectional encoder representations from transformers (BERT). We show the utility of multi-task learning (MTL) on the two tasks and identify task-specific attention as a superior choice in this context. We also find that a single-task BERT model outperform our best MTL models on the two tasks. We report tweet-level accuracy of 51.43% for the age task (three-way) and 65.30% on the gender task (binary), both of which outperforms our baselines with a large margin. Our models are language-agnostic, and so can be applied to other languages.

cs.CL↗

arXiv2019

Multi-Task Bidirectional Transformer Representations for Irony Detection

Supervised deep learning requires large amounts of training data. In the context of the FIRE2019 Arabic irony detection shared task (IDAT@FIRE2019), we show how we mitigate this need by fine-tuning the pre-trained bidirectional encoders from transformers (BERT) on gold data in a multi-task setting. We further improve our models by by further pre-training BERT on `in-domain' data, thus alleviating an issue of dialect mismatch in the Google-released BERT model. Our best model acquires 82.4 macro F1 score, and has the unique advantage of being feature-engineering free (i.e., based exclusively on deep learning).

cs.CL↗

arXiv2019

BERT-Based Arabic Social Media Author Profiling

We report our models for detecting age, language variety, and gender from social media data in the context of the Arabic author profiling and deception detection shared task (APDA). We build simple models based on pre-trained bidirectional encoders from transformers (BERT). We first fine-tune the pre-trained BERT model on each of the three datasets with shared task released data. Then we augment shared task data with in-house data for gender and dialect, showing the utility of augmenting training data. Our best models on the shared task test data are acquired with a majority voting of various BERT models trained under different data conditions. We acquire 54.72% accuracy for age, 93.75% for dialect, 81.67% for gender, and 40.97% joint accuracy across the three tasks.

cs.CL↗

arXiv2019

DiaNet: BERT and Hierarchical Attention Multi-Task Learning of Fine-Grained Dialect

Prediction of language varieties and dialects is an important language processing task, with a wide range of applications. For Arabic, the native tongue of ~ 300 million people, most varieties remain unsupported. To ease this bottleneck, we present a very large scale dataset covering 319 cities from all 21 Arab countries. We introduce a hierarchical attention multi-task learning (HA-MTL) approach for dialect identification exploiting our data at the city, state, and country levels. We also evaluate use of BERT on the three tasks, comparing it to the MTL approach. We benchmark and release our data and models.

cs.CL↗

arXiv2019

Happy Together: Learning and Understanding Appraisal From Natural Language

In this paper, we explore various approaches for learning two types of appraisal components from happy language. We focus on 'agency' of the author and the 'sociality' involved in happy moments based on the HappyDB dataset. We develop models based on deep neural networks for the task, including uni- and bi-directional long short-term memory networks, with and without attention. We also experiment with a number of novel embedding methods, such as embedding from neural machine translation (as in CoVe) and embedding from language models (as in ELMo). We compare our results to those acquired by several traditional machine learning methods. Our best models achieve 87.97% accuracy on agency and 93.13% accuracy on sociality, both of which are significantly higher than our baselines.

cs.CL↗

arXiv2019

UBC-NLP at SemEval-2019 Task 6:Ensemble Learning of Offensive Content With Enhanced Training Data

We examine learning offensive content on Twitter with limited, imbalanced data. For the purpose, we investigate the utility of using various data enhancement methods with a host of classical ensemble classifiers. Among the 75 participating teams in SemEval-2019 sub-task B, our system ranks 6th (with 0.706 macro F1-score). For sub-task C, among the 65 participating teams, our system ranks 9th (with 0.587 macro F1-score).

cs.CL↗

arXiv2013

SARS-CoV originated from bats in 1998 and may still exist in humans

SARS-CoV is believed to originate from civets and was thought to have been eliminated as a threat after the 2003 outbreak. Here, we show that human SARS-CoV (huSARS-CoV) originated directly from bats, rather than civets, by a cross-species jump in 1991, and formed a human-adapted strain in 1998. Since then huSARS-CoV has evolved further into highly virulent strains with genotype T and a 29-nt deletion mutation, and weakly virulent strains with genotype C but without the 29-nt deletion. The former can cause pneumonia in humans and could be the major causative pathogen of the SARS outbreak, whereas the latter might not cause pneumonia in humans, but evolved the ability to co-utilize civet ACE2 as an entry receptor, leading to interspecies transmission between humans and civets. Three crucial time points - 1991, for the cross-species jump from bats to humans; 1998, for the formation of the human-adapted SARS-CoV; and 2003, when there was an outbreak of SARS in humans - were found to associate with anomalously low annual precipitation and high temperatures in Guangdong. Anti-SARS-CoV sero-positivity was detected in 20% of all the samples tested from Guangzhou children who were born after 2005, suggesting that weakly virulent huSARS-CoVs might still exist in humans. These existing but undetected SARS-CoVs have a large potential to evolve into highly virulent strains when favorable climate conditions occur, highlighting a potential risk for the reemergence of SARS.

q-bio.PE↗