arXiv subjects

Nai Ding

Publications and source records attributed to Nai Ding.

21 records · Page 2Linked to original sources

arXiv2022

PALRACE: Reading Comprehension Dataset with Human Data and Labeled Rationales

Pre-trained language models achieves high performance on machine reading comprehension (MRC) tasks but the results are hard to explain. An appealing approach to make models explainable is to provide rationales for its decision. To investigate whether human rationales can further improve current models and to facilitate supervised learning of human rationales, here we present PALRACE (Pruned And Labeled RACE), a new MRC dataset with human labeled rationales for 800 passages selected from the RACE dataset. We further classified the question to each passage into 6 types. Each passage was read by at least 26 human readers, who labeled their rationales to answer the question. It is demonstrated that models such as RoBERTa-large outperforms human readers in all 6 types of questions, including inference questions, but its performance can be further improved when having access to the human rationales. Simpler models and pre-trained models that are not fine-tuned based on the task benefit more from human rationales, and their performance can be boosted by more than 30% by rationales. With access to human rationales, a simple model based on the GloVe word embedding can reach the performance of BERT-base.

cs.CL↗

arXiv2021

Using Adversarial Attacks to Reveal the Statistical Bias in Machine Reading Comprehension Models

Pre-trained language models have achieved human-level performance on many Machine Reading Comprehension (MRC) tasks, but it remains unclear whether these models truly understand language or answer questions by exploiting statistical biases in datasets. Here, we demonstrate a simple yet effective method to attack MRC models and reveal the statistical biases in these models. We apply the method to the RACE dataset, for which the answer to each MRC question is selected from 4 options. It is found that several pre-trained language models, including BERT, ALBERT, and RoBERTa, show consistent preference to some options, even when these options are irrelevant to the question. When interfered by these irrelevant options, the performance of MRC models can be reduced from human-level performance to the chance-level performance. Human readers, however, are not clearly affected by these irrelevant options. Finally, we propose an augmented training method that can greatly reduce models' statistical biases.

cs.CL↗

arXiv2020

A Structure-based Memory Maintenance Model for Neural Tracking of Linguistic Structures

It is recently demonstrated that cortical activity can track the time courses of phrases and sentences during speech listening. Here, we propose a plausible neural processing framework to explain this phenomenon. It is argued that the brain maintains the neural representation of a linguistic unit, i.e., a word or a phrase, in a processing buffer until the unit is integrated into a higher-level structure. After being integrated, the unit is removed from the buffer and becomes activated long-term memory. In this model, the duration each unit is maintained in the processing buffer depends on the linguistic structure of the speech input. It is shown that the number of items retained in the processing buffer follows the time courses of phrases and sentences, in line with neurophysiological data, whether the syntactic structure of a sentence is mentally parsed using a bottom-up or top-down predictive model. This model generates a range of testable predictions about the link between linguistic structures, their dynamic psychological representations and their neural underpinnings.

q-bio.NC↗