arXiv subjects

Abbas Ghaddar

Publications and source records attributed to Abbas Ghaddar.

23 records · Page 2Linked to original sources

arXiv2021

End-to-End Self-Debiasing Framework for Robust NLU Training

Existing Natural Language Understanding (NLU) models have been shown to incorporate dataset biases leading to strong performance on in-distribution (ID) test sets but poor performance on out-of-distribution (OOD) ones. We introduce a simple yet effective debiasing framework whereby the shallow representations of the main model are used to derive a bias model and both models are trained simultaneously. We demonstrate on three well studied NLU tasks that despite its simplicity, our method leads to competitive OOD results. It significantly outperforms other debiasing approaches on two tasks, while still delivering high in-distribution performance.

cs.CL↗

arXiv2021

Context-aware Adversarial Training for Name Regularity Bias in Named Entity Recognition

In this work, we examine the ability of NER models to use contextual information when predicting the type of an ambiguous entity. We introduce NRB, a new testbed carefully designed to diagnose Name Regularity Bias of NER models. Our results indicate that all state-of-the-art models we tested show such a bias; BERT fine-tuned models significantly outperforming feature-based (LSTM-CRF) ones on NRB, despite having comparable (sometimes lower) performance on standard benchmarks. To mitigate this bias, we propose a novel model-agnostic training method that adds learnable adversarial noise to some entity mentions, thus enforcing models to focus more strongly on the contextual signal, leading to significant gains on NRB. Combining it with two other training strategies, data augmentation and parameter freezing, leads to further gains.

cs.CL↗

arXiv2020

Towards Zero-Shot Knowledge Distillation for Natural Language Processing

Knowledge Distillation (KD) is a common knowledge transfer algorithm used for model compression across a variety of deep learning based natural language processing (NLP) solutions. In its regular manifestations, KD requires access to the teacher's training data for knowledge transfer to the student network. However, privacy concerns, data regulations and proprietary reasons may prevent access to such data. We present, to the best of our knowledge, the first work on Zero-Shot Knowledge Distillation for NLP, where the student learns from the much larger teacher without any task specific data. Our solution combines out of domain data and adversarial training to learn the teacher's output distribution. We investigate six tasks from the GLUE benchmark and demonstrate that we can achieve between 75% and 92% of the teacher's classification score (accuracy or F1) while compressing the model 30 times.

cs.CL↗

arXiv2018

Robust Lexical Features for Improved Neural Network Named-Entity Recognition

Neural network approaches to Named-Entity Recognition reduce the need for carefully hand-crafted features. While some features do remain in state-of-the-art systems, lexical features have been mostly discarded, with the exception of gazetteers. In this work, we show that this is unfair: lexical features are actually quite useful. We propose to embed words and entity types into a low-dimensional vector space we train from annotated data produced by distant supervision thanks to Wikipedia. From this, we compute - offline - a feature vector representing each word. When used with a vanilla recurrent neural network model, this representation yields substantial improvements. We establish a new state-of-the-art F1 score of 87.95 on ONTONOTES 5.0, while matching state-of-the-art performance with a F1 score of 91.73 on the over-studied CONLL-2003 dataset.

cs.CL↗

arXiv2017

Energy loss of terahertz electromagnetic waves by nano-sized connections in near-self-complementary metallic checkerboard patterns

The design of a self-complementary metallic checkerboard pattern achieves broadband, dispersion-less, and maximized absorption, concentrating in the deep subwavelength resistive connections between squares, without any theoretical limitation on the energy absorbing area. Here, we experimentally and numerically investigate the electromagnetic response in the limit of extremely small connections. We show that finite conductivity and randomness in a near-self-complementary checkerboard pattern plays a crucial role in producing a frequency-independent energy loss in the terahertz frequency region. Here metals behave like an almost perfect conductor. When the checkerboard pattern approaches the perfect self-complementary pattern, the perfect conductor approximation spontaneously breaks down, owing to the finite conductivity at the nano- scale connection, leading to broadband absorption. It is also shown that the random connections between metallic squares also lead to broadband and maximized energy loss through scattering loss, similar to finite conductivity.

physics.optics↗