arXiv subjects

Mohammad Saleh

Publications and source records attributed to Mohammad Saleh.

23 records · Page 2Linked to original sources

arXiv2021

WebRED: Effective Pretraining And Finetuning For Relation Extraction On The Web

Relation extraction is used to populate knowledge bases that are important to many applications. Prior datasets used to train relation extraction models either suffer from noisy labels due to distant supervision, are limited to certain domains or are too small to train high-capacity models. This constrains downstream applications of relation extraction. We therefore introduce: WebRED (Web Relation Extraction Dataset), a strongly-supervised human annotated dataset for extracting relationships from a variety of text found on the World Wide Web, consisting of ~110K examples. We also describe the methods we used to collect ~200M examples as pre-training data for this task. We show that combining pre-training on a large weakly supervised dataset with fine-tuning on a small strongly-supervised dataset leads to better relation extraction performance. We provide baselines for this new dataset and present a case for the importance of human annotation in improving the performance of relation extraction from text found on the web.

cs.CL↗

arXiv2020

PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization

Recent work pre-training Transformers with self-supervised objectives on large text corpora has shown great success when fine-tuned on downstream NLP tasks including text summarization. However, pre-training objectives tailored for abstractive text summarization have not been explored. Furthermore there is a lack of systematic evaluation across diverse domains. In this work, we propose pre-training large Transformer-based encoder-decoder models on massive text corpora with a new self-supervised objective. In PEGASUS, important sentences are removed/masked from an input document and are generated together as one output sequence from the remaining sentences, similar to an extractive summary. We evaluated our best PEGASUS model on 12 downstream summarization tasks spanning news, science, stories, instructions, emails, patents, and legislative bills. Experiments demonstrate it achieves state-of-the-art performance on all 12 downstream datasets measured by ROUGE scores. Our model also shows surprising performance on low-resource summarization, surpassing previous state-of-the-art results on 6 datasets with only 1000 examples. Finally we validated our results using human evaluation and show that our model summaries achieve human performance on multiple datasets.

cs.CL↗

arXiv2020

SEAL: Segment-wise Extractive-Abstractive Long-form Text Summarization

Most prior work in the sequence-to-sequence paradigm focused on datasets with input sequence lengths in the hundreds of tokens due to the computational constraints of common RNN and Transformer architectures. In this paper, we study long-form abstractive text summarization, a sequence-to-sequence setting with input sequence lengths up to 100,000 tokens and output sequence lengths up to 768 tokens. We propose SEAL, a Transformer-based model, featuring a new encoder-decoder attention that dynamically extracts/selects input snippets to sparsely attend to for each output segment. Using only the original documents and summaries, we derive proxy labels that provide weak supervision for extractive layers simultaneously with regular supervision from abstractive summaries. The SEAL model achieves state-of-the-art results on existing long-form summarization tasks, and outperforms strong baseline models on a new dataset/task we introduce, Search2Wiki, with much longer input text. Since content selection is explicit in the SEAL model, a desirable side effect is that the selection can be inspected for enhanced interpretability.

cs.CL↗

arXiv2018

Generating Wikipedia by Summarizing Long Sequences

We show that generating English Wikipedia articles can be approached as a multi- document summarization of source documents. We use extractive summarization to coarsely identify salient information and a neural abstractive model to generate the article. For the abstractive model, we introduce a decoder-only architecture that can scalably attend to very long sequences, much longer than typical encoder- decoder architectures used in sequence transduction. We show that this model can generate fluent, coherent multi-sentence paragraphs and even whole Wikipedia articles. When given reference documents, we show it can extract relevant factual information as reflected in perplexity, ROUGE scores and human evaluations.

cs.CL↗

arXiv2017

Azimuthally differential pion femtoscopy relative to the second and third harmonic in Pb-Pb 2.76 TeV collision from ALICE

Azimuthally differential femtoscopic measurements, being sensitive to spatio-temporal characteristics of the source as well as to the collective velocity fields at freeze-out, provide very important information on the nature and dynamics of the system evolution. While the HBT radii modulations relative to the second harmonic event plane reflect mostly the spatial geometry of the source, the third harmonic results are mostly defined by the velocity fields. Radii variations with respect to the third harmonic event plane unambiguously signal a collective expansion and anisotropy in the flow fields. Event shape engineering (ESE) is a technique proposed to select events corresponding to a particular shape. Azimuthally differential HBT combined with ESE allows for a detailed analysis of the relation between initial geometry, anisotropic flow and the deformation of source shape. We present azimuthally differential pion femtoscopy with respect to second and third harmonic event planes as a function of the pion transverse momentum for different collision centralities in Pb-Pb collisions at sqrt(s_NN) = 2.76 TeV. All these results are compared to existing models. The effects of the selection of the events with high elliptic or triangular flow are also presented.

hep-ex↗