arXiv subjects

Vivek Gupta

Publications and source records attributed to Vivek Gupta.

At least 127 records · Page 7Linked to original sources

arXiv2020

The Cosmic Dispersion Measure in the EAGLE Simulations

The dispersion measure (DM) of fast radio bursts (FRBs) provides a unique way to probe ionised baryons in the intergalactic medium (IGM). Cosmological models with different parameters lead to different DM-redshift ($\mathrm{DM}-z$) relations. Additionally, the over/under-dense regions in the IGM and the circumgalactic medium of intervening galaxies lead to scatter around the mean $\mathrm{DM}-z$ relations. We have used the Evolution and Assembly of GaLaxies and their Environments (EAGLE) simulations to measure the mean $\mathrm{DM}-z$ relation and the scatter around it using over one billion lines-of-sight between redshifts $0<z<3$. We investigated two techniques to estimate line-of-sight DM: `pixel scrambling' and `box transformations'. We find that using box transformations (a technique from the literature) causes strong correlations due to repeated replication of structure. Comparing a linear and non-linear model, we find that the non-linear model with cosmological parameters, provides a better fit to the $\mathrm{DM}-z$ relation. The differences between these models are the most significant at low redshifts ($z<0.5$). The scatter around the $\mathrm{DM}-z$ relation is highly asymmetric, especially at low redshift $\left(z<0.5\right)$, and becomes more Gaussian as redshift approaches $z\sim3$, the limit of this study. The increase in Gaussianity with redshift is indicative of the large scale structures that is better probed with longer lines-of-sight. The minimum simulation size suitable for investigations into the scatter around the $\mathrm{DM}-z$ relation is 100~comoving~Mpc. The $\mathrm{DM}-z$ relation measured in EAGLE is available with an easy-to-use python interface in the open-source FRB redshift estimation package FRUITBAT.

astro-ph.CO↗

arXiv2020

Estimating fast transient detection pipeline efficiencies at UTMOST via real-time injection of mock FRBs

Dedicated surveys using different detection pipelines are being carried out at multiple observatories to find more Fast Radio Bursts (FRBs). Understanding the efficiency of detection algorithms and the survey completeness function is important to enable unbiased estimation of the underlying FRB population properties. One method to achieve end-to-end testing of the system is by injecting mock FRBs in the live data-stream and searching for them blindly. Mock FRB injection is particularly effective for machine-learning-based classifiers, for which analytic characterisation is impractical. We describe a first-of-its-kind implementation of a real-time mock FRB injection system at the upgraded Molonglo Observatory Synthesis Telescope (UTMOST) and present our results for a set of 20,000 mock FRB injections. The injections have yielded clear insight into the detection efficiencies and have provided a survey completeness function for pulse width, fluence and DM. Mock FRBs are recovered with uniform efficiency over the full range of injected DMs, however the recovery fraction is found to be a strong function of the width and Signal-to-Noise (SNR). For low widths ($\lesssim 20$ ms) and high SNR ($\gtrsim$ 9) the recovery is highly effective with recovery fractions exceeding 90%. We find that the presence of radio frequency interference causes the recovered SNR values to be systematically lower by up to 20% compared to the injected values. We find that wider FRBs become increasingly hard to recover for the machine-learning-based classifier employed at UTMOST. We encourage other observatories to implement live injection set-ups for similar testing of their surveys.

astro-ph.IM↗

arXiv2020

On Long-Tailed Phenomena in Neural Machine Translation

State-of-the-art Neural Machine Translation (NMT) models struggle with generating low-frequency tokens, tackling which remains a major challenge. The analysis of long-tailed phenomena in the context of structured prediction tasks is further hindered by the added complexities of search during inference. In this work, we quantitatively characterize such long-tailed phenomena at two levels of abstraction, namely, token classification and sequence generation. We propose a new loss function, the Anti-Focal loss, to better adapt model training to the structural dependencies of conditional text generation by incorporating the inductive biases of beam search in the training process. We show the efficacy of the proposed technique on a number of Machine Translation (MT) datasets, demonstrating that it leads to significant gains over cross-entropy across different language pairs, especially on the generation of low-frequency words. We have released the code to reproduce our results.

cs.CL↗

arXiv2020

On Dimensional Linguistic Properties of the Word Embedding Space

Word embeddings have become a staple of several natural language processing tasks, yet much remains to be understood about their properties. In this work, we analyze word embeddings in terms of their principal components and arrive at a number of novel and counterintuitive observations. In particular, we characterize the utility of variance explained by the principal components as a proxy for downstream performance. Furthermore, through syntactic probing of the principal embedding space, we show that the syntactic information captured by a principal component does not correlate with the amount of variance it explains. Consequently, we investigate the limitations of variance based embedding post-processing and demonstrate that such post-processing is counter-productive in sentence classification and machine translation tasks. Finally, we offer a few precautionary guidelines on applying variance based embedding post-processing and explain why non-isotropic geometry might be integral to word embedding performance.

cs.CL↗

arXiv2020

P-SIF: Document Embeddings Using Partition Averaging

Simple weighted averaging of word vectors often yields effective representations for sentences which outperform sophisticated seq2seq neural models in many tasks. While it is desirable to use the same method to represent documents as well, unfortunately, the effectiveness is lost when representing long documents involving multiple sentences. One of the key reasons is that a longer document is likely to contain words from many different topics; hence, creating a single vector while ignoring all the topical structure is unlikely to yield an effective document representation. This problem is less acute in single sentences and other short text fragments where the presence of a single topic is most likely. To alleviate this problem, we present P-SIF, a partitioned word averaging model to represent long documents. P-SIF retains the simplicity of simple weighted word averaging while taking a document's topical structure into account. In particular, P-SIF learns topic-specific vectors from a document and finally concatenates them all to represent the overall document. We provide theoretical justifications on the correctness of P-SIF. Through a comprehensive set of experiments, we demonstrate P-SIF's effectiveness compared to simple weighted averaging and many other baselines.

cs.CL↗

arXiv2020

INFOTABS: Inference on Tables as Semi-structured Data

In this paper, we observe that semi-structured tabulated text is ubiquitous; understanding them requires not only comprehending the meaning of text fragments, but also implicit relationships between them. We argue that such data can prove as a testing ground for understanding how we reason about information. To study this, we introduce a new dataset called INFOTABS, comprising of human-written textual hypotheses based on premises that are tables extracted from Wikipedia info-boxes. Our analysis shows that the semi-structured, multi-domain and heterogeneous nature of the premises admits complex, multi-faceted reasoning. Experiments reveal that, while human annotators agree on the relationships between a table-hypothesis pair, several standard modeling strategies are unsuccessful at the task, suggesting that reasoning about tables can pose a difficult modeling challenge.

cs.CL↗

arXiv2020

DeepSumm -- Deep Code Summaries using Neural Transformer Architecture

Source code summarizing is a task of writing short, natural language descriptions of source code behavior during run time. Such summaries are extremely useful for software development and maintenance but are expensive to manually author,hence it is done for small fraction of the code that is produced and is often ignored. Automatic code documentation can possibly solve this at a low cost. This is thus an emerging research field with further applications to program comprehension, and software maintenance. Traditional methods often relied on cognitive models that were built in the form of templates and by heuristics and had varying degree of adoption by the developer community. But with recent advancements, end to end data-driven approaches based on neural techniques have largely overtaken the traditional techniques. Much of the current landscape employs neural translation based architectures with recurrence and attention which is resource and time intensive training procedure. In this paper, we employ neural techniques to solve the task of source code summarizing and specifically compare NMT based techniques to more simplified and appealing Transformer architecture on a dataset of Java methods and comments. We bring forth an argument to dispense the need of recurrence in the training procedure. To the best of our knowledge, transformer based models have not been used for the task before. With supervised samples of more than 2.1m comments and code, we reduce the training time by more than 50% and achieve the BLEU score of 17.99 for the test set of examples.

cs.LG↗

arXiv2020

The UTMOST pulsar timing programme II: Timing noise across the pulsar population

While pulsars possess exceptional rotational stability, large scale timing studies have revealed at least two distinct types of irregularities in their rotation: red timing noise and glitches. Using modern Bayesian techniques, we investigated the timing noise properties of 300 bright southern-sky radio pulsars that have been observed over 1.0-4.8 years by the upgraded Molonglo Observatory Synthesis Telescope (MOST). We reanalysed the spin and spin-down changes associated with nine previously reported pulsar glitches, report the discovery of three new glitches and four unusual glitch-like events in the rotational evolution of PSR J1825$-$0935. We develop a refined Bayesian framework for determining how red noise strength scales with pulsar spin frequency ($ν$) and spin-down frequency ($\dotν$), which we apply to a sample of 280 non-recycled pulsars. With this new method and a simple power-law scaling relation, we show that red noise strength scales across the non-recycled pulsar population as $ν^{a} |\dotν|^{b}$, where $a = -0.84^{+0.47}_{-0.49}$ and $b = 0.97^{+0.16}_{-0.19}$. This method can be easily adapted to utilise more complex, astrophysically motivated red noise models. Lastly, we highlight our timing of the double neutron star PSR J0737$-$3039, and the rediscovery of a bright radio pulsar originally found during the first Molonglo pulsar surveys with an incorrectly catalogued position.

astro-ph.HE↗

arXiv2019

Detection of a Glitch in PSR J0908$-$4913 by UTMOST

We report the first detection of a glitch in the radio pulsar PSR J0908$-$4913 (PSR B0906$-$49) during regular timing observations by the Molonglo Observatory Synthesis Telescope (MOST) as part of the UTMOST project.

astro-ph.HE↗

arXiv2019

Improving Document Classification with Multi-Sense Embeddings

Efficient representation of text documents is an important building block in many NLP tasks. Research on long text categorization has shown that simple weighted averaging of word vectors for sentence representation often outperforms more sophisticated neural models. Recently proposed Sparse Composite Document Vector (SCDV) (Mekala et. al, 2017) extends this approach from sentences to documents using soft clustering over word vectors. However, SCDV disregards the multi-sense nature of words, and it also suffers from the curse of higher dimensionality. In this work, we address these shortcomings and propose SCDV-MS. SCDV-MS utilizes multi-sense word embeddings and learns a lower dimensional manifold. Through extensive experiments on multiple real-world datasets, we show that SCDV-MS embeddings outperform previous state-of-the-art embeddings on multi-class and multi-label text categorization tasks. Furthermore, SCDV-MS embeddings are more efficient than SCDV in terms of time and space complexity on textual classification tasks.

cs.CL↗

arXiv2019

A Logic-Driven Framework for Consistency of Neural Models

While neural models show remarkable accuracy on individual predictions, their internal beliefs can be inconsistent across examples. In this paper, we formalize such inconsistency as a generalization of prediction error. We propose a learning framework for constraining models using logic rules to regularize them away from inconsistency. Our framework can leverage both labeled and unlabeled examples and is directly compatible with off-the-shelf learning schemes without model redesign. We instantiate our framework on natural language inference, where experiments show that enforcing invariants stated in logic can help make the predictions of neural models both accurate and consistent.

cs.AI↗

arXiv2019

Equalizing Recourse across Groups

The rise in machine learning-assisted decision-making has led to concerns about the fairness of the decisions and techniques to mitigate problems of discrimination. If a negative decision is made about an individual (denying a loan, rejecting an application for housing, and so on) justice dictates that we be able to ask how we might change circumstances to get a favorable decision the next time. Moreover, the ability to change circumstances (a better education, improved credentials) should not be limited to only those with access to expensive resources. In other words, \emph{recourse} for negative decisions should be considered a desirable value that can be equalized across (demographically defined) groups. This paper describes how to build models that make accurate predictions while still ensuring that the penalties for a negative outcome do not disadvantage different groups disproportionately. We measure recourse as the distance of an individual from the decision boundary of a classifier. We then introduce a regularized objective to minimize the difference in recourse across groups. We explore linear settings and further extend recourse to non-linear settings as well as model-agnostic settings where the exact distance from boundary cannot be calculated. Our results show that we can successfully decrease the unfairness in recourse while maintaining classifier performance.

cs.LG↗

arXiv2018

Characterizing the spread of exaggerated news content over social media

In this paper, we consider a dataset comprising press releases about health research from different universities in the UK along with a corresponding set of news articles. First, we do an exploratory analysis to understand how the basic information published in the scientific journals get exaggerated as they are reported in these press releases or news articles. This initial analysis shows that some news agencies exaggerate almost 60\% of the articles they publish in the health domain; more than 50\% of the press releases from certain universities are exaggerated; articles in topics like lifestyle and childhood are heavily exaggerated. Motivated by the above observation we set the central objective of this paper to investigate how exaggerated news spreads over an online social network like Twitter. The LIWC analysis points to a remarkable observation these late tweets are essentially laden in words from opinion and realize categories which indicates that, given sufficient time, the wisdom of the crowd is actually able to tell apart the exaggerated news. As a second step we study the characteristics of the users who never or rarely post exaggerated news content and compare them with those who post exaggerated news content more frequently. We observe that the latter class of users have less retweets or mentions per tweet, have significantly more number of followers, use more slang words, less hyperbolic words and less word contractions. We also observe that the LIWC categories like bio, health, body and negative emotion are more pronounced in the tweets posted by the users in the latter class. As a final step we use these observations as features and automatically classify the two groups achieving an F1 score of 0.83.

cs.SI↗

arXiv2018

Detection of a glitch in the pulsar J1709-4429

We report the detection of a glitch event in the pulsar J1709$-$4429 (also known as B1706$-$44) during regular monitoring observations with the Molonglo Observatory Synthesis Telescope (UTMOST). The glitch was found during timing operations, in which we regularly observe over 400 pulsars with up to daily cadence, while commensally searching for Rotating Radio Transients, pulsars, and FRBs. With a fractional size of $Δν/ν\approx 52.4 \times10^{-9}$, the glitch reported here is by far the smallest known for this pulsar, attesting to the efficacy of glitch searches with high cadence using UTMOST.

astro-ph.HE↗

arXiv2018

Bayes-optimal Hierarchical Classification over Asymmetric Tree-Distance Loss

Hierarchical classification is supervised multi-class classification problem over the set of class labels organized according to a hierarchy. In this report, we study the work by Ramaswamy et. al. on hierarchical classification over symmetric tree distance loss. We extend the consistency of hierarchical classification algorithm over asymmetric tree distance loss. We design a $\mathcal{O}(nk\log{}n)$ algorithm to find Bayes optimal classification for a k-ary tree as a hierarchy. We show that under reasonable assumptions over asymmetric loss function, the Bayes optimal classification over this asymmetric loss can be found in $\mathcal{O}(k\log{}n)$. We exploit this insight and attempt to extend the Ova-Cascade algorithm \citet{ramaswamy2015convex} for hierarchical classification over the asymmetric loss.

cs.LG↗

arXiv2017

Efficient Estimation of Generalization Error and Bias-Variance Components of Ensembles

For many applications, an ensemble of base classifiers is an effective solution. The tuning of its parameters(number of classes, amount of data on which each classifier is to be trained on, etc.) requires G, the generalization error of a given ensemble. The efficient estimation of G is the focus of this paper. The key idea is to approximate the variance of the class scores/probabilities of the base classifiers over the randomness imposed by the training subset by normal/beta distribution at each point x in the input feature space. We estimate the parameters of the distribution using a small set of randomly chosen base classifiers and use those parameters to give efficient estimation schemes for G. We give empirical evidence for the quality of the various estimators. We also demonstrate their usefulness in making design choices such as the number of classifiers in the ensemble and the size of a subset of data used for training that is needed to achieve a certain value of generalization error. Our approach also has great potential for designing distributed ensemble classifiers.

cs.LG↗

arXiv2017

Leveraging Distributional Semantics for Multi-Label Learning

We present a novel and scalable label embedding framework for large-scale multi-label learning a.k.a ExMLDS (Extreme Multi-Label Learning using Distributional Semantics). Our approach draws inspiration from ideas rooted in distributional semantics, specifically the Skip Gram Negative Sampling (SGNS) approach, widely used to learn word embeddings for natural language processing tasks. Learning such embeddings can be reduced to a certain matrix factorization. Our approach is novel in that it highlights interesting connections between label embedding methods used for multi-label learning and paragraph/document embedding methods commonly used for learning representations of text data. The framework can also be easily extended to incorporate auxiliary information such as label-label correlations; this is crucial especially when there are a lot of missing labels in the training data. We demonstrate the effectiveness of our approach through an extensive set of experiments on a variety of benchmark datasets, and show that the proposed learning methods perform favorably compared to several baselines and state-of-the-art methods for large-scale multi-label learning. To facilitate end-to-end learning, we develop a joint learning algorithm that can learn the embeddings as well as a regression model that predicts these embeddings given input features, via efficient gradient-based methods.

cs.LG↗

arXiv2017

Text Summarization using Abstract Meaning Representation

With an ever increasing size of text present on the Internet, automatic summary generation remains an important problem for natural language understanding. In this work we explore a novel full-fledged pipeline for text summarization with an intermediate step of Abstract Meaning Representation (AMR). The pipeline proposed by us first generates an AMR graph of an input story, through which it extracts a summary graph and finally, generate summary sentences from this summary graph. Our proposed method achieves state-of-the-art results compared to the other text summarization routines based on AMR. We also point out some significant problems in the existing evaluation methods, which make them unsuitable for evaluating summary quality.

cs.CL↗