arXiv subjects

Mayank Singh

Publications and source records attributed to Mayank Singh.

At least 73 records · Page 4Linked to original sources

arXiv2023

Unveiling the Multi-Annotation Process: Examining the Influence of Annotation Quantity and Instance Difficulty on Model Performance

The NLP community has long advocated for the construction of multi-annotator datasets to better capture the nuances of language interpretation, subjectivity, and ambiguity. This paper conducts a retrospective study to show how performance scores can vary when a dataset expands from a single annotation per instance to multiple annotations. We propose a novel multi-annotator simulation process to generate datasets with varying annotation budgets. We show that similar datasets with the same annotation budget can lead to varying performance gains. Our findings challenge the popular belief that models trained on multi-annotation examples always lead to better performance than models trained on single or few-annotation examples.

cs.CL↗

arXiv2023

Unlocking Model Insights: A Dataset for Automated Model Card Generation

Language models (LMs) are no longer restricted to ML community, and instruction-tuned LMs have led to a rise in autonomous AI agents. As the accessibility of LMs grows, it is imperative that an understanding of their capabilities, intended usage, and development cycle also improves. Model cards are a popular practice for documenting detailed information about an ML model. To automate model card generation, we introduce a dataset of 500 question-answer pairs for 25 ML models that cover crucial aspects of the model, such as its training configurations, datasets, biases, architecture details, and training resources. We employ annotators to extract the answers from the original paper. Further, we explore the capabilities of LMs in generating model cards by answering questions. Our initial experiments with ChatGPT-3.5, LLaMa, and Galactica showcase a significant gap in the understanding of research papers by these aforementioned LMs as well as generating factual textual responses. We posit that our dataset can be used to train models to automate the generation of model cards from paper text and reduce human effort in the model card curation process. The complete dataset is available on https://osf.io/hqt7p/?view_only=3b9114e3904c4443bcd9f5c270158d37

cs.CL↗

arXiv2023

Modeling interdisciplinary interactions among Physics, Mathematics & Computer Science

Interdisciplinarity has over the recent years have gained tremendous importance and has become one of the key ways of doing cutting edge research. In this paper we attempt to model the citation flow across three different fields -- Physics (PHY), Mathematics (MA) and Computer Science (CS). For instance, is there a specific pattern in which these fields cite one another? We carry out experiments on a dataset comprising more than 1.2 million articles taken from these three fields. We quantify the citation interactions among these three fields through temporal bucket signatures. We present numerical models based on variants of the recently proposed relay-linking framework to explain the citation dynamics across the three disciplines. These models make a modest attempt to unfold the underlying principles of how citation links could have been formed across the three fields over time.

cs.DL↗

arXiv2023

Metric@CustomerN: Evaluating Metrics at a Customer Level in E-Commerce

Accuracy measures such as Recall, Precision, and Hit Rate have been a standard way of evaluating Recommendation Systems. The assumption is to use a fixed Top-N to represent them. We propose that median impressions viewed from historical sessions per diner be used as a personalized value for N. We present preliminary exploratory results and list future steps to improve upon and evaluate the efficacy of these personalized metrics.

cs.IR↗

arXiv2023

Studying QGP transport properties in a concurrent minijet+hydro framework

Minijets are ubiquitous in heavy-ion collision experiments. However, they are often excluded from the hydrodynamic simulations of QGP as they do not thermalize at short time scales and are not treated as part of the collective medium. Using a concurrent jet+hydro framework, we show that the minijets could account for a significant portion of particle multiplicity. Therefore, the energy deposition from minijet-medium interactions can substantially modify the QGP transport properties inferred from model-to-data comparisons.

hep-ph↗

arXiv2023

BLOOM: A 176B-Parameter Open-Access Multilingual Language Model

Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to widespread adoption, most LLMs are developed by resource-rich organizations and are frequently kept from the public. As a step towards democratizing this powerful technology, we present BLOOM, a 176B-parameter open-access language model designed and built thanks to a collaboration of hundreds of researchers. BLOOM is a decoder-only Transformer language model that was trained on the ROOTS corpus, a dataset comprising hundreds of sources in 46 natural and 13 programming languages (59 in total). We find that BLOOM achieves competitive performance on a wide variety of benchmarks, with stronger results after undergoing multitask prompted finetuning. To facilitate future research and applications using LLMs, we publicly release our models and code under the Responsible AI License.

cs.CL↗

arXiv2023

Analogy-Forming Transformers for Few-Shot 3D Parsing

We present Analogical Networks, a model that encodes domain knowledge explicitly, in a collection of structured labelled 3D scenes, in addition to implicitly, as model parameters, and segments 3D object scenes with analogical reasoning: instead of mapping a scene to part segments directly, our model first retrieves related scenes from memory and their corresponding part structures, and then predicts analogous part structures for the input scene, via an end-to-end learnable modulation mechanism. By conditioning on more than one retrieved memories, compositions of structures are predicted, that mix and match parts across the retrieved memories. One-shot, few-shot or many-shot learning are treated uniformly in Analogical Networks, by conditioning on the appropriate set of memories, whether taken from a single, few or many memory exemplars, and inferring analogous parses. We show Analogical Networks are competitive with state-of-the-art 3D segmentation transformers in many-shot settings, and outperform them, as well as existing paradigms of meta-learning and few-shot learning, in few-shot settings. Analogical Networks successfully segment instances of novel object categories simply by expanding their memory, without any weight updates. Our code and models are publicly available in the project webpage: http://analogicalnets.github.io/.

cs.CV↗

arXiv2023

NLPExplorer: Exploring the Universe of NLP Papers

Understanding the current research trends, problems, and their innovative solutions remains a bottleneck due to the ever-increasing volume of scientific articles. In this paper, we propose NLPExplorer, a completely automatic portal for indexing, searching, and visualizing Natural Language Processing (NLP) research volume. NLPExplorer presents interesting insights from papers, authors, venues, and topics. In contrast to previous topic modelling based approaches, we manually curate five course-grained non-exclusive topical categories namely Linguistic Target (Syntax, Discourse, etc.), Tasks (Tagging, Summarization, etc.), Approaches (unsupervised, supervised, etc.), Languages (English, Chinese,etc.) and Dataset types (news, clinical notes, etc.). Some of the novel features include a list of young popular authors, popular URLs, and datasets, a list of topically diverse papers and recent popular papers. Also, it provides temporal statistics such as yearwise popularity of topics, datasets, and seminal papers. To facilitate future research and system development, we make all the processed datasets accessible through API calls. The current system is available at http://lingo.iitgn.ac.in:5001/

cs.IR↗

arXiv2023

MUTANT: A Multi-sentential Code-mixed Hinglish Dataset

The multi-sentential long sequence textual data unfolds several interesting research directions pertaining to natural language processing and generation. Though we observe several high-quality long-sequence datasets for English and other monolingual languages, there is no significant effort in building such resources for code-mixed languages such as Hinglish (code-mixing of Hindi-English). In this paper, we propose a novel task of identifying multi-sentential code-mixed text (MCT) from multilingual articles. As a use case, we leverage multilingual articles from two different data sources and build a first-of-its-kind multi-sentential code-mixed Hinglish dataset i.e., MUTANT. We propose a token-level language-aware pipeline and extend the existing metrics measuring the degree of code-mixing to a multi-sentential framework and automatically identify MCT in the multilingual articles. The MUTANT dataset comprises 67k articles with 85k identified Hinglish MCTs. To facilitate future research, we make the publicly available.

cs.CL↗

arXiv2022

Tables to LaTeX: structure and content extraction from scientific tables

Scientific documents contain tables that list important information in a concise fashion. Structure and content extraction from tables embedded within PDF research documents is a very challenging task due to the existence of visual features like spanning cells and content features like mathematical symbols and equations. Most existing table structure identification methods tend to ignore these academic writing features. In this paper, we adapt the transformer-based language modeling paradigm for scientific table structure and content extraction. Specifically, the proposed model converts a tabular image to its corresponding LaTeX source code. Overall, we outperform the current state-of-the-art baselines and achieve an exact match accuracy of 70.35 and 49.69% on table structure and content extraction, respectively. Further analysis demonstrates that the proposed models efficiently identify the number of rows and columns, the alphanumeric characters, the LaTeX tokens, and symbols.

cs.IR↗

arXiv2022

Mini-jet quenching in a concurrent jet+hydro evolution and the non-equilibrium quark-gluon plasma

Mini-jets, created by perturbative hard QCD collisions at moderate energies, can represent a significant portion of the total multiplicity of a heavy-ion collision event. Since their transverse momenta are initially larger than the typical saturation scale describing the bulk of the equilibrating QGP, their typical stopping distances are larger than the usual hydrodynamization time, so they do not in general hydrodynamize at the same pace than the bulk of the collision. Therefore, in general mini-jets cannot be described solely by a unique pre-equilibrium stage that bridges the initial, over-occupied glasma state, with the hydrodynamical evolution. In this work we make use of a new concurrent mini-jet+hydrodynamic framework in which the properties of the hydrodynamically evolving QGP are modified due to the injection of energy and momentum from the mini-jets. We study the system for different choices of the minimum transverse momentum associated to mini-jet production. In order to achieve a realistic description of charged particle multiplicity, the amount of entropy associated to the low-$x$ initial state needs to be reduced. Moreover, the fact that the injected momentum from the randomly oriented mini-jets is not correlated with the spatial gradients of the system reduces overall flow, and the value of the QGP transport coefficients needs to be reduced accordingly. They are an important new source of fluctuations, resulting in a spikier, notably modified hydrodynamical evolution. We discuss the impact of the mini-jets on a number of observables, such as $p_T$ spectra and $p_T$-differential flow $v_n$ for a wide range of centrality classes. In contrast to elliptic, triangular or quadrangular flow, we find that directed flow, $v_1$, has the strongest potential to discriminate between different mini-jet production rates.

nucl-th↗

arXiv2022

Confronting anomalous kaon correlations measured in Pb-Pb collisions at $\sqrt{s_{NN}} = 2.76$ TeV

Measurements of the dynamical correlations between neutral and charged kaons in central Pb-Pb collisions at $\sqrt{s_{NN}} = 2.76$ TeV by the ALICE Collaboration display anomalous behavior relative to conventional heavy-ion collision simulators such as AMPT, EPOS, and HIJING. We consider other conventional statistical models, none of which can reproduce the magnitude and centrality dependence of the correlations. The data can be reproduced by coherent emission from domains which grow in number and volume with increasing centrality. We show that the energy released by condensation of strange quarks may be sufficient to explain the anomaly.

hep-ph↗

arXiv2022

The Inefficiency of Language Models in Scholarly Retrieval: An Experimental Walk-through

Language models are increasingly becoming popular in AI-powered scientific IR systems. This paper evaluates popular scientific language models in handling (i) short-query texts and (ii) textual neighbors. Our experiments showcase the inability to retrieve relevant documents for a short-query text even under the most relaxed conditions. Additionally, we leverage textual neighbors, generated by small perturbations to the original text, to demonstrate that not all perturbations lead to close neighbors in the embedding space. Further, an exhaustive categorization yields several classes of orthographically and semantically related, partially related, and completely unrelated neighbors. Retrieval performance turns out to be more influenced by the surface form rather than the semantics of the text.

cs.IR↗

arXiv2021

The BEST framework for the search for the QCD critical point and the chiral magnetic effect

The Beam Energy Scan Theory (BEST) Collaboration was formed with the goal of providing a theoretical framework for analyzing data from the Beam Energy Scan (BES) program at the relativistic heavy ion collider (RHIC) at Brookhaven National Laboratory. The physics goal of the BES program is the search for a conjectured QCD critical point as well as for manifestations of the chiral magnetic effect. We describe progress that has been made over the previous five years. This includes studies of the equation of state and equilibrium susceptibilities, the development of suitable initial state models, progress in constructing a hydrodynamic framework that includes fluctuations and anomalous transport effects, as well as the development of freezeout prescriptions and hadronic transport models. Finally, we address the challenge of integrating these components into a complete analysis framework. This document describes the collective effort of the BEST Collaboration and its collaborators around the world.

nucl-th↗

arXiv2021

ICDAR 2021 Competition on Scientific Table Image Recognition to LaTeX

Tables present important information concisely in many scientific documents. Visual features like mathematical symbols, equations, and spanning cells make structure and content extraction from tables embedded in research documents difficult. This paper discusses the dataset, tasks, participants' methods, and results of the ICDAR 2021 Competition on Scientific Table Image Recognition to LaTeX. Specifically, the task of the competition is to convert a tabular image to its corresponding LaTeX source code. We proposed two subtasks. In Subtask 1, we ask the participants to reconstruct the LaTeX structure code from an image. In Subtask 2, we ask the participants to reconstruct the LaTeX content code from an image. This report describes the datasets and ground truth specification, details the performance evaluation metrics used, presents the final results, and summarizes the participating methods. Submission by team VCGroup got the highest Exact Match accuracy score of 74% for Subtask 1 and 55% for Subtask 2, beating previous baselines by 5% and 12%, respectively. Although improvements can still be made to the recognition capabilities of models, this competition contributes to the development of fully automated table recognition systems by challenging practitioners to solve problems under specific constraints and sharing their approaches; the platform will remain available for post-challenge submissions at https://competitions.codalab.org/competitions/26979 .

cs.IR↗

arXiv2021

Data InStance Prior (DISP) in Generative Adversarial Networks

Recent advances in generative adversarial networks (GANs) have shown remarkable progress in generating high-quality images. However, this gain in performance depends on the availability of a large amount of training data. In limited data regimes, training typically diverges, and therefore the generated samples are of low quality and lack diversity. Previous works have addressed training in low data setting by leveraging transfer learning and data augmentation techniques. We propose a novel transfer learning method for GANs in the limited data domain by leveraging informative data prior derived from self-supervised/supervised pre-trained networks trained on a diverse source domain. We perform experiments on several standard vision datasets using various GAN architectures (BigGAN, SNGAN, StyleGAN2) to demonstrate that the proposed method effectively transfers knowledge to domains with few target images, outperforming existing state-of-the-art techniques in terms of image quality and diversity. We also show the utility of data instance prior in large-scale unconditional image generation.

cs.CV↗

arXiv2021

PoliWAM: An Exploration of a Large Scale Corpus of Political Discussions on WhatsApp Messenger

WhatsApp Messenger is one of the most popular channels for spreading information with a current reach of more than 180 countries and 2 billion people. Its widespread usage has made it one of the most popular media for information propagation among the masses during any socially engaging event. In the recent past, several countries have witnessed its effectiveness and influence in political and social campaigns. We observe a high surge in information and propaganda flow during election campaigning. In this paper, we explore a high-quality large-scale user-generated dataset curated from WhatsApp comprising of 281 groups, 31,078 unique users, and 223,404 messages shared before, during, and after the Indian General Elections 2019, encompassing all major Indian political parties and leaders. In addition to the raw noisy user-generated data, we present a fine-grained annotated dataset of 3,848 messages that will be useful to understand the various dimensions of WhatsApp political campaigning. We present several complementary insights into the investigative and sensational news stories from the same period. Exploratory data analysis and experiments showcase several exciting results and future research opportunities. To facilitate reproducible research, we make the anonymized datasets available in the public domain.

cs.SI↗

arXiv2021

COMPARE: A Taxonomy and Dataset of Comparison Discussions in Peer Reviews

Comparing research papers is a conventional method to demonstrate progress in experimental research. We present COMPARE, a taxonomy and a dataset of comparison discussions in peer reviews of research papers in the domain of experimental deep learning. From a thorough observation of a large set of review sentences, we build a taxonomy of categories in comparison discussions and present a detailed annotation scheme to analyze this. Overall, we annotate 117 reviews covering 1,800 sentences. We experiment with various methods to identify comparison sentences in peer reviews and report a maximum F1 Score of 0.49. We also pretrain two language models specifically on ML, NLP, and CV paper abstracts and reviews to learn informative representations of peer reviews. The annotated dataset and the pretrained models are available at https://github.com/shruti-singh/COMPARE .

cs.CL↗