arXiv subjects

Rahul Jha

Publications and source records attributed to Rahul Jha.

13 recordsLinked to original sources

arXiv2024

Reindex-Then-Adapt: Improving Large Language Models for Conversational Recommendation

Large language models (LLMs) are revolutionizing conversational recommender systems by adeptly indexing item content, understanding complex conversational contexts, and generating relevant item titles. However, controlling the distribution of recommended items remains a challenge. This leads to suboptimal performance due to the failure to capture rapidly changing data distributions, such as item popularity, on targeted conversational recommendation platforms. In conversational recommendation, LLMs recommend items by generating the titles (as multiple tokens) autoregressively, making it difficult to obtain and control the recommendations over all items. Thus, we propose a Reindex-Then-Adapt (RTA) framework, which converts multi-token item titles into single tokens within LLMs, and then adjusts the probability distributions over these single-token item titles accordingly. The RTA framework marries the benefits of both LLMs and traditional recommender systems (RecSys): understanding complex queries as LLMs do; while efficiently controlling the recommended item distributions in conversational recommendations as traditional RecSys do. Our framework demonstrates improved accuracy metrics across three different conversational recommendation datasets and two adaptation settings

cs.IR↗

arXiv2023

Large Language Models as Zero-Shot Conversational Recommenders

In this paper, we present empirical studies on conversational recommendation tasks using representative large language models in a zero-shot setting with three primary contributions. (1) Data: To gain insights into model behavior in "in-the-wild" conversational recommendation scenarios, we construct a new dataset of recommendation-related conversations by scraping a popular discussion website. This is the largest public real-world conversational recommendation dataset to date. (2) Evaluation: On the new dataset and two existing conversational recommendation datasets, we observe that even without fine-tuning, large language models can outperform existing fine-tuned conversational recommendation models. (3) Analysis: We propose various probing tasks to investigate the mechanisms behind the remarkable performance of large language models in conversational recommendation. We analyze both the large language models' behaviors and the characteristics of the datasets, providing a holistic understanding of the models' effectiveness, limitations and suggesting directions for the design of future conversational recommenders

cs.IR↗

arXiv2021

GO FIGURE: A Meta Evaluation of Factuality in Summarization

While neural language models can generate text with remarkable fluency and coherence, controlling for factual correctness in generation remains an open research question. This major discrepancy between the surface-level fluency and the content-level correctness of neural generation has motivated a new line of research that seeks automatic metrics for evaluating the factuality of machine text. In this paper, we introduce GO FIGURE, a meta-evaluation framework for evaluating factuality evaluation metrics. We propose five necessary and intuitive conditions to evaluate factuality metrics on diagnostic factuality data across three different summarization tasks. Our benchmark analysis on ten factuality metrics reveals that our meta-evaluation framework provides a robust and efficient evaluation that is extensible to multiple types of factual consistency and standard generation metrics, including QA metrics. It also reveals that while QA metrics generally improve over standard metrics that measure factuality across domains, performance is highly dependent on the way in which questions are generated.

cs.CL↗

arXiv2021

QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization

Meetings are a key component of human collaboration. As increasing numbers of meetings are recorded and transcribed, meeting summaries have become essential to remind those who may or may not have attended the meetings about the key decisions made and the tasks to be completed. However, it is hard to create a single short summary that covers all the content of a long meeting involving multiple people and topics. In order to satisfy the needs of different types of users, we define a new query-based multi-domain meeting summarization task, where models have to select and summarize relevant spans of meetings in response to a query, and we introduce QMSum, a new benchmark for this task. QMSum consists of 1,808 query-summary pairs over 232 meetings in multiple domains. Besides, we investigate a locate-then-summarize method and evaluate a set of strong summarization baselines on the task. Experimental results and manual analysis reveal that QMSum presents significant challenges in long meeting summarization for future research. Dataset is available at \url{https://github.com/Yale-LILY/QMSum}.

cs.CL↗

arXiv2021

AREDSUM: Adaptive Redundancy-Aware Iterative Sentence Ranking for Extractive Document Summarization

Redundancy-aware extractive summarization systems score the redundancy of the sentences to be included in a summary either jointly with their salience information or separately as an additional sentence scoring step. Previous work shows the efficacy of jointly scoring and selecting sentences with neural sequence generation models. It is, however, not well-understood if the gain is due to better encoding techniques or better redundancy reduction approaches. Similarly, the contribution of salience versus diversity components on the created summary is not studied well. Building on the state-of-the-art encoding methods for summarization, we present two adaptive learning models: AREDSUM-SEQ that jointly considers salience and novelty during sentence selection; and a two-step AREDSUM-CTX that scores salience first, then learns to balance salience and redundancy, enabling the measurement of the impact of each aspect. Empirical results on CNN/DailyMail and NYT50 datasets show that by modeling diversity explicitly in a separate step, AREDSUM-CTX achieves significantly better performance than AREDSUM-SEQ as well as state-of-the-art extractive summarization baselines.

cs.CL↗

arXiv2020

Artemis: A Novel Annotation Methodology for Indicative Single Document Summarization

We describe Artemis (Annotation methodology for Rich, Tractable, Extractive, Multi-domain, Indicative Summarization), a novel hierarchical annotation process that produces indicative summaries for documents from multiple domains. Current summarization evaluation datasets are single-domain and focused on a few domains for which naturally occurring summaries can be easily found, such as news and scientific articles. These are not sufficient for training and evaluation of summarization models for use in document management and information retrieval systems, which need to deal with documents from multiple domains. Compared to other annotation methods such as Relative Utility and Pyramid, Artemis is more tractable because judges don't need to look at all the sentences in a document when making an importance judgment for one of the sentences, while providing similarly rich sentence importance annotations. We describe the annotation process in detail and compare it with other similar evaluation systems. We also present analysis and experimental results over a sample set of 532 annotated documents.

cs.CL↗

arXiv2018

Zero-Shot Adaptive Transfer for Conversational Language Understanding

Conversational agents such as Alexa and Google Assistant constantly need to increase their language understanding capabilities by adding new domains. A massive amount of labeled data is required for training each new domain. While domain adaptation approaches alleviate the annotation cost, prior approaches suffer from increased training time and suboptimal concept alignments. To tackle this, we introduce a novel Zero-Shot Adaptive Transfer method for slot tagging that utilizes the slot description for transferring reusable concepts across domains, and enjoys efficient training without any explicit concept alignments. Extensive experimentation over a dataset of 10 domains relevant to our commercial personal digital assistant shows that our model outperforms previous state-of-the-art systems by a large margin, and achieves an even higher improvement in the low data regime.

cs.CL↗

arXiv2017

Neutron Stars in Screened Modified Gravity: Chameleon vs Dilaton

We consider the scalar field profile around relativistic compact objects such as neutron stars for a range of modified gravity models with screening mechanisms of the chameleon and Damour-Polyakov types. We focus primarily on inverse power law chameleons and the environmentally dependent dilaton as examples of both mechanisms. We discuss the modified Tolman-Oppenheimer-Volkoff equation and then implement a relaxation algorithm to solve for the scalar profiles numerically. We find that chameleons and dilatons behave in a similar manner and that there is a large degeneracy between the modified gravity parameters and the neutron star equation of state. This is exemplified by the modifications to the mass-radius relationship for a variety of model parameters.

gr-qc↗

arXiv2016

Black hole accretion discs and screened scalar hair

We present a novel way to investigate scalar field profiles around black holes with an accretion disc for a range of models where the Compton wavelength of the scalar is large compared to other length scales. By analysing the problem in "Weyl" coordinates, we are able to calculate the scalar profiles for accretion discs in the static Schwarzschild, as well as rotating Kerr, black holes. We comment on observational effects.

gr-qc↗

arXiv2015

Non-Existence of Black Holes with Non-Canonical Scalar Fields

We study the existence of stationary black holes with a non-canonical scalar field as a matter source. We prove a simple black hole no-hair theorem which rules out the existence of asymptotically flat black holes which are static or stationary and axisymmetric possessing scalar hair for a wide class of non-canonical scalar field theories. This applies to scalar field theories which are of the form of K-essence theories. In particular, we rule out the existence of such black holes in the ghost condensate model, and in large sectors of the Dirac-Born-Infeld model.

gr-qc↗

arXiv2015

Humor in Collective Discourse: Unsupervised Funniness Detection in the New Yorker Cartoon Caption Contest

The New Yorker publishes a weekly captionless cartoon. More than 5,000 readers submit captions for it. The editors select three of them and ask the readers to pick the funniest one. We describe an experiment that compares a dozen automatic methods for selecting the funniest caption. We show that negative sentiment, human-centeredness, and lexical centrality most strongly match the funniest captions, followed by positive sentiment. These results are useful for understanding humor and also in the design of more engaging conversational agents in text and multimodal (vision+text) systems. As part of this work, a large set of cartoons and captions is being made available to the community.

cs.CL↗

arXiv2014

Astrophysical black holes in screened modified gravity

Chameleon, environmentally dependent dilaton, and symmetron gravity are three models of modified gravity in which the effects of the additional scalar degree of freedom are screened in dense environments. They have been extensively studied in laboratory, cosmological, and astrophysical contexts. In this paper, we present a preliminary investigation into whether additional constraints can be provided by studying these scalar fields around black holes. By looking at the properties of a static, spherically symmetric black hole, we find that the presence of a non-uniform matter distribution induces a non-constant scalar profile in chameleon and dilaton, but not necessarily symmetron gravity. An order of magnitude estimate shows that the effects of these profiles on in-falling test particles will be sub-leading compared to gravitational waves and hence observationally challenging to detect.

astro-ph.CO↗

arXiv2014

Stationary Black Holes with Time-Dependent Scalar Fields

It has been well known since the 1970s that stationary black holes do not generically support scalar hair. Most of the no-hair theorems which support this depend crucially upon the assumption that the scalar field has no time dependence. Here we fill in this omission by ruling out the existence of stationary black hole solutions even when the scalar field may have time dependence. Our proof is fairly general, and in particular applies to non-canonical scalar fields and certain non-asymptotically flat spacetimes. It also does not rely upon the spacetime being a black hole.

gr-qc↗