arXiv subjects

Nikita Gupta

Publications and source records attributed to Nikita Gupta.

5 recordsLinked to original sources

arXiv2026

DeepSearchQA: Bridging the Comprehensiveness Gap for Deep Research Agents

We introduce DeepSearchQA, a 900-prompt benchmark for evaluating agents on difficult multi-step information-seeking tasks across 17 different fields. Unlike traditional benchmarks that target single answer retrieval or broad-spectrum factuality, DeepSearchQA features a dataset of challenging, handcrafted tasks designed to evaluate an agent's ability to execute complex search plans to generate exhaustive answer lists. This shift in design explicitly tests three critical, yet under-evaluated capabilities: 1) systematic collation of fragmented information from disparate sources, 2) de-duplication and entity resolution to ensure precision, and 3) the ability to reason about stopping criteria within an open-ended search space. Each task is structured as a causal chain, where discovering information for one step is dependent on the successful completion of the previous one, stressing long-horizon planning and context retention. All tasks are grounded in the open web with objectively verifiable answer sets. Our comprehensive evaluation of state-of-the-art agent architectures reveals significant performance limitations: even the most advanced models struggle to balance high recall with precision. We observe distinct failure modes ranging from premature stopping (under-retrieval) to hedging behaviors, where agents cast an overly wide net of low-confidence answers to artificially boost recall. These findings highlight critical headroom in current agent designs and position DeepSearchQA as an essential diagnostic tool for driving future research toward more robust, deep-research capabilities.

cs.CL↗

arXiv2025

The FACTS Leaderboard: A Comprehensive Benchmark for Large Language Model Factuality

We introduce The FACTS Leaderboard, an online leaderboard suite and associated set of benchmarks that comprehensively evaluates the ability of language models to generate factually accurate text across diverse scenarios. The suite provides a holistic measure of factuality by aggregating the performance of models on four distinct sub-leaderboards: (1) FACTS Multimodal, which measures the factuality of responses to image-based questions; (2) FACTS Parametric, which assesses models' world knowledge by answering closed-book factoid questions from internal parameters; (3) FACTS Search, which evaluates factuality in information-seeking scenarios, where the model must use a search API; and (4) FACTS Grounding (v2), which evaluates whether long-form responses are grounded in provided documents, featuring significantly improved judge models. Each sub-leaderboard employs automated judge models to score model responses, and the final suite score is an average of the four components, designed to provide a robust and balanced assessment of a model's overall factuality. The FACTS Leaderboard Suite will be actively maintained, containing both public and private splits to allow for external participation while guarding its integrity. It can be found at https://www.kaggle.com/benchmarks/google/facts .

cs.CL↗

arXiv2023

Patterns of Social Vulnerability -- An Interactive Dashboard to Explore Risks to Public Health on the US County Level

Social vulnerability is the susceptibility of a community to be adversely impacted by natural hazards and public health emergencies, such as drought, earthquakes, flooding, virus outbreaks, and the like. Climate change is at the root of many recent natural hazards while the COVID-19 pandemic is still an active threat. Social vulnerability also refers to resilience, or the ability to recover from such adverse events. To gauge the many aspects of social vulnerability the US Center of Disease Control (CDC) has subdivided social vulnerabilities into distinct themes, such as socioeconomic status, household composition, and others. Knowing a community's social vulnerabilities can help policymakers and responders to recognize risks to community health, prepare for possible hazards, or recover from disasters. In this paper we study social vulnerabilities on the US county level and present research that suggests that there are certain combinations, or patterns, of social vulnerability indicators into which US counties can be grouped. We then present an interactive dashboard that allows analysts to explore these patterns in various ways. We demonstrate our methodology using COVID-19 death rate as the hazard and show that the patterns we identified have high predictive capabilities of the pandemic's local impact.

cs.HC↗

arXiv2022

Entity-Centric Query Refinement

We introduce the task of entity-centric query refinement. Given an input query whose answer is a (potentially large) collection of entities, the task output is a small set of query refinements meant to assist the user in efficient domain exploration and entity discovery. We propose a method to create a training dataset for this task. For a given input query, we use an existing knowledge base taxonomy as a source of candidate query refinements, and choose a final set of refinements from among these candidates using a search procedure designed to partition the set of entities answering the input query. We demonstrate that our approach identifies refinement sets which human annotators judge to be interesting, comprehensive, and non-redundant. In addition, we find that a text generation model trained on our newly-constructed dataset is able to offer refinements for novel queries not covered by an existing taxonomy. Our code and data are available at https://github.com/google-research/language/tree/master/language/qresp.

cs.CL↗

arXiv2015

Photoemission and optical constant measurements of Cesium Iodide thin film photocathode

The performance of cesium iodide as a reflective photocathode is presented. The absolute quantum efficiency of a 500 nm thick film of cesium iodide has been measured in the wavelength range 150 nm to 200 nm. The optical absorbance has been analyzed in the wavelength range 190 nm to 900 nm and the optical band gap energy has been calculated. The dispersion properties were determined from the refractive index using an envelope plot of the transmittance data. The morphological and elemental film composition have been investigated by atomic force microscopy and X-ray photo-electron spectroscopy techniques.

physics.ins-det↗