arXiv subjects

Adam Visokay

Publications and source records attributed to Adam Visokay.

6 recordsLinked to original sources

arXiv2026

Multi-Perspective LLM Annotations for Valid Analyses in Subjective Tasks

Large language models are increasingly used to annotate texts, but their outputs reflect some human perspectives better than others. Existing methods for correcting LLM annotation error assume a single ground truth. However, this assumption fails in subjective tasks where disagreement across demographic groups is meaningful. Here we introduce Perspective-Driven Inference, a method that treats the distribution of annotations across groups as the quantity of interest, and estimates it using a small human annotation budget. We contribute an adaptive sampling strategy that concentrates human annotation effort on groups where LLM proxies are least accurate. We evaluate on politeness and offensiveness rating tasks, showing targeted improvements for harder-to-model demographic groups relative to uniform sampling baselines, while maintaining coverage.

cs.CL↗

arXiv2026

Randomized Recruitment Driven Sampling

Surveys are critical inputs for research and policy, yet, enumerating a sampling frame is logistically infeasible or financially nonviable in many circumstances, such as during pandemics, natural disasters, or armed conflict. Respondent Driven Sampling (RDS) does not require a sampling frame, yet non-random peer recruitment often introduces substantial bias, particularly under high homophily. We introduce and evaluate Randomized Recruitment Driven Sampling (RRDS), a cellphone-based adaptation of RDS that incorporates researcher-controlled randomization into each recruitment wave. While standard RDS is necessary for stigmatized groups where network transparency is infeasible, RRDS is designed for low-stigma populations that become difficult to access due to logistical barriers. In these contexts, RRDS enforces the random recruitment assumption that traditional RDS relies upon but rarely achieves. Through simulation and an experiment surveying Bangladeshi garment workers during the COVID-19 pandemic, we demonstrate that RRDS produces less biased estimates and improved confidence interval coverage compared to traditional RDS. RRDS offers a scalable, remote-compatible alternative for studying low-stigma groups in challenging contexts where large-scale probability sampling is unsafe or infeasible.

stat.AP↗

arXiv2025

Social Construction of Urban Space: Using LLMs to Identify Neighborhood Boundaries From Craigslist Ads

Rental listings offer a window into how urban space is socially constructed through language. We analyze Chicago Craigslist rental advertisements from 2018 to 2024 to examine how listing agents characterize neighborhoods, identifying mismatches between institutional boundaries and neighborhood claims. Through manual and large language model annotation, we classify unstructured listings from Craigslist according to their neighborhood. Further geospatial analysis reveals three distinct patterns: properties with conflicting neighborhood designations due to competing spatial definitions, border properties with valid claims to adjacent neighborhoods, and "reputation laundering" where listings claim association with distant, desirable neighborhoods. Through topic modeling, we identify patterns that correlate with spatial positioning: listings further from neighborhood centers emphasize different amenities than centrally-located units. Natural language processing techniques reveal how definitions of urban spaces are contested in ways that traditional methods overlook.

cs.CL↗

arXiv2024

GPT Deciphering Fedspeak: Quantifying Dissent Among Hawks and Doves

Markets and policymakers around the world hang on the consequential monetary policy decisions made by the Federal Open Market Committee (FOMC). Publicly available textual documentation of their meetings provides insight into members' attitudes about the economy. We use GPT-4 to quantify dissent among members on the topic of inflation. We find that transcripts and minutes reflect the diversity of member views about the macroeconomic outlook in a way that is lost or omitted from the public statements. In fact, diverging opinions that shed light upon the committee's "true" attitudes are almost entirely omitted from the final statements. Hence, we argue that forecasting FOMC sentiment based solely on statements will not sufficiently reflect dissent among the hawks and doves.

cs.AI↗

arXiv2024

From Narratives to Numbers: Valid Inference Using Language Model Predictions from Verbal Autopsy Narratives

In settings where most deaths occur outside the healthcare system, verbal autopsies (VAs) are a common tool to monitor trends in causes of death (COD). VAs are interviews with a surviving caregiver or relative that are used to predict the decedent's COD. Turning VAs into actionable insights for researchers and policymakers requires two steps (i) predicting likely COD using the VA interview and (ii) performing inference with predicted CODs (e.g. modeling the breakdown of causes by demographic factors using a sample of deaths). In this paper, we develop a method for valid inference using outcomes (in our case COD) predicted from free-form text using state-of-the-art NLP techniques. This method, which we call multiPPI++, extends recent work in "prediction-powered inference" to multinomial classification. We leverage a suite of NLP techniques for COD prediction and, through empirical analysis of VA data, demonstrate the effectiveness of our approach in handling transportability issues. multiPPI++ recovers ground truth estimates, regardless of which NLP model produced predictions and regardless of whether they were produced by a more accurate predictor like GPT-4-32k or a less accurate predictor like KNN. Our findings demonstrate the practical importance of inference correction for public health decision-making and suggests that if inference tasks are the end goal, having a small amount of contextually relevant, high quality labeled data is essential regardless of the NLP algorithm.

cs.CL↗

arXiv2023

Respondent-Driven Sampling: An Overview in the Context of Human Trafficking

Respondent-driven sampling (RDS) is both a sampling strategy and an estimation method. It is commonly used to study individuals that are difficult to access with standard sampling techniques. As with any sampling strategy, RDS has advantages and challenges. This article examines recent work using RDS in the context of human trafficking. We begin with an overview of the RDS process and methodology, then discuss RDS in the particular context of trafficking. We end with a description of recent work and potential future directions.

stat.AP↗