arXiv subjects

Jonathan Nagler

Publications and source records attributed to Jonathan Nagler.

6 recordsLinked to original sources

arXiv2025

The relationship between offline partisan geographical segregation and online partisan segregation

Social media is often blamed for the creation of echo chambers. However, these claims fail to consider the prevalence of offline echo chambers resulting from high levels of partisan segregation in the United States. Our article empirically assesses these online versus offline dynamics by linking a novel dataset of voters' offline partisan segregation extracted from publicly available voter files for 180 million US voters with their online network segregation on Twitter. We investigate offline and online partisan segregation using measures of geographical and network isolation of every matched voter-twitter user to their co-partisans online and offline. Our results show that while social media users tend to form politically homogeneous online networks, these levels of partisan sorting are significantly lower than those found in offline settings. Notably, Democrats are more isolated than Republicans in both settings, and only older Republicans exhibit higher online than offline segregation. Our results contribute to the emerging literature on political communication and the homophily of online networks, providing novel evidence on partisan sorting both online and offline.

cs.SI↗

arXiv2025

Concept-Guided Chain-of-Thought Prompting for Pairwise Comparison Scoring of Texts with Large Language Models

Existing text scoring methods require a large corpus, struggle with short texts, or require hand-labeled data. We develop a text scoring framework that leverages generative large language models (LLMs) to (1) set texts against the backdrop of information from the near-totality of the web and digitized media, and (2) effectively transform pairwise text comparisons from a reasoning problem to a pattern recognition task. Our approach, concept-guided chain-of-thought (CGCoT), utilizes a chain of researcher-designed prompts with an LLM to generate a concept-specific breakdown for each text, akin to guidance provided to human coders. We then pairwise compare breakdowns using an LLM and aggregate answers into a score using a probability model. We apply this approach to better understand speech reflecting aversion to specific political parties on Twitter, a topic that has commanded increasing interest because of its potential contributions to democratic backsliding. We achieve stronger correlations with human judgments than widely used unsupervised text scoring methods like Wordfish. In a supervised setting, besides a small pilot dataset to develop CGCoT prompts, our measures require no additional hand-labeled data and produce predictions on par with RoBERTa-Large fine-tuned on thousands of hand-labeled tweets. This project showcases the potential of combining human expertise and LLMs for scoring tasks.

cs.CL↗

arXiv2023

Large Language Models Can Be Used to Estimate the Latent Positions of Politicians

Existing approaches to estimating politicians' latent positions along specific dimensions often fail when relevant data is limited. We leverage the embedded knowledge in generative large language models (LLMs) to address this challenge and measure lawmakers' positions along specific political or policy dimensions. We prompt an instruction/dialogue-tuned LLM to pairwise compare lawmakers and then scale the resulting graph using the Bradley-Terry model. We estimate novel measures of U.S. senators' positions on liberal-conservative ideology, gun control, and abortion. Our liberal-conservative scale, used to validate LLM-driven scaling, strongly correlates with existing measures and offsets interpretive gaps, suggesting LLMs synthesize relevant data from internet and digitized media rather than memorizing existing measures. Our gun control and abortion measures -- the first of their kind -- differ from the liberal-conservative scale in face-valid ways and predict interest group ratings and legislator votes better than ideology alone. Our findings suggest LLMs hold promise for solving complex social science measurement problems.

cs.CY↗

arXiv2022

Dictionary-Assisted Supervised Contrastive Learning

Text analysis in the social sciences often involves using specialized dictionaries to reason with abstract concepts, such as perceptions about the economy or abuse on social media. These dictionaries allow researchers to impart domain knowledge and note subtle usages of words relating to a concept(s) of interest. We introduce the dictionary-assisted supervised contrastive learning (DASCL) objective, allowing researchers to leverage specialized dictionaries when fine-tuning pretrained language models. The text is first keyword simplified: a common, fixed token replaces any word in the corpus that appears in the dictionary(ies) relevant to the concept of interest. During fine-tuning, a supervised contrastive objective draws closer the embeddings of the original and keyword-simplified texts of the same class while pushing further apart the embeddings of different classes. The keyword-simplified texts of the same class are more textually similar than their original text counterparts, which additionally draws the embeddings of the same class closer together. Combining DASCL and cross-entropy improves classification performance metrics in few-shot learning settings and social science applications compared to using cross-entropy alone and alternative contrastive and data augmentation methods.

cs.CL↗

arXiv2022

Understanding how people consume low quality and extreme news using web traffic data

To mitigate the spread of fake news, researchers need to understand who visit fake new sites, what brings people to those sites, where visitors come from, and what content they prefer to consume. In this paper, we analyze web traffic data from The Gateway Pundit (TGP), a popular far-right website that is known for repeatedly sharing false information that has made its web traffic available to the general public. We collect data on 68 million web traffic visits to the site over a month period and analyze how people consume news via multiple features. Our traffic analysis shows that search engines and social media platforms are main drivers of traffic; our geo-location analysis reveals that TGP is more popular in counties that voted for Trump in 2020; and our topic analysis shows that conspiratorial articles receive more visits than factual articles. Due to the inability to observe direct website traffic, existing research uses alternative data source such as engagement signals from social media posts. To validate if social media engagement signals correlate with actual web visit counts, we collect all Facebook and Twitter posts with URLs from TGP during the same time period. We show that all engagement signals positively correlate with web visit counts, but with varying correlation strengths. Metrics based on Facebook posts correlate better than metrics based on Twitter. Our unique web traffic data set and insights can help researchers to better measure the impact of far-right and fake news URLs on social media platforms.

cs.SI↗

arXiv2021

YouTube Recommendations and Effects on Sharing Across Online Social Platforms

In January 2019, YouTube announced it would exclude potentially harmful content from video recommendations but allow such videos to remain on the platform. While this step intends to reduce YouTube's role in propagating such content, continued availability of these videos in other online spaces makes it unclear whether this compromise actually reduces their spread. To assess this impact, we apply interrupted time series models to measure whether different types of YouTube sharing in Twitter and Reddit changed significantly in the eight months around YouTube's announcement. We evaluate video sharing across three curated sets of potentially harmful, anti-social content: a set of conspiracy videos that have been shown to experience reduced recommendations in YouTube, a larger set of videos posted by conspiracy-oriented channels, and a set of videos posted by alternative influence network (AIN) channels. As a control, we also evaluate effects on video sharing in a dataset of videos from mainstream news channels. Results show conspiracy-labeled and AIN videos that have evidence of YouTube's de-recommendation experience a significant decreasing trend in sharing on both Twitter and Reddit. For videos from conspiracy-oriented channels, however, we see no significant effect in Twitter but find a significant increase in the level of conspiracy-channel sharing in Reddit. For mainstream news sharing, we actually see an increase in trend on both platforms, suggesting YouTube's suppressing particular content types has a targeted effect. This work finds evidence that reducing exposure to anti-social videos within YouTube, without deletion, has potential pro-social, cross-platform effects. At the same time, increases in the level of conspiracy-channel sharing raise concerns about content producers' responses to these changes, and platform transparency is needed to evaluate these effects further.

cs.SI↗