arXiv subjects

Roberto Ulloa

Publications and source records attributed to Roberto Ulloa.

At least 19 recordsLinked to original sources

arXiv2026

The Knowledge Gap in a High-Choice Media Environment: Experimental Evidence from Online Search

Persistent inequalities in political knowledge are a central concern in political communication. We organize the mechanisms underlying the knowledge-gap literature by distinguishing between individual preconditions, structural features of the information environment, and topic characteristics. Within this framework, we note that self-directed information seeking, a prototypical form of intentional exposure, has received little attention despite its importance in navigating today's complex information environment. We conducted a field experiment in Germany combining randomized encouragements and passive browser tracking to examine how individuals with varying education levels acquire policy-specific knowledge through online search. Participants were randomly assigned to one of three conditions (verbal encouragement, financial encouragement, or control) to seek information on three salient policy topics differing in divisiveness and complexity (child support, energy transition, and cannabis legalization). We estimate both intention-to-treat (ITT) and local average treatment effects (LATE) of information seeking on post-search knowledge outcomes, with a focus on education and civic knowledge as moderators. While the interventions equalized information-seeking behavior, the results provide some support for the knowledge gap hypothesis: knowledge gains were concentrated among participants with higher education or baseline civic knowledge, who, according to our post-hoc exploratory analyses, appeared more effective at navigating search results. These findings indicate that a narrowing of knowledge inequalities goes beyond motivation: it calls for both individual-level interventions to strengthen citizens' skills and structural-level adaptations to foster more equitable learning environments.

cs.CY↗

arXiv2025

Opting Out of Generative AI: a Behavioral Experiment on the Role of Education in Perplexity AI Avoidance

The rise of conversational AI (CAI), powered by large language models, is transforming how individuals access and interact with digital information. However, these tools may inadvertently amplify existing digital inequalities. This study investigates whether differences in formal education are associated with CAI avoidance, leveraging behavioral data from an online experiment (N = 1,636). Participants were randomly assigned to a control or an information-seeking task, either a traditional online search or a CAI (Perplexity AI). Task avoidance (operationalized as survey abandonment or providing unrelated responses during task assignment) was significantly higher in the CAI group (51%) compared to the search (30.9%) and control (16.8%) groups, with the highest CAI avoidance among participants with lower education levels (~74.4%). Structural equation modeling based on the theoretical framework UTAUT2 and LASSO regressions reveal that education is strongly associated with CAI avoidance, even after accounting for various cognitive and affective predictors of technology adoption. These findings underscore education's central role in shaping AI adoption and the role of self-selection biases in AI-related research, stressing the need for inclusive design to ensure equitable access to emerging technologies.

cs.CY↗

arXiv2025

From prosthetic memory to prosthetic denial: Auditing whether large language models are prone to mass atrocity denialism

The proliferation of large language models (LLMs) can influence how historical narratives are disseminated and perceived. This study explores the implications of LLMs' responses on the representation of mass atrocity memory, examining whether generative AI systems contribute to prosthetic memory, i.e., mediated experiences of historical events, or to what we term "prosthetic denial," the AI-mediated erasure or distortion of atrocity memories. We argue that LLMs function as interfaces that can elicit prosthetic memories and, therefore, act as experiential sites for memory transmission, but also introduce risks of denialism, particularly when their outputs align with contested or revisionist narratives. To empirically assess these risks, we conducted a comparative audit of five LLMs (Claude, GPT, Llama, Mixtral, and Gemini) across four historical case studies: the Holodomor, the Holocaust, the Cambodian Genocide, and the genocide against the Tutsis in Rwanda. Each model was prompted with questions addressing common denialist claims in English and an alternative language relevant to each case (Ukrainian, German, Khmer, and French). Our findings reveal that while LLMs generally produce accurate responses for widely documented events like the Holocaust, significant inconsistencies and susceptibility to denialist framings are observed for more underrepresented cases like the Cambodian Genocide. The disparities highlight the influence of training data availability and the probabilistic nature of LLM responses on memory integrity. We conclude that while LLMs extend the concept of prosthetic memory, their unmoderated use risks reinforcing historical denialism, raising ethical concerns for (digital) memory preservation, and potentially challenging the advantageous role of technology associated with the original values of prosthetic memory.

cs.CY↗

arXiv2025

Tag-Pag: A Dedicated Tool for Systematic Web Page Annotations

Tag-Pag is an application designed to simplify the categorization of web pages, a task increasingly common for researchers who scrape web pages to analyze individuals' browsing patterns or train machine learning classifiers. Unlike existing tools that focus on annotating sections of text, Tag-Pag systematizes page-level annotations, allowing users to determine whether an entire document relates to one or multiple predefined topics. Tag-Pag offers an intuitive interface to configure the input web pages and annotation labels. It integrates libraries to extract content from the HTML and URL indicators to aid the annotation process. It provides direct access to both scraped and live versions of the web page. Our tool is designed to expedite the annotation process with features like quick navigation, label assignment, and export functionality, making it a versatile and efficient tool for various research applications. Tag-Pag is available at https://github.com/Pantonius/TagPag.

cs.IR↗

arXiv2025

Self-directed online information search can affect policy support: a randomized encouragement design with digital behavioral data

As citizens increasingly encounter political information in digital environments, understanding whether this engagement shapes their policy views has become a central concern. Drawing on dual-process theories of persuasion, we argue that motivational activation is an enabling condition for policy support change in high-choice online environments. We test this in a three-wave field experiment with German participants (n = 791) across three policy topics (basic child support, renewable energy transition, cannabis legalization), in which participants were randomly assigned to a control group, and two encouragement conditions: a verbal encouragement, or a monetary incentive tied to a knowledge test. Browsing behavior was passively tracked via digital trace data over a 20-hour window. We find that self-directed online information search produced changes in policy support for child support and cannabis legalization but not for the energy transition, with monetary incentives producing significant effects rather than verbal prompts. We discuss motivational salience, issue malleability, and search-environment quality as joint conditions under which political information engagement can produce detectable changes in policy support.

cs.CY↗

arXiv2024

Beyond time delays: How web scraping distorts measures of online news consumption

As the exploration of digital behavioral data revolutionizes communication research, understanding the nuances of data collection methodologies becomes increasingly pertinent. This study focuses on one prominent data collection approach, web scraping, and more specifically, its application in the growing field of research relying on web browsing data. We investigate discrepancies between content obtained directly during user interaction with a website (in-situ) and content scraped using the URLs of participants' logged visits (ex-situ) with various time delays (0, 30, 60, and 90 days). We find substantial disparities between the methodologies, uncovering that errors are not uniformly distributed across news categories regardless of classification method (domain, URL, or content analysis). These biases compromise the precision of measurements used in existing literature. The ex-situ collection environment is the primary source of the discrepancies (~33.8%), while the time delays in the scraping process play a smaller role (adding ~6.5 percentage points in 90 days). Our research emphasizes the need for data collection methods that capture web content directly in the user's environment. However, acknowledging its complexities, we further explore strategies to mitigate biases in web-scraped browsing histories, offering recommendations for researchers who rely on this method and laying the groundwork for developing error-correction frameworks.

cs.CY↗

arXiv2024

Googling the Big Lie: Search Engines, News Media, and the US 2020 Election Conspiracy

The conspiracy theory that the US 2020 presidential election was fraudulent - the Big Lie - remained a prominent part of the media agenda months after the election. Whether and how search engines prioritized news stories that sought to thoroughly debunk the claims, provide a simple negation, or support the conspiracy is crucial for understanding information exposure on the topic. We investigate how search engines provided news on this conspiracy by conducting a large-scale algorithm audit evaluating differences between three search engines (Google, DuckDuckGo, and Bing), across three locations (Ohio, California, and the UK), and using eleven search queries. Results show that simply denying the conspiracy is the largest debunking strategy across all search engines. While Google has a strong mainstreaming effect on articles explicitly focused on the Big Lie - providing thorough debunks and alternative explanations - DuckDuckGo and Bing display, depending on the location, a large share of articles either supporting the conspiracy or failing to debunk it. Lastly, we find that niche ideologically driven search queries (e.g., "sharpie marker ballots Arizona") do not lead to more conspiracy-supportive material. Instead, content supporting the conspiracy is largely a product of broader ideology-agnostic search queries (e.g., "voter fraud 2020").

cs.IR↗

arXiv2024

Assessing In-context Learning and Fine-tuning for Topic Classification of German Web Data

Researchers in the political and social sciences often rely on classification models to analyze trends in information consumption by examining browsing histories of millions of webpages. Automated scalable methods are necessary due to the impracticality of manual labeling. In this paper, we model the detection of topic-related content as a binary classification task and compare the accuracy of fine-tuned pre-trained encoder models against in-context learning strategies. Using only a few hundred annotated data points per topic, we detect content related to three German policies in a database of scraped webpages. We compare multilingual and monolingual models, as well as zero and few-shot approaches, and investigate the impact of negative sampling strategies and the combination of URL & content-based features. Our results show that a small sample of annotated data is sufficient to train an effective classifier. Fine-tuning encoder-based models yields better results than in-context learning. Classifiers using both URL & content-based features perform best, while using URLs alone provides adequate results when content is unavailable.

cs.CL↗

arXiv2024

Improving the quality of individual-level online information tracking: challenges of existing approaches and introduction of a new content- and long-tail sensitive academic solution

This article evaluates the quality of data collection in individual-level desktop information tracking used in the social sciences and shows that the existing approaches face sampling issues, validity issues due to the lack of content-level data and their disregard of the variety of devices and long-tail consumption patterns as well as transparency and privacy issues. To overcome some of these problems, the article introduces a new academic tracking solution, WebTrack, an open source tracking tool maintained by a major European research institution. The design logic, the interfaces and the backend requirements for WebTrack, followed by a detailed examination of strengths and weaknesses of the tool, are discussed. Finally, using data from 1185 participants, the article empirically illustrates how an improvement in the data collection through WebTrack leads to new innovative shifts in the processing of tracking data. As WebTrack allows collecting the content people are exposed to on more than classical news platforms, we can strongly improve the detection of politics-related information consumption in tracking data with the application of automated content analysis compared to traditional approaches that rely on the list-based identification of news.

cs.CY↗

arXiv2023

Enhancing autonomous vehicle acceptance with age and education sensitive simulation interventions: An experimental trial

The familiarity principle posits that acceptance increases with exposure, which has previously been shown with in vivo and simulated experiences with connected and autonomous vehicles (CAVs). We investigate the impact of a simulated video-based first-person drive on CAV acceptance, as well as the impact of information customization, with a particular focus on acceptance by older individuals and those with lower education. Findings from an online experiment with N=799 German residents reveal that the simulated experience improved acceptance across response variables such as intention to use and ease of use, particularly among older individuals. However, the opportunity to customize navigation information decreased acceptance of older individuals and those with university degrees and increased acceptance for younger individuals and those with lower educational levels.

cs.HC↗

arXiv2023

Examining bias perpetuation in academic search engines: an algorithm audit of Google and Semantic Scholar

Researchers rely on academic Web search engines to find scientific sources, but search engine mechanisms may selectively present content that aligns with biases embedded in queries. This study examines whether confirmation biased queries prompted into Google Scholar and Semantic Scholar will yield results aligned with a query's bias. Six queries (topics across health and technology domains such as vaccines, Internet use) were analyzed for disparities in search results. We confirm that biased queries (targeting benefits or risks) affect search results in line with bias, with technology-related queries displaying more significant disparities. Overall, Semantic Scholar exhibited fewer disparities than Google Scholar. Topics rated as more polarizing did not consistently show more disparate results. Academic search results that perpetuate confirmation bias have strong implications for both researchers and citizens searching for evidence. More research is needed to explore how scientific inquiry and academic search engines interact.

cs.CY↗

arXiv2023

Characteristics of ChatGPT users from Germany: implications for the digital divide from web tracking data

A major challenge of our time is reducing disparities in access to and effective use of digital technologies, with recent discussions highlighting the role of AI in exacerbating the digital divide. We examine user characteristics that predict usage of the AI-powered conversational agent ChatGPT. We combine behavioral and survey data in a web tracked sample of N = 1376 German citizens to investigate differences in ChatGPT activity (usage, visits, and adoption) during the first 11 months from the launch of the service (November 30, 2022). Guided by a model of technology acceptance (UTAUT-2), we examine the role of socio-demographics commonly associated with the digital divide in ChatGPT activity and explore further socio-political attributes identified via stability selection in Lasso regressions. We confirm that lower age and higher education affect ChatGPT usage, but do not find that gender or income do. We find full-time employment and more children to be barriers to ChatGPT activity. Using a variety of social media was positively associated with ChatGPT activity. In terms of political variables, political knowledge and political self-efficacy as well as some political behaviors such as voting, debating political issues online and offline and political action online were all associated with ChatGPT activity, with online political debating and political self-efficacy negatively so. Finally, need for cognition and communication skills such as writing, attending meetings, or giving presentations, were also associated with ChatGPT engagement, though chairing/organizing meetings was negatively associated. Our research informs efforts to address digital disparities and promote digital literacy among underserved populations by presenting implications, recommendations, and discussions on ethical and social issues of our findings.

cs.CY↗

arXiv2023

Shall androids dream of genocides? How generative AI can change the future of memorialization of mass atrocities

The memorialization of mass atrocities such as war crimes and genocides facilitates the remembrance of past suffering, honors those who resisted the perpetrators, and helps prevent the distortion of historical facts. Digital technologies have transformed memorialization practices by enabling less top-down and more creative approaches to remember mass atrocities. At the same time, they may also facilitate the spread of denialism and distortion, attempt to justify past crimes and attack the dignity of victims. The emergence of generative forms of artificial intelligence (AI), which produce textual and visual content, has the potential to revolutionize the field of memorialization even further. AI can identify patterns in training data to create new narratives for representing and interpreting mass atrocities - and do so in a fraction of the time it takes for humans. The use of generative AI in this context raises numerous questions: For example, can the paucity of training data on mass atrocities distort how AI interprets some atrocity-related inquiries? How important is the ability to differentiate between human- and AI-made content concerning mass atrocities? Can AI-made content be used to promote false information concerning atrocities? This article addresses these and other questions by examining the opportunities and risks associated with using generative AIs for memorializing mass atrocities. It also discusses recommendations for AIs integration in memorialization practices to steer the use of these technologies toward a more ethical and sustainable direction.

cs.CY↗

arXiv2022

Novelty in news search: a longitudinal study of the 2020 US elections

The 2020 US elections news coverage was extensive, with new pieces of information generated rapidly. This evolving scenario presented an opportunity to study the performance of search engines in a context in which they had to quickly process information as it was published. We analyze novelty, a measurement of new items that emerge in the top news search results, to compare the coverage and visibility of different topics. We conduct a longitudinal study of news results of five search engines collected in short-bursts (every 21 minutes) from two regions (Oregon, US and Frankfurt, Germany), starting on election day and lasting until one day after the announcement of Biden as the winner. We find more new items emerging for election related queries ("joe biden", "donald trump" and "us elections") compared to topical (e.g., "coronavirus") or stable (e.g., "holocaust") queries. We demonstrate differences across search engines and regions over time, and we highlight imbalances between candidate queries. When it comes to news search, search engines are responsible for such imbalances, either due to their algorithms or the set of news sources they rely on. We argue that such imbalances affect the visibility of political candidates in news searches during electoral periods.

cs.CY↗

arXiv2022

This is what a pandemic looks like: Visual framing of COVID-19 on search engines

In today's high-choice media environment, search engines play an integral role in informing individuals and societies about the latest events. The importance of search algorithms is even higher at the time of crisis, when users search for information to understand the causes and the consequences of the current situation and decide on their course of action. In our paper, we conduct a comparative audit of how different search engines prioritize visual information related to COVID-19 and what consequences it has for the representation of the pandemic. Using a virtual agent-based audit approach, we examine image search results for the term "coronavirus" in English, Russian and Chinese on five major search engines: Google, Yandex, Bing, Yahoo, and DuckDuckGo. Specifically, we focus on how image search results relate to generic news frames (e.g., the attribution of responsibility, human interest, and economics) used in relation to COVID-19 and how their visual composition varies between the search engines.

cs.IR↗

arXiv2022

Search engine effects on news consumption: ranking and representativeness outweigh familiarity in news selection

Online platforms have transformed the way in which individuals access and interact with news, with a high degree of trust particularly placed in search engine results. We use web tracked behavioral data across a 2-month period and analyze three competing factors, two algorithmic (ranking and representativeness) and one psychological (familiarity) that could influence the selection of news articles that appear in search results. Participants' (n=280) news engagement is our proxy for familiarity, and we investigate news articles presented on Google search pages (n=1221). Our results demonstrate the steering power of the algorithmic factors on news consumption as compared to familiarity. But despite the strong effect of ranking, we find that it plays a lesser role for news articles compared to non-news. We confirm that Google Search drives individuals to unfamiliar sources and find that it increases the diversity of the political audience to news sources. With our methodology, we take a step in tackling the challenges of testing social science theories in digital contexts shaped by algorithms.

cs.CY↗

arXiv2022

Scaling up Search Engine Audits: Practical Insights for Algorithm Auditing

Algorithm audits have increased in recent years due to a growing need to independently assess the performance of automatically curated services that process, filter, and rank the large and dynamic amount of information available on the internet. Among several methodologies to perform such audits, virtual agents stand out because they offer the ability to perform systematic experiments, simulating human behaviour without the associated costs of recruiting participants. Motivated by the importance of research transparency and replicability of results, this paper focuses on the challenges of such an approach. It provides methodological details, recommendations, lessons learned, and limitations based on our experience of setting up experiments for eight search engines (including main, news, image and video sections) with hundreds of virtual agents placed in different regions. We demonstrate the successful performance of our research infrastructure across multiple data collections, with diverse experimental designs, and point to different changes and strategies that improve the quality of the method. We conclude that virtual agents are a promising venue for monitoring the performance of algorithms across long periods of time, and we hope that this paper can serve as a basis for further research in this area.

cs.CY↗

arXiv2021

Where the Earth is flat and 9/11 is an inside job: A comparative algorithm audit of conspiratorial information in web search results

Web search engines are important online information intermediaries that are frequently used and highly trusted by the public despite multiple evidence of their outputs being subjected to inaccuracies and biases. One form of such inaccuracy, which so far received little scholarly attention, is the presence of conspiratorial information, namely pages promoting conspiracy theories. We address this gap by conducting a comparative algorithm audit to examine the distribution of conspiratorial information in search results across five search engines: Google, Bing, DuckDuckGo, Yahoo and Yandex. Using a virtual agent-based infrastructure, we systematically collect search outputs for six conspiracy theory-related queries (flat earth, new world order, qanon, 9/11, illuminati, george soros) across three locations (two in the US and one in the UK) and two observation periods (March and May 2021). We find that all search engines except Google consistently displayed conspiracy-promoting results and returned links to conspiracy-dedicated websites in their top results, although the share of such content varied across queries. Most conspiracy-promoting results came from social media and conspiracy-dedicated websites while conspiracy-debunking information was shared by scientific websites and, to a lesser extent, legacy media. The fact that these observations are consistent across different locations and time periods highlight the possibility of some search engines systematically prioritizing conspiracy-promoting content and, thus, amplifying their distribution in the online environments.

cs.IR↗