Searcharxiv⌕ Search

arXiv subjects

Filippo Menczer

Publications and source records attributed to Filippo Menczer.

At least 37 records · Page 2Linked to original sources

Effects of Antivaccine Tweets on COVID-19 Vaccinations, Cases, and Deaths

Despite the wide availability of COVID-19 vaccines in the United States and their effectiveness in reducing hospitalizations and mortality during the pandemic, a majority of Americans chose not to be vaccinated during 2021. Recent work shows that vaccine misinformation affects intentions in controlled settings, but does not link it to real-world vaccination rates. Here, we present observational evidence of a causal relationship between exposure to antivaccine content and vaccination rates, and estimate the size of this effect. We present a compartmental epidemic model that includes vaccination, vaccine hesitancy, and exposure to antivaccine content. We fit the model to data to determine that a geographical pattern of exposure to online antivaccine content across US counties explains reduced vaccine uptake in the same counties. We find observational evidence that exposure to antivaccine content on Twitter caused about 14,000 people to refuse vaccination between February and August 2021 in the US, resulting in at least 545 additional cases and 8 additional deaths. This work provides a methodology for linking online speech with offline epidemic outcomes. Our findings should inform social media moderation policy as well as public health interventions.

cs.SI↗

Identifying Constructive Conflict in Online Discussions through Controversial yet Toxicity Resilient Posts

Bridging content that brings together individuals with opposing viewpoints on social media remains elusive, overshadowed by echo chambers and toxic exchanges. We propose that algorithmic curation could surface such content by considering constructive conflicts as a foundational criterion. We operationalize this criterion through controversiality to identify challenging dialogues and toxicity resilience to capture respectful conversations. We develop high-accuracy models to capture these dimensions. Analyses based on these models demonstrate that assessing resilience to toxic responses is not the same as identifying low-toxicity posts. We also find that political posts are often controversial and tend to attract more toxic responses. However, some posts, even the political ones, are resilient to toxicity despite being highly controversial, potentially sparking civil engagement. Toxicity resilient posts tend to use politeness cues, such as showing gratitude and hedging. These findings suggest the potential for framing the tone of posts to encourage constructive political discussions.

cs.SI↗

Demystifying Misconceptions in Social Bots Research

Research on social bots aims at advancing knowledge and providing solutions to one of the most debated forms of online manipulation. Yet, social bot research is plagued by widespread biases, hyped results, and misconceptions that set the stage for ambiguities, unrealistic expectations, and seemingly irreconcilable findings. Overcoming such issues is instrumental towards ensuring reliable solutions and reaffirming the validity of the scientific method. Here, we discuss a broad set of consequential methodological and conceptual issues that affect current social bots research, illustrating each with examples drawn from recent studies. More importantly, we demystify common misconceptions, addressing fundamental points on how social bots research is discussed. Our analysis surfaces the need to discuss research about online disinformation and manipulation in a rigorous, unbiased, and responsible way. This article bolsters such effort by identifying and refuting common fallacious arguments used by both proponents and opponents of social bots research, as well as providing directions toward sound methodologies for future research.

cs.SI↗

Community Moderation and the New Epistemology of Fact Checking on Social Media

Social media platforms have traditionally relied on internal moderation teams and partnerships with independent fact-checking organizations to identify and flag misleading content. Recently, however, platforms including X (formerly Twitter) and Meta have shifted towards community-driven content moderation by launching their own versions of crowd-sourced fact-checking -- Community Notes. If effectively scaled and governed, such crowd-checking initiatives have the potential to combat misinformation with increased scale and speed as successfully as community-driven efforts once did with spam. Nevertheless, general content moderation, especially for misinformation, is inherently more complex. Public perceptions of truth are often shaped by personal biases, political leanings, and cultural contexts, complicating consensus on what constitutes misleading content. This suggests that community efforts, while valuable, cannot replace the indispensable role of professional fact-checkers. Here we systemically examine the current approaches to misinformation detection across major platforms, explore the emerging role of community-driven moderation, and critically evaluate both the promises and challenges of crowd-checking at scale.

cs.SI↗

A longitudinal analysis of misinformation, polarization and toxicity on Bluesky after its public launch

Bluesky is a decentralized, Twitter-like social media platform that has rapidly gained popularity. Following an invite-only phase, it officially opened to the public on February 6th, 2024, leading to a significant expansion of its user base. In this paper, we present a longitudinal analysis of user activity in the two months surrounding its public launch, examining how the platform evolved due to this rapid growth. Our analysis reveals that Bluesky exhibits an activity distribution comparable to more established social platforms, yet it features a higher volume of original content relative to reshared posts and maintains low toxicity levels. We further investigate the political leanings of its user base, misinformation dynamics, and engagement in harmful conversations. Our findings indicate that Bluesky users predominantly lean left politically and tend to share high-credibility sources. After the platform's public launch, an influx of new users, particularly those posting in English and Japanese, contributed to a surge in activity. Among them, several accounts displayed suspicious behaviors, such as mass-following users and sharing content from low-credibility news sources. Some of these accounts have already been flagged as spam or suspended, suggesting that Bluesky's moderation efforts have been effective.

cs.SI↗

The Rise of Bluesky

This study investigates the rapid growth and evolving network structure of Bluesky from August 2023 to February 2025. Through multiple waves of user migrations, the platform has reached a stable, persistently active user base. The growth process has given rise to a dense follower network with clustering and hub features that favor viral information diffusion. These developments highlight engagement and structural similarities between Bluesky and established platforms.

cs.SI↗

Beyond Interaction Patterns: Assessing Claims of Coordinated Inter-State Information Operations on Twitter/X

Social media platforms have become key tools for coordinated influence operations, enabling state actors to manipulate public opinion through strategic, collective actions. While previous research has suggested collaboration between states, such research failed to leverage state-of-the-art coordination indicators or control datasets. In this study, we investigate inter-state coordination by analyzing multiple online behavioral traces and using sophisticated coordination detection models. By incorporating a control dataset to differentiate organic user activity from coordinated efforts, our findings reveal no evidence of inter-state coordination. These results challenge earlier claims and underscore the importance of robust methodologies and control datasets in accurately detecting online coordination.

cs.SI↗

Accuracy and Political Bias of News Source Credibility Ratings by Large Language Models

Search engines increasingly leverage large language models (LLMs) to generate direct answers, and AI chatbots now access the Internet for fresh data. As information curators for billions of users, LLMs must assess the accuracy and reliability of different sources. This paper audits nine widely used LLMs from three leading providers -- OpenAI, Google, and Meta -- to evaluate their ability to discern credible and high-quality information sources from low-credibility ones. We find that while LLMs can rate most tested news outlets, larger models more frequently refuse to provide ratings due to insufficient information, whereas smaller models are more prone to making errors in their ratings. For sources where ratings are provided, LLMs exhibit a high level of agreement among themselves (average Spearman's $ρ= 0.79$), but their ratings align only moderately with human expert evaluations (average $ρ= 0.50$). Analyzing news sources with different political leanings in the US, we observe a liberal bias in credibility ratings yielded by all LLMs in default configurations. Additionally, assigning partisan roles to LLMs consistently induces strong politically congruent bias in their ratings. These findings have important implications for the use of LLMs in curating news and political information.

cs.CL↗

Modeling the amplification of epidemic spread by individuals exposed to misinformation on social media

Understanding how misinformation affects the spread of disease is crucial for public health, especially given recent research indicating that misinformation can increase vaccine hesitancy and discourage vaccine uptake. However, it is difficult to investigate the interaction between misinformation and epidemic outcomes due to the dearth of data-informed holistic epidemic models. Here, we employ an epidemic model that incorporates a large, mobility-informed physical contact network as well as the distribution of misinformed individuals across counties derived from social media data. The model allows us to simulate various scenarios to understand how epidemic spreading can be affected by misinformation spreading through one particular social media platform. Using this model, we compare a worst-case scenario, in which individuals become misinformed after a single exposure to low-credibility content, to a best-case scenario where the population is highly resilient to misinformation. We estimate the additional portion of the U.S. population that would become infected over the course of the COVID-19 epidemic in the worst-case scenario. This work can provide policymakers with insights about the potential harms of exposure to online vaccine misinformation.

cs.SI↗

Labeled Datasets for Research on Information Operations

Social media platforms have become a hub for political activities and discussions, democratizing participation in these endeavors. However, they have also become an incubator for manipulation campaigns, like information operations (IOs). Some social media platforms have released datasets related to such IOs originating from different countries. However, we lack comprehensive control data that can enable the development of IO detection methods. To bridge this gap, we present new labeled datasets about 26 campaigns, which contain both IO posts verified by a social media platform and over 13M posts by 303k accounts that discussed similar topics in the same time frames (control data). The datasets will facilitate the study of narratives, network interactions, and engagement strategies employed by coordinated accounts across various campaigns and countries. By comparing these coordinated accounts against organic ones, researchers can develop and benchmark IO detection algorithms.

cs.CY↗

Coordinated Reply Attacks in Influence Operations: Characterization and Detection

Coordinated reply attacks are a tactic observed in online influence operations and other coordinated campaigns to support or harass targeted individuals, or influence them or their followers. Despite its potential to influence the public, past studies have yet to analyze or provide a methodology to detect this tactic. In this study, we characterize coordinated reply attacks in the context of influence operations on Twitter. Our analysis reveals that the primary targets of these attacks are influential people such as journalists, news media, state officials, and politicians. We propose two supervised machine-learning models, one to classify tweets to determine whether they are targeted by a reply attack, and one to classify accounts that reply to a targeted tweet to determine whether they are part of a coordinated attack. The classifiers achieve AUC scores of 0.88 and 0.97, respectively. These results indicate that accounts involved in reply attacks can be detected, and the targeted accounts themselves can serve as sensors for influence operation detection.

cs.LG↗

Social media algorithms can curb misinformation, but do they?

A recent article in $\textit{Science}$ by Guess et al. estimated the effect of Facebook's news feed algorithm on exposure to misinformation and political information among Facebook users. However, its reporting and conclusions did not account for a series of temporary emergency changes to Facebook's news feed algorithm in the wake of the 2020 U.S. presidential election that were designed to diminish the spread of voter-fraud misinformation. Here, we demonstrate that these emergency measures systematically reduced the amount of misinformation in the control group of the study, which was using the news feed algorithm. This issue may have led readers to misinterpret the results of the study and to conclude that the Facebook news feed algorithm used outside of the study period mitigates political misinformation as compared to reverse chronological feed.

cs.SI↗

Fact-checking information from large language models can decrease headline discernment

Fact checking can be an effective strategy against misinformation, but its implementation at scale is impeded by the overwhelming volume of information online. Recent artificial intelligence (AI) language models have shown impressive ability in fact-checking tasks, but how humans interact with fact-checking information provided by these models is unclear. Here, we investigate the impact of fact-checking information generated by a popular large language model (LLM) on belief in, and sharing intent of, political news headlines in a preregistered randomized control experiment. Although the LLM accurately identifies most false headlines (90%), we find that this information does not significantly improve participants' ability to discern headline accuracy or share accurate news. In contrast, viewing human-generated fact checks enhances discernment in both cases. Subsequent analysis reveals that the AI fact-checker is harmful in specific cases: it decreases beliefs in true headlines that it mislabels as false and increases beliefs in false headlines that it is unsure about. On the positive side, AI fact-checking information increases the sharing intent for correctly labeled true headlines. When participants are given the option to view LLM fact checks and choose to do so, they are significantly more likely to share both true and false news but only more likely to believe false headlines. Our findings highlight an important source of potential harm stemming from AI applications and underscore the critical need for policies to prevent or mitigate such unintended consequences.

cs.HC↗

The Dawn of Decentralized Social Media: An Exploration of Bluesky's Public Opening

Bluesky is a Twitter-like decentralized social media platform that has recently grown in popularity. After an invite-only period, it opened to the public worldwide on February 6th, 2024. In this paper, we provide a longitudinal analysis of user activity in the two months around the opening, studying changes in the general characteristics of the platform due to the rapid growth of the user base. We observe a broad distribution of activity similar to more established platforms, but a higher volume of original than reshared content, and very low toxicity. After opening to the public, Bluesky experienced a large surge in new users and activity, especially posting English and Japanese content. In particular, several accounts entered the discussion with suspicious behavior, like following many accounts and sharing content from low-credibility news outlets. Some of these have already been classified as spam or suspended, suggesting effective moderation.

cs.SI↗

Characteristics and prevalence of fake social media profiles with AI-generated faces

Recent advancements in generative artificial intelligence (AI) have raised concerns about their potential to create convincing fake social media accounts, but empirical evidence is lacking. In this paper, we present a systematic analysis of Twitter (X) accounts using human faces generated by Generative Adversarial Networks (GANs) for their profile pictures. We present a dataset of 1,420 such accounts and show that they are used to spread scams, spam, and amplify coordinated messages, among other inauthentic activities. Leveraging a feature of GAN-generated faces -- consistent eye placement -- and supplementing it with human annotation, we devise an effective method for identifying GAN-generated profiles in the wild. Applying this method to a random sample of active Twitter users, we estimate a lower bound for the prevalence of profiles using GAN-generated faces between 0.021% and 0.044% -- around 10K daily active accounts. These findings underscore the emerging threats posed by multimodal generative AI. We release the source code of our detection method and the data we collect to facilitate further investigation. Additionally, we provide practical heuristics to assist social media users in recognizing such accounts.

cs.CY↗

Quantifying the Vulnerabilities of the Online Public Square to Adversarial Manipulation Tactics

Social media, seen by some as the modern public square, is vulnerable to manipulation. By controlling inauthentic accounts impersonating humans, malicious actors can amplify disinformation within target communities. The consequences of such operations are difficult to evaluate due to the challenges posed by collecting data and carrying out ethical experiments that would influence online communities. Here we use a social media model that simulates information diffusion in an empirical network to quantify the impacts of several adversarial manipulation tactics on the quality of content. We find that the presence of influential accounts, a hallmark of social media, exacerbates the vulnerabilities of online communities to manipulation. Among the explored tactics that bad actors can employ, infiltrating a community is the most likely to make low-quality content go viral. Such harm can be further compounded by inauthentic agents flooding the network with low-quality, yet appealing content, but is mitigated when bad actors focus on specific targets, such as influential or vulnerable individuals. These insights suggest countermeasures that platforms could employ to increase the resilience of social media users to manipulation.

cs.CY↗

Toxic Synergy Between Hate Speech and Fake News Exposure

Hate speech on social media is a pressing concern. Understanding the factors associated with hate speech may help mitigate it. Here we explore the association between hate speech and exposure to fake news by studying the correlation between exposure to news from low-credibility sources through following connections and the use of hate speech on Twitter. Using news source credibility labels and a dataset of posts with hate speech targeting various populations, we find that hate speakers are exposed to lower percentages of posts linking to credible news sources. When taking the target population into account, we find that this association is mainly driven by anti-semitic and anti-Muslim content. We also observe that hate speakers are more likely to be exposed to low-credibility news with low popularity. Finally, while hate speech is associated with low-credibility news from partisan sources, we find that those sources tend to skew to the political left for antisemitic content and to the political right for hate speech targeting Muslim and Latino populations. Our results suggest that mitigating fake news and hate speech may have synergistic effects.

cs.CY↗

Rematch: Robust and Efficient Matching of Local Knowledge Graphs to Improve Structural and Semantic Similarity

Knowledge graphs play a pivotal role in various applications, such as question-answering and fact-checking. Abstract Meaning Representation (AMR) represents text as knowledge graphs. Evaluating the quality of these graphs involves matching them structurally to each other and semantically to the source text. Existing AMR metrics are inefficient and struggle to capture semantic similarity. We also lack a systematic evaluation benchmark for assessing structural similarity between AMR graphs. To overcome these limitations, we introduce a novel AMR similarity metric, rematch, alongside a new evaluation for structural similarity called RARE. Among state-of-the-art metrics, rematch ranks second in structural similarity; and first in semantic similarity by 1--5 percentage points on the STS-B and SICK-R benchmarks. Rematch is also five times faster than the next most efficient metric.

cs.CL↗