arXiv subjects

Filippo Menczer

Publications and source records attributed to Filippo Menczer.

At least 37 records · Page 2Linked to original sources

arXiv2025

A longitudinal analysis of misinformation, polarization and toxicity on Bluesky after its public launch

Bluesky is a decentralized, Twitter-like social media platform that has rapidly gained popularity. Following an invite-only phase, it officially opened to the public on February 6th, 2024, leading to a significant expansion of its user base. In this paper, we present a longitudinal analysis of user activity in the two months surrounding its public launch, examining how the platform evolved due to this rapid growth. Our analysis reveals that Bluesky exhibits an activity distribution comparable to more established social platforms, yet it features a higher volume of original content relative to reshared posts and maintains low toxicity levels. We further investigate the political leanings of its user base, misinformation dynamics, and engagement in harmful conversations. Our findings indicate that Bluesky users predominantly lean left politically and tend to share high-credibility sources. After the platform's public launch, an influx of new users, particularly those posting in English and Japanese, contributed to a surge in activity. Among them, several accounts displayed suspicious behaviors, such as mass-following users and sharing content from low-credibility news sources. Some of these accounts have already been flagged as spam or suspended, suggesting that Bluesky's moderation efforts have been effective.

cs.SI↗

arXiv2025

The Rise of Bluesky

This study investigates the rapid growth and evolving network structure of Bluesky from August 2023 to February 2025. Through multiple waves of user migrations, the platform has reached a stable, persistently active user base. The growth process has given rise to a dense follower network with clustering and hub features that favor viral information diffusion. These developments highlight engagement and structural similarities between Bluesky and established platforms.

cs.SI↗

arXiv2025

Beyond Interaction Patterns: Assessing Claims of Coordinated Inter-State Information Operations on Twitter/X

Social media platforms have become key tools for coordinated influence operations, enabling state actors to manipulate public opinion through strategic, collective actions. While previous research has suggested collaboration between states, such research failed to leverage state-of-the-art coordination indicators or control datasets. In this study, we investigate inter-state coordination by analyzing multiple online behavioral traces and using sophisticated coordination detection models. By incorporating a control dataset to differentiate organic user activity from coordinated efforts, our findings reveal no evidence of inter-state coordination. These results challenge earlier claims and underscore the importance of robust methodologies and control datasets in accurately detecting online coordination.

cs.SI↗

arXiv2025

Accuracy and Political Bias of News Source Credibility Ratings by Large Language Models

Search engines increasingly leverage large language models (LLMs) to generate direct answers, and AI chatbots now access the Internet for fresh data. As information curators for billions of users, LLMs must assess the accuracy and reliability of different sources. This paper audits nine widely used LLMs from three leading providers -- OpenAI, Google, and Meta -- to evaluate their ability to discern credible and high-quality information sources from low-credibility ones. We find that while LLMs can rate most tested news outlets, larger models more frequently refuse to provide ratings due to insufficient information, whereas smaller models are more prone to making errors in their ratings. For sources where ratings are provided, LLMs exhibit a high level of agreement among themselves (average Spearman's $ρ= 0.79$), but their ratings align only moderately with human expert evaluations (average $ρ= 0.50$). Analyzing news sources with different political leanings in the US, we observe a liberal bias in credibility ratings yielded by all LLMs in default configurations. Additionally, assigning partisan roles to LLMs consistently induces strong politically congruent bias in their ratings. These findings have important implications for the use of LLMs in curating news and political information.

cs.CL↗

arXiv2025

Modeling the amplification of epidemic spread by individuals exposed to misinformation on social media

Understanding how misinformation affects the spread of disease is crucial for public health, especially given recent research indicating that misinformation can increase vaccine hesitancy and discourage vaccine uptake. However, it is difficult to investigate the interaction between misinformation and epidemic outcomes due to the dearth of data-informed holistic epidemic models. Here, we employ an epidemic model that incorporates a large, mobility-informed physical contact network as well as the distribution of misinformed individuals across counties derived from social media data. The model allows us to simulate various scenarios to understand how epidemic spreading can be affected by misinformation spreading through one particular social media platform. Using this model, we compare a worst-case scenario, in which individuals become misinformed after a single exposure to low-credibility content, to a best-case scenario where the population is highly resilient to misinformation. We estimate the additional portion of the U.S. population that would become infected over the course of the COVID-19 epidemic in the worst-case scenario. This work can provide policymakers with insights about the potential harms of exposure to online vaccine misinformation.

cs.SI↗

arXiv2024

Labeled Datasets for Research on Information Operations

Social media platforms have become a hub for political activities and discussions, democratizing participation in these endeavors. However, they have also become an incubator for manipulation campaigns, like information operations (IOs). Some social media platforms have released datasets related to such IOs originating from different countries. However, we lack comprehensive control data that can enable the development of IO detection methods. To bridge this gap, we present new labeled datasets about 26 campaigns, which contain both IO posts verified by a social media platform and over 13M posts by 303k accounts that discussed similar topics in the same time frames (control data). The datasets will facilitate the study of narratives, network interactions, and engagement strategies employed by coordinated accounts across various campaigns and countries. By comparing these coordinated accounts against organic ones, researchers can develop and benchmark IO detection algorithms.

cs.CY↗

arXiv2024

Coordinated Reply Attacks in Influence Operations: Characterization and Detection

Coordinated reply attacks are a tactic observed in online influence operations and other coordinated campaigns to support or harass targeted individuals, or influence them or their followers. Despite its potential to influence the public, past studies have yet to analyze or provide a methodology to detect this tactic. In this study, we characterize coordinated reply attacks in the context of influence operations on Twitter. Our analysis reveals that the primary targets of these attacks are influential people such as journalists, news media, state officials, and politicians. We propose two supervised machine-learning models, one to classify tweets to determine whether they are targeted by a reply attack, and one to classify accounts that reply to a targeted tweet to determine whether they are part of a coordinated attack. The classifiers achieve AUC scores of 0.88 and 0.97, respectively. These results indicate that accounts involved in reply attacks can be detected, and the targeted accounts themselves can serve as sensors for influence operation detection.

cs.LG↗

arXiv2024

Social media algorithms can curb misinformation, but do they?

A recent article in $\textit{Science}$ by Guess et al. estimated the effect of Facebook's news feed algorithm on exposure to misinformation and political information among Facebook users. However, its reporting and conclusions did not account for a series of temporary emergency changes to Facebook's news feed algorithm in the wake of the 2020 U.S. presidential election that were designed to diminish the spread of voter-fraud misinformation. Here, we demonstrate that these emergency measures systematically reduced the amount of misinformation in the control group of the study, which was using the news feed algorithm. This issue may have led readers to misinterpret the results of the study and to conclude that the Facebook news feed algorithm used outside of the study period mitigates political misinformation as compared to reverse chronological feed.

cs.SI↗

arXiv2024

Fact-checking information from large language models can decrease headline discernment

Fact checking can be an effective strategy against misinformation, but its implementation at scale is impeded by the overwhelming volume of information online. Recent artificial intelligence (AI) language models have shown impressive ability in fact-checking tasks, but how humans interact with fact-checking information provided by these models is unclear. Here, we investigate the impact of fact-checking information generated by a popular large language model (LLM) on belief in, and sharing intent of, political news headlines in a preregistered randomized control experiment. Although the LLM accurately identifies most false headlines (90%), we find that this information does not significantly improve participants' ability to discern headline accuracy or share accurate news. In contrast, viewing human-generated fact checks enhances discernment in both cases. Subsequent analysis reveals that the AI fact-checker is harmful in specific cases: it decreases beliefs in true headlines that it mislabels as false and increases beliefs in false headlines that it is unsure about. On the positive side, AI fact-checking information increases the sharing intent for correctly labeled true headlines. When participants are given the option to view LLM fact checks and choose to do so, they are significantly more likely to share both true and false news but only more likely to believe false headlines. Our findings highlight an important source of potential harm stemming from AI applications and underscore the critical need for policies to prevent or mitigate such unintended consequences.

cs.HC↗

arXiv2024

The Dawn of Decentralized Social Media: An Exploration of Bluesky's Public Opening

Bluesky is a Twitter-like decentralized social media platform that has recently grown in popularity. After an invite-only period, it opened to the public worldwide on February 6th, 2024. In this paper, we provide a longitudinal analysis of user activity in the two months around the opening, studying changes in the general characteristics of the platform due to the rapid growth of the user base. We observe a broad distribution of activity similar to more established platforms, but a higher volume of original than reshared content, and very low toxicity. After opening to the public, Bluesky experienced a large surge in new users and activity, especially posting English and Japanese content. In particular, several accounts entered the discussion with suspicious behavior, like following many accounts and sharing content from low-credibility news outlets. Some of these have already been classified as spam or suspended, suggesting effective moderation.

cs.SI↗

arXiv2024

Characteristics and prevalence of fake social media profiles with AI-generated faces

Recent advancements in generative artificial intelligence (AI) have raised concerns about their potential to create convincing fake social media accounts, but empirical evidence is lacking. In this paper, we present a systematic analysis of Twitter (X) accounts using human faces generated by Generative Adversarial Networks (GANs) for their profile pictures. We present a dataset of 1,420 such accounts and show that they are used to spread scams, spam, and amplify coordinated messages, among other inauthentic activities. Leveraging a feature of GAN-generated faces -- consistent eye placement -- and supplementing it with human annotation, we devise an effective method for identifying GAN-generated profiles in the wild. Applying this method to a random sample of active Twitter users, we estimate a lower bound for the prevalence of profiles using GAN-generated faces between 0.021% and 0.044% -- around 10K daily active accounts. These findings underscore the emerging threats posed by multimodal generative AI. We release the source code of our detection method and the data we collect to facilitate further investigation. Additionally, we provide practical heuristics to assist social media users in recognizing such accounts.

cs.CY↗

arXiv2024

Quantifying the Vulnerabilities of the Online Public Square to Adversarial Manipulation Tactics

Social media, seen by some as the modern public square, is vulnerable to manipulation. By controlling inauthentic accounts impersonating humans, malicious actors can amplify disinformation within target communities. The consequences of such operations are difficult to evaluate due to the challenges posed by collecting data and carrying out ethical experiments that would influence online communities. Here we use a social media model that simulates information diffusion in an empirical network to quantify the impacts of several adversarial manipulation tactics on the quality of content. We find that the presence of influential accounts, a hallmark of social media, exacerbates the vulnerabilities of online communities to manipulation. Among the explored tactics that bad actors can employ, infiltrating a community is the most likely to make low-quality content go viral. Such harm can be further compounded by inauthentic agents flooding the network with low-quality, yet appealing content, but is mitigated when bad actors focus on specific targets, such as influential or vulnerable individuals. These insights suggest countermeasures that platforms could employ to increase the resilience of social media users to manipulation.

cs.CY↗

arXiv2024

Toxic Synergy Between Hate Speech and Fake News Exposure

Hate speech on social media is a pressing concern. Understanding the factors associated with hate speech may help mitigate it. Here we explore the association between hate speech and exposure to fake news by studying the correlation between exposure to news from low-credibility sources through following connections and the use of hate speech on Twitter. Using news source credibility labels and a dataset of posts with hate speech targeting various populations, we find that hate speakers are exposed to lower percentages of posts linking to credible news sources. When taking the target population into account, we find that this association is mainly driven by anti-semitic and anti-Muslim content. We also observe that hate speakers are more likely to be exposed to low-credibility news with low popularity. Finally, while hate speech is associated with low-credibility news from partisan sources, we find that those sources tend to skew to the political left for antisemitic content and to the political right for hate speech targeting Muslim and Latino populations. Our results suggest that mitigating fake news and hate speech may have synergistic effects.

cs.CY↗

arXiv2024

Rematch: Robust and Efficient Matching of Local Knowledge Graphs to Improve Structural and Semantic Similarity

Knowledge graphs play a pivotal role in various applications, such as question-answering and fact-checking. Abstract Meaning Representation (AMR) represents text as knowledge graphs. Evaluating the quality of these graphs involves matching them structurally to each other and semantically to the source text. Existing AMR metrics are inefficient and struggle to capture semantic similarity. We also lack a systematic evaluation benchmark for assessing structural similarity between AMR graphs. To overcome these limitations, we introduce a novel AMR similarity metric, rematch, alongside a new evaluation for structural similarity called RARE. Among state-of-the-art metrics, rematch ranks second in structural similarity; and first in semantic similarity by 1--5 percentage points on the STS-B and SICK-R benchmarks. Rematch is also five times faster than the next most efficient metric.

cs.CL↗

arXiv2024

Identifying and characterizing superspreaders of low-credibility content on Twitter

The world's digital information ecosystem continues to struggle with the spread of misinformation. Prior work has suggested that users who consistently disseminate a disproportionate amount of low-credibility content -- so-called superspreaders -- are at the center of this problem. We quantitatively confirm this hypothesis and introduce simple metrics to predict the top superspreaders several months into the future. We then conduct a qualitative review to characterize the most prolific superspreaders and analyze their sharing behaviors. Superspreaders include pundits with large followings, low-credibility media outlets, personal accounts affiliated with those media outlets, and a range of influencers. They are primarily political in nature and use more toxic language than the typical user sharing misinformation. We also find concerning evidence that suggests Twitter may be overlooking prominent superspreaders. We hope this work will further public understanding of bad actors and promote steps to mitigate their negative impacts on healthy digital discourse.

cs.SI↗

arXiv2024

Account credibility inference based on news-sharing networks

The spread of misinformation poses a threat to the social media ecosystem. Effective countermeasures to mitigate this threat require that social media platforms be able to accurately detect low-credibility accounts even before the content they share can be classified as misinformation. Here we present methods to infer account credibility from information diffusion patterns, in particular leveraging two networks: the reshare network, capturing an account's trust in other accounts, and the bipartite account-source network, capturing an account's trust in media sources. We extend network centrality measures and graph embedding techniques, systematically comparing these algorithms on data from diverse contexts and social media platforms. We demonstrate that both kinds of trust networks provide useful signals for estimating account credibility. Some of the proposed methods yield high accuracy, providing promising solutions to promote the dissemination of reliable information in online communities. Two kinds of homophily emerge from our results: accounts tend to have similar credibility if they reshare each other's content or share content from similar sources. Our methodology invites further investigation into the relationship between accounts and news sources to better characterize misinformation spreaders.

cs.SI↗

arXiv2023

Social Bots: Detection and Challenges

While social media are a key source of data for computational social science, their ease of manipulation by malicious actors threatens the integrity of online information exchanges and their analysis. In this Chapter, we focus on malicious social bots, a prominent vehicle for such manipulation. We start by discussing recent studies about the presence and actions of social bots in various online discussions to show their real-world implications and the need for detection methods. Then we discuss the challenges of bot detection methods and use Botometer, a publicly available bot detection tool, as a case study to describe recent developments in this area. We close with a practical guide on how to handle social bots in social media research.

cs.SI↗

arXiv2023

Factuality Challenges in the Era of Large Language Models

The emergence of tools based on Large Language Models (LLMs), such as OpenAI's ChatGPT, Microsoft's Bing Chat, and Google's Bard, has garnered immense public attention. These incredibly useful, natural-sounding tools mark significant advances in natural language generation, yet they exhibit a propensity to generate false, erroneous, or misleading content -- commonly referred to as "hallucinations." Moreover, LLMs can be exploited for malicious applications, such as generating false but credible-sounding content and profiles at scale. This poses a significant challenge to society in terms of the potential deception of users and the increasing dissemination of inaccurate information. In light of these risks, we explore the kinds of technological innovations, regulatory reforms, and AI literacy initiatives needed from fact-checkers, news organizations, and the broader research and policy communities. By identifying the risks, the imminent threats, and some viable solutions, we seek to shed light on navigating various aspects of veracity in the era of generative AI.

cs.CL↗