Searcharxiv⌕ Search

arXiv subjects

Bastiaan Bruinsma

Publications and source records attributed to Bastiaan Bruinsma.

4 recordsLinked to original sources

Issue Bias in Generative AI Writing Assistance: Political Issues and LLMs in the Swedish 2026 Election

Generative AI writing assistants and the Large Language Models (LLMs) that power them are increasingly part of how voters gather information before elections. With growing evidence that they influence users' opinions, it is increasingly important to understand the views and positions of these tools. To better understand these views, we examine the stances supplied by six LLMs on a variety of Swedish-language writing tasks ahead of the 2026 Swedish parliamentary election. We cross 107 policy propositions with 77 writing templates and neutral, positive, and negative prompt framings, producing 24,717 prompts per model and 148,302 responses. To study these, we look at the models' default stance tendencies, compare how they respond to similar issues, and compare their responses with those of each of Sweden's eight parliamentary parties on the same issue. We find that Claude, DeepSeek, Gemini, and Mistral have similar profiles; ChatGPT more often supplies neutral or ambivalent text; and Grok differs most on topics such as migration, crime, and gender. When comparing the political parties, we find that the Social Democrats are closest to all six models. Still, after correcting for multiple comparisons, none of the within-model differences in party distances remains significant. Overall, we find that no model has a clear preference, nor a clear preference for a party, but that this depends on the specific issue or task the user asks about.

cs.AI↗

Identifying Non-Replicable Social Science Studies with Language Models

In this study, we investigate whether LLMs can be used to indicate if a study in the behavioural social sciences is replicable. Using a dataset of 14 previously replicated studies (9 successful, 5 unsuccessful), we evaluate the ability of both open-source (Llama 3 8B, Qwen 2 7B, Mistral 7B) and proprietary (GPT-4o) instruction-tuned LLMs to discriminate between replicable and non-replicable findings. We use LLMs to generate synthetic samples of responses from behavioural studies and estimate whether the measured effects support the original findings. When compared with human replication results for these studies, we achieve F1 values of up to $77\%$ with Mistral 7B, $67\%$ with GPT-4o and Llama 3 8B, and $55\%$ with Qwen 2 7B, suggesting their potential for this task. We also analyse how effect size calculations are affected by sampling temperature and find that low variance (due to temperature) leads to biased effect estimates.

cs.CL↗

Setting the AI Agenda -- Evidence from Sweden in the ChatGPT Era

This paper examines the development of the Artificial Intelligence (AI) meta-debate in Sweden before and after the release of ChatGPT. From the perspective of agenda-setting theory, we propose that it is an elite outside of party politics that is leading the debate -- i.e. that the politicians are relatively silent when it comes to this rapid development. We also suggest that the debate has become more substantive and risk-oriented in recent years. To investigate this claim, we draw on an original dataset of elite-level documents from the early 2010s to the present, using op-eds published in a number of leading Swedish newspapers. By conducting a qualitative content analysis of these materials, our preliminary findings lend support to the expectation that an academic, rather than a political elite is steering the debate.

cs.AI↗

Validating Wordscores

Wordscores is a popular quantitative text scaling method to estimate parties' positions on a priori specified dimensions, without requiring the researchers to read or even understand the language in the documents they are analysing. This study tries to establish whereas Wordscores is able to deliver this promise by conducting a rigorous validation of its output using the Euromanifestos of 164 parties across 23 countries. We assess content validity by looking at the scored words in their context, criterion validity by comparing the Wordscores output to expert surveys and other judgemental estimates of party positions, and construct validity by using the Wordscores estimates to predict party membership in the European Parliament groups. We conclude that, despite the promises, Wordscores fails to deliver valid party positions, and outline three conditions under which its performance can be improved.

stat.AP↗