arXiv subjects

Harry Collins

Publications and source records attributed to Harry Collins.

7 recordsLinked to original sources

arXiv2026

Large language models eroding science understanding: an experimental study

This paper is under review in AI and Ethics This study examines whether large language models (LLMs) can reliably answer scientific questions and demonstrates how easily they can be influenced by fringe scientific material. The authors modified custom LLMs to prioritise knowledge in selected fringe papers on the Fine Structure Constant and Gravitational Waves, then compared their responses with those of domain experts and standard LLMs. The altered models produced fluent, convincing answers that contradicted scientific consensus and were difficult for non-experts to detect as misleading. The results show that LLMs are vulnerable to manipulation and cannot replace expert judgment, highlighting risks for public understanding of science and the potential spread of misinformation.

cs.CY↗

arXiv2026

Large Language Models and Scientific Discourse: Where's the Intelligence?

We explore the capabilities of Large Language Models (LLMs) by comparing the way they gather data with the way humans build knowledge. Here we examine how scientific knowledge is made and compare it with LLMs. The argument is structured by reference to two figures, one representing scientific knowledge and the other LLMs. In a 2014 study, scientists explain how they choose to ignore a 'fringe science' paper in the domain of gravitational wave physics: the decisions are made largely as a result of tacit knowledge built up in social discourse, most spoken discourse, within closed groups of experts. It is argued that LLMs cannot or do not currently access such discourse, but it is typical of the early formation of scientific knowledge. LLMs 'understanding' builds on written literatures and is therefore insecure in the case of the initial stages of knowledge building. We refer to Colin Fraser's 'Dumb Monty Hall problem' where in 2023 ChatGPT failed though a year later or so later LLMs were succeeding. We argue that this is not a matter of improvement in LLMs ability to reason but in the change in the body of human written discourse on which they can draw (or changes being put in by humans 'by hand'). We then invent a new Monty Hall prompt and compare the responses of a panel of LLMs and a panel of humans: they are starkly different but we explain that the previous mechanisms will soon allow the LLMs to align themselves to humans once more. Finally, we look at 'overshadowing' where a settled body of discourse becomes so dominant that LLMs fail to respond to small variations in prompts which render the old answers nonsensical. The 'intelligence' we argue is in the humans not the LLMs

physics.soc-ph↗

arXiv2017

Trading Zones Revisited

We describe and then elaborate the model of trading zones first presented in Collins et al 2007, Trading Zones and Interactional Expertise. We believe this expanded version of the model includes some very important but previously overlooked ways for separate language communities to communicate. We illustrate the argument with examples.

physics.hist-ph↗

arXiv2016

Expertise revisited I: Interactional Expertise

The notion of interactional expertise is explained starting with its origins and discussing its many applications. Interactional expertise is the ability to understand a technical area purely be deeply immersed in its 'practice-language' without actually practising. One of its many applications is to explain how large sciences are managed.

physics.soc-ph↗

arXiv2016

Um, er: How meaning varies between speech and its typed transcript

We use an extract from an interview concerning gravitational wave physics to show that the meaning of hesitancies within speech are different when spoken and when read from the corresponding transcript. When used in speech, hesitancies can indicate a pause for thought, when read in a transcript they indicate uncertainty. In a series of experiments the perceived uncertainty of the transcript was shown to be higher than the perceived uncertainty of the spoken version with almost no overlap for any respondent. We propose that finding and the method could be the beginning of a new subject we call 'Language Code Analysis' which would systematically examine how meanings change when the 'same' words are communicated via different media and symbol systems.

physics.soc-ph↗

arXiv2016

Interactional Expertise and Embodiment

In Part 1 of this paper, I introduce the idea of interactional expertise while in Part 2, I focus on its implications for philosophical theories of the importance of the body in forming our conceptual world. I argue that the way philosophers have dealt with the body turns attention away from the most important questions and that we cannot answer these questions without making the notion of socialisation, and therefore interactional expertise, a central concept in our thinking. This makes language at least as important, and often more important than bodily practice in our understanding of the world. The notion of a disembodied socialised agent leads in the direction of interesting questions while the notion of an embodied but unsocialised human actor is unimaginable.

physics.hist-ph↗

arXiv2016

An Imitation Game concerning gravitational wave physics

The 'Imitation Game' is a Turing Test played with a human participant instead of a computer. Here the author, a sociologist, who has been immersed in the field of gravitational wave physics since 1972, tried to pass an Imitation Game as a gravitational wave physicist. He already passed such a test in mid-2000s but this test was more elaborate and compared his performance with that of other kinds of physicists and with other sociologists as well as gravitational wave physicists. The test was based on 8 technical questions about gravitational wave physics asked by Professor Sathyprakash of Cardiff University. Collins marks compared well with that of the other gravitational wave physicists and were markedly better than that of other classes of respondent. Collins also marked the test and it can be seen that the way he marked was also much closer to the gravitational wave physicists than other categories. Though Collins's expertise can be shown to have degraded a little in the last ten years it seems not to have degraded a lot. This is important for his most recent book on the detection of gravitational waves from a black hole binary, this being Chapter 14 of this book, which is due to by published my MIT Press in February 2017.

physics.soc-ph↗