SearcharxivSearch

arXiv subjects

David M. Berry

Publications and source records attributed to David M. Berry.

7 recordsLinked to original sources

LLMbench: A Comparative Close Reading Workbench for Large Language Models

LLMbench is a browser-based workbench for the comparative close reading of large language model (LLM) outputs. Where existing tools for LLM comparison, such as Google PAIR's LLM Comparator are engineered for quantitative evaluation and user-rating metrics, LLMbench is oriented towards the hermeneutic practices of the digital humanities. Two model responses to the same prompt are side by side in annotatable panels with four analytical overlays (Probabilities for token-level log-probability inspection, Differences for word-level diff across the two panels, Tone for Hyland-style metadiscourse analysis, and Structure for sentence-level parsing with discourse connective highlighting), alongside five analytical modes, Stochastic Variation, Temperature Gradient, Prompt Sensitivity, Token Probabilities, and Cross-Model Divergence, that make the probabilistic structure of generated text legible at the token level. The tool treats the generated text as a research object in its own right from a probability distribution, a text that could have been otherwise, and provides visualisations including continuous heatmaps, entropy sparklines, pixel maps, and three-dimensional probability terrains, that show the counterfactual history from which each word emerged. This paper describes the tool's architecture, its six modes, and its design rationale, and argues that log-probability data, currently underused in humanistic and social-scientific readings of AI, is an important resource for a critical studies of generative AI models.

cs.CY

AI Sprints: Towards a Critical Method for Human-AI Collaboration

The emergence of Large Language Models presents a remarkable opportunity for humanities and social science research. I argue these technologies instantiate what I have called the algorithmic condition, whereby computational systems increasingly mediate not just our analytical tools but how we understand nature and society more generally. This article introduces the possibility for new forms of humanistic inquiry through what I term 'AI sprints', as intensive time-boxed research sessions. This is a research method combining the critical reflexivity essential to humanistic inquiry with iterative dialogue with generative AI. Drawing on experimental work in critical code studies, I demonstrate how tight loops of iterative development can adapt data and book sprint methodologies whilst acknowledging the profound transformations generative AI introduces. Through examining the process of human-AI collaboration when undertaken in these intensive research sessions, I seek to outline this approach as a broader research method. The article builds on Rogers' digital methods approach, proposing that we extend methodologies to study digital objects through their native protocols, using AI systems not merely to process digital traces but to analyse materials traditionally requiring manual coding or transcription. I aim to show this by introducing three cognitive modes, cognitive delegation, productive augmentation, and cognitive overhead, explaining how researchers can maintain a strategic overview whilst using LLM capabilities. The paper contributes both a practical methodology for intensive AI-augmented research and a theoretical framework for understanding the epistemological transformations of this hybrid method. A critical methodology must therefore operate in both technical and theoretical registers, sustaining a rigorous ethical-computational engagement with AI systems and outputs.

cs.CY

The heteronomy of algorithms: Traditional knowledge and computational knowledge

If an active citizen should increasingly be a computationally enlightened one, replacing the autonomy of reason with the heteronomy of algorithms, then I argue in this article that we must begin teaching the principles of critiquing the computal through new notions of what we might call digital Bildung. Indeed, if civil society itself is mediated by computational systems and media, the public use of reason must also be complemented by skills for negotiating and using these computal forms to articulate such critique. Not only is there a need to raise the intellectual tone regarding computation and its related softwarization processes, but there is an urgent need to attend to the likely epistemic challenges from computation which, as presently constituted, tends towards justification through a philosophy of utility rather than through a philosophy of care for the territory of the intellect. We therefore need to develop an approach to this field that uses concepts and methods drawn from philosophy, politics, history, anthropology, sociology, media studies, computer science, and the humanities more generally, to try to understand these issues - particularly the way in which software and data increasingly penetrate our everyday life and the pressures and fissures that are created. We must, in other words, move to undertake a critical interdisciplinary research program to understand the way in which these systems are created, instantiated, and normatively engendered in both specific and general contexts.

cs.CY

Synthetic media and computational capitalism: towards a critical theory of artificial intelligence

This paper develops a critical theory of artificial intelligence, within a historical constellation where computational systems increasingly generate cultural content that destabilises traditional distinctions between human and machine production. Through this analysis, I introduce the concept of the algorithmic condition, a cultural moment when machine-generated work not only becomes indistinguishable from human creation but actively reshapes our understanding of ideas of authenticity. This transformation, I argue, moves beyond false consciousness towards what I call post-consciousness, where the boundaries between individual and synthetic consciousness become porous. Drawing on critical theory and extending recent work on computational ideology, I develop three key theoretical contributions, first, the concept of the Inversion to describe a new computational turn in algorithmic society; second, automimetric production as a framework for understanding emerging practices of automated value creation; and third, constellational analysis as a methodological approach for mapping the complex interplay of technical systems, cultural forms and political economic structures. Through these contributions, I argue that we need new critical methods capable of addressing both the technical specificity of AI systems and their role in restructuring forms of life under computational capitalism. The paper concludes by suggesting that critical reflexivity is needed to engage with the algorithmic condition without being subsumed by it and that it represents a growing challenge for contemporary critical theory.

cs.CY

ELIZA Reanimated: The world's first chatbot restored on the world's first time sharing system

ELIZA, created by Joseph Weizenbaum at MIT in the early 1960s, is usually considered the world's first chatbot. It was developed in MAD-SLIP on MIT's CTSS, the world's first time-sharing system, on an IBM 7094. We discovered an original ELIZA printout in Prof. Weizenbaum's archives at MIT, including an early version of the famous DOCTOR script, a nearly complete version of the MAD-SLIP code, and various support functions in MAD and FAP. Here we describe the reanimation of this original ELIZA on a restored CTSS, itself running on an emulated IBM 7094. The entire stack is open source, so that any user of a unix-like OS can run the world's first chatbot on the world's first time-sharing system.

cs.AI

Post-Digital Humanities: Computation and Cultural Critique in the Arts and Humanities

Today we live in computational abundance whereby our everyday lives and the environment that surrounds us are suffused with digital technologies. This is a world of anticipatory technology and contextual computing that uses smart diffused computational processing to create a fine web of computational resources that are embedded into the material world. Thus, the historical distinction between the digital and the non-digital becomes increasingly blurred, to the extent that to talk about the digital presupposes an experiential disjuncture that makes less and less sense. Indeed, just as the ideas of online or being online have become anachronistic as a result of our always-on smartphones and tablets and widespread wireless networking technologies, so too the term digital perhaps assumes a world of the past.

cs.DL

Explanatory Publics: Explainability and Democratic Thought

In order to legitimate and defend democratic politics under conditions of computational capital, my aim is to contribute a notion of what I am calling explanatory publics. I will explore what is at stake when we question the social and political effects of the disruptive technologies, networks and values that are hidden within the "black boxes" of computational systems. By "explanatory publics", I am gesturing to the need for frameworks of knowledge - whether social, political, technical, economic, or cultural - to be justified through a social right to explanation. That is, for a polity to be considered democratic, it must ensure that its citizens are able to develop a capacity for explanatory thought (in addition to other capacities), and, thereby, able to question ideas, practices, and institutions in society. This is to extend the notion of a public sphere where citizens are able to question ideas, practices, and institutions in society more generally. But it also adds the corollary that citizens can demand explanatory accounts from institutions and, crucially, the digital technologies that they use.

cs.CY