arXiv subjects

James Campbell

Publications and source records attributed to James Campbell.

4 recordsLinked to original sources

arXiv2023

Localizing Lying in Llama: Understanding Instructed Dishonesty on True-False Questions Through Prompting, Probing, and Patching

Large language models (LLMs) demonstrate significant knowledge through their outputs, though it is often unclear whether false outputs are due to a lack of knowledge or dishonesty. In this paper, we investigate instructed dishonesty, wherein we explicitly prompt LLaMA-2-70b-chat to lie. We perform prompt engineering to find which prompts best induce lying behavior, and then use mechanistic interpretability approaches to localize where in the network this behavior occurs. Using linear probing and activation patching, we localize five layers that appear especially important for lying. We then find just 46 attention heads within these layers that enable us to causally intervene such that the lying model instead answers honestly. We show that these interventions work robustly across many prompts and dataset splits. Overall, our work contributes a greater understanding of dishonesty in LLMs so that we may hope to prevent it.

cs.LG↗

arXiv2023

Representation Engineering: A Top-Down Approach to AI Transparency

In this paper, we identify and characterize the emerging area of representation engineering (RepE), an approach to enhancing the transparency of AI systems that draws on insights from cognitive neuroscience. RepE places population-level representations, rather than neurons or circuits, at the center of analysis, equipping us with novel methods for monitoring and manipulating high-level cognitive phenomena in deep neural networks (DNNs). We provide baselines and an initial analysis of RepE techniques, showing that they offer simple yet effective solutions for improving our understanding and control of large language models. We showcase how these methods can provide traction on a wide range of safety-relevant problems, including honesty, harmlessness, power-seeking, and more, demonstrating the promise of top-down transparency research. We hope that this work catalyzes further exploration of RepE and fosters advancements in the transparency and safety of AI systems.

cs.LG↗

arXiv2021

The Lightning Model

We introduce a non-standard model for percolation on the integer lattice $\mathbb Z^2$. Randomly assign to each vertex $a \in \mathbb Z^2$ a potential, denoted $ϕ_a$, chosen independently and uniformly from the interval $[0, 1]$. For fixed $ε\in [0,1]$, draw a directed edge from vertex $a$ to a nearest-neighbor vertex $b$ if $ϕ_b < ϕ_a + ε$, yielding a directed subgraph of the infinite directed graph $\overrightarrow{G}$ whose vertex set is $\mathbb Z^2$, with nearest-neighbor edge set. We define notions of weak and strong percolation for our model, and observe that when $ε= 0$ the model fails to percolate weakly, while for $ε= 1$ it percolates strongly. We show that there is a positive $ε_0$ so that for $0 \le ε\le ε_0$, the model fails to percolate weakly, and that when $ε> p_\text{site}$, the critical probability for standard site percolation in $\mathbb Z^2$, the model percolates strongly. We study the number of infinite strongly connected clusters occurring in a typical configuration. We show that for these models of percolation on directed graphs, there are some subtle issues that do not arise for undirected percolation. Although our model does not have the finite energy property, we are able to show that, as in the standard model, the number of infinite strongly connected clusters is almost surely 0, 1 or $\infty$.

math.PR↗

arXiv2016

An open reproducible framework for the study of the iterated prisoner's dilemma

The Axelrod library is an open source Python package that allows for reproducible game theoretic research into the Iterated Prisoner's Dilemma. This area of research began in the 1980s but suffers from a lack of documentation and test code. The goal of the library is to provide such a resource, with facilities for the design of new strategies and interactions between them, as well as conducting tournaments and ecological simulations for populations of strategies. With a growing collection of 139 strategies, the library is a also a platform for an original tournament that, in itself, is of interest to the game theoretic community. This paper describes the Iterated Prisoner's Dilemma, the Axelrod library and its development, and insights gained from some novel research.

cs.GT↗