arXiv subjects

Kaarel Hänni

Publications and source records attributed to Kaarel Hänni.

8 recordsLinked to original sources

arXiv2026

Width-Robust Learnability in Mean-Field Bayesian Neural Networks

Infinite-width limits are a standard way to reason about neural networks, but it is not automatic that the limiting learner has the same complexity-theoretic inductive bias as large finite networks. We study this question for Bayesian neural networks at the mean-field, or critical feature-learning, scaling. The central quantity is the \emph{reduced entropy} \[ s_\infty(y,\varepsilon)=\limsup_N -\frac{1}{N}\log π_N^0(L\le \varepsilon), \] the intensive prior cost of representing a target function $y$ to population mean-squared error $\varepsilon$. Our main result is a width-robust learnability theorem. At fixed depth, a family of Boolean-cube targets is learnable from polynomially many samples at infinite width if and only if it is learnable at polynomial width, if and only if its reduced entropy is polynomially bounded. Equivalently, up to polynomial slack in accuracy, the Bayesian mean-field learner generalizes exactly on the targets that can be represented by polynomial-size networks. The forward direction is proved by a form of subsampling: from the infinitely many hidden neurons in the mean-field solution, one can select polynomially many representatives and still preserve the learned function on every input simultaneously. At the critical scaling this subsampling has both an ``active'' component, which keeps the data-dependent low-dimensional statistics, and a ``lazy'' component, which resamples the entropy-dominated directions from the prior. Thus the infinite-width mean-field limit gives a clean analytic description of learning without introducing spurious width-dependent generalization power.

stat.ML↗

arXiv2024

Cluster-norm for Unsupervised Probing of Knowledge

The deployment of language models brings challenges in generating reliable information, especially when these models are fine-tuned using human preferences. To extract encoded knowledge without (potentially) biased human labels, unsupervised probing techniques like Contrast-Consistent Search (CCS) have been developed (Burns et al., 2022). However, salient but unrelated features in a given dataset can mislead these probes (Farquhar et al., 2023). Addressing this, we propose a cluster normalization method to minimize the impact of such features by clustering and normalizing activations of contrast pairs before applying unsupervised probing techniques. While this approach does not address the issue of differentiating between knowledge in general and simulated knowledge - a major issue in the literature of latent knowledge elicitation (Christiano et al., 2021) - it significantly improves the ability of unsupervised probes to identify the intended knowledge amidst distractions.

cs.AI↗

arXiv2024

Mathematical Models of Computation in Superposition

Superposition -- when a neural network represents more ``features'' than it has dimensions -- seems to pose a serious challenge to mechanistically interpreting current AI systems. Existing theory work studies \emph{representational} superposition, where superposition is only used when passing information through bottlenecks. In this work, we present mathematical models of \emph{computation} in superposition, where superposition is actively helpful for efficiently accomplishing the task. We first construct a task of efficiently emulating a circuit that takes the AND of the $\binom{m}{2}$ pairs of each of $m$ features. We construct a 1-layer MLP that uses superposition to perform this task up to $\varepsilon$-error, where the network only requires $\tilde{O}(m^{\frac{2}{3}})$ neurons, even when the input features are \emph{themselves in superposition}. We generalize this construction to arbitrary sparse boolean circuits of low depth, and then construct ``error correction'' layers that allow deep fully-connected networks of width $d$ to emulate circuits of width $\tilde{O}(d^{1.5})$ and \emph{any} polynomial depth. We conclude by providing some potential applications of our work for interpreting neural networks that implement computation in superposition.

cs.LG↗

arXiv2024

Using Degeneracy in the Loss Landscape for Mechanistic Interpretability

Mechanistic Interpretability aims to reverse engineer the algorithms implemented by neural networks by studying their weights and activations. An obstacle to reverse engineering neural networks is that many of the parameters inside a network are not involved in the computation being implemented by the network. These degenerate parameters may obfuscate internal structure. Singular learning theory teaches us that neural network parameterizations are biased towards being more degenerate, and parameterizations with more degeneracy are likely to generalize further. We identify 3 ways that network parameters can be degenerate: linear dependence between activations in a layer; linear dependence between gradients passed back to a layer; ReLUs which fire on the same subset of datapoints. We also present a heuristic argument that modular networks are likely to be more degenerate, and we develop a metric for identifying modules in a network that is based on this argument. We propose that if we can represent a neural network in a way that is invariant to reparameterizations that exploit the degeneracies, then this representation is likely to be more interpretable, and we provide some evidence that such a representation is likely to have sparser interactions. We introduce the Interaction Basis, a tractable technique to obtain a representation that is invariant to degeneracies from linear dependence of activations or Jacobians.

cs.LG↗

arXiv2024

The Local Interaction Basis: Identifying Computationally-Relevant and Sparsely Interacting Features in Neural Networks

Mechanistic interpretability aims to understand the behavior of neural networks by reverse-engineering their internal computations. However, current methods struggle to find clear interpretations of neural network activations because a decomposition of activations into computational features is missing. Individual neurons or model components do not cleanly correspond to distinct features or functions. We present a novel interpretability method that aims to overcome this limitation by transforming the activations of the network into a new basis - the Local Interaction Basis (LIB). LIB aims to identify computational features by removing irrelevant activations and interactions. Our method drops irrelevant activation directions and aligns the basis with the singular vectors of the Jacobian matrix between adjacent layers. It also scales features based on their importance for downstream computation, producing an interaction graph that shows all computationally-relevant features and interactions in a model. We evaluate the effectiveness of LIB on modular addition and CIFAR-10 models, finding that it identifies more computationally-relevant features that interact more sparsely, compared to principal component analysis. However, LIB does not yield substantial improvements in interpretability or interaction sparsity when applied to language models. We conclude that LIB is a promising theory-driven approach for analyzing neural networks, but in its current form is not applicable to large language models.

cs.LG↗

arXiv2020

Asymptotics of descent functions

In 1916, MacMahon showed that permutations in $S_n$ with a fixed descent set $I$ are enumerated by a polynomial $d_I(n)$. Diaz-Lopez, Harris, Insko, Omar, and Sagan recently revived interest in this descent polynomial, and suggested the direction of studying such enumerative questions for other consecutive patterns (descents being the consecutive pattern $21$). Zhu studied this question for the consecutive pattern $321$. We continue this line of work by studying the case of any consecutive pattern of the form $k,k-1,\ldots,1$, which we call a $k$-descent. In this paper, we reduce the problem of determining the asymptotic number of permutations with a certain $k$-descent set to computing an explicit integral. We also prove an equidistribution theorem, showing that any two sparse $k$-descent sets are equally likely. Counting the number of $k$-descent-avoiding permutations while conditioning on the length $n$ and first element $m$ simultaneously, one obtains a number triangle $f_k(m,n)$ with some useful properties. For $k=3$, the $m=1$ and $m=n$ diagonals are OEIS sequences A049774 and A080635. We prove a $k$th difference recurrence relation for entries of this number triangle. This also leads to an $O(n^2)$ algorithm for computing $k$-descent functions. Along the way to these results, we prove an explicit formula for the distribution of first elements of $k$-descent-avoiding permutations, as well as for the joint distribution of first and last elements. We also develop an understanding of discrete order statistics. In our approach, we combine algebraic, analytic, and probabilistic tools. A number of open problems are stated at the end.

math.CO↗

arXiv2020

Counting Signed Vexillary Permutations

We show that the number of signed permutations avoiding 1234 equals the number of signed permutations avoiding 2143 (also called vexillary signed permutations), resolving a conjecture by Anderson and Fulton. The main tool that we use is the generating tree developed by West. Many further directions are mentioned in the end.

math.CO↗

arXiv2020

Boolean elements in the Bruhat order

We show that $w\in W$ is boolean if and only if it avoids a set of Billey-Postnikov patterns, which we describe explicitly. Our proof is based on an analysis of inversion sets, and it is in large part type-uniform. We also introduce the notion of linear pattern avoidance, and show that boolean elements are characterized by avoiding just the $3$ linear patterns $s_1 s_2 s_1 \in W(A_2)$, $s_2 s_1 s_3 s_2 \in W(A_3)$, and $s_2 s_1 s_3 s_4 s_2 \in W(D_4)$. We also consider the more general case of $k$-boolean Weyl group elements. We say that $w\in W$ is $k$-boolean if every reduced expression for $w$ contains at most $k$ copies of each generator. We show that the $2$-boolean elements of the symmetric group $S_n$ are characterized by avoiding the patterns $3421,4312,4321,$ and $456123$, and give a rational generating function for the number of $2$-boolean elements of $S_n$.

math.CO↗