Ádám Fraknói | Searcharxiv

arXiv2025

Do Attention Heads Compete or Cooperate during Counting?

We present an in-depth mechanistic interpretability analysis of training small transformers on an elementary task, counting, which is a crucial deductive step in many algorithms. In particular, we investigate the collaboration/competition among the attention heads: we ask whether the attention heads behave as a pseudo-ensemble, all solving the same subtask, or they perform different subtasks, meaning that they can only solve the original task in conjunction. Our work presents evidence that on the semantics of the counting task, attention heads behave as a pseudo-ensemble, but their outputs need to be aggregated in a non-uniform manner in order to create an encoding that conforms to the syntax. Our source code will be available upon publication.

cs.LG↗

arXiv2023

On the Rényi-Ulam Game with Restricted Size Queries

We investigate the following version of the well-known Rényi-Ulam game. Two players - the Questioner and the Responder - play against each other. The Responder thinks of a number from the set $\{1,\ldots,n\}$, and the Questioner has to find this number. To do this, he can ask whether a chosen set of at most $k$ elements contains the thought number. The Responder answers with YES or NO immediately, but during the game, he may lie at most $\ell$ times. The minimum number of queries needed for the Questioner to surely find the unknown element is denoted by $RU_\ell^k(n)$. First, we develop a highly effective tool that we call Convexity Lemma. By using this lemma, we give a general lower bound of $RU_\ell^k(n)$ and an upper bound which differs from the lower one by at most $2\ell+1$. We also give its exact value when $n$ is sufficiently large compared to $k$. With these, we managed to improve and generalize the results obtained by Meng, Lin, and Yang in a 2013 paper about the case $\ell=1$.

math.CO↗