Groups with irreducibly unfaithful subsets for unitary representations

Abstract

Let $G$ be a group. A subset $F \subset G$ is called irreducibly faithful if there exists an irreducible unitary representation $\pi$ of $G$ such that $\pi(x) \neq \mathrm{id}$ for all $x \in F \smallsetminus \{e\}$. Otherwise $F$ is called irreducibly unfaithful. Given a positive integer $n$, we say that $G$ has Property $P(n)$ if every subset of size $n$ is irreducibly faithful. Every group has $P(1)$, by a classical result of Gelfand and Raikov. Walter proved that every group has $P(2)$. It is easy to see that some groups do not have $P(3)$. We provide a complete description of the irreducibly unfaithful subsets of size $n$ in a countable group $G$ (finite or infinite) with Property $P(n-1)$: it turns out that such a subset is contained in a finite elementary abelian normal subgroup of $G$ of a particular kind. We deduce a characterization of Property $P(n)$ purely in terms of the group structure. It follows that, if a countable group $G$ has $P(n-1)$ and does not have $P(n)$, then $n$ is the cardinality of a projective space over a finite field. A group $G$ has Property $Q(n)$ if, for every subset $F \subset G$ of size at most $n$, there exists an irreducible unitary representation $\pi$ of $G$ such that $\pi(x) \ne \pi(y)$ for any distinct $x, y$ in $F$. Every group has $Q(2)$. For countable groups, it is shown that Property $Q(3)$ is equivalent to $P(3)$, Property $Q(4)$ to $P(6)$, and Property $Q(5)$ to $P(9)$. For $m, n \ge 4$, the relation between Properties $P(m)$ and $Q(n)$ is closely related to a well-documented open problem in additive combinatorics.