Primitive normalisers in quasipolynomial time

Abstract

The normaliser problem has as input two subgroups $H$ and $K$ of the symmetric group $S_n$, and asks for a generating set for $N_K(H)$: it is not known to have a subexponential time solution. It is proved in [Roney-Dougal & Siccha, 2020] that if $H$ is primitive then the normaliser problem can be solved in quasipolynomial time. We show that for all subgroups $H$ and $K$ of $S_n$, in quasipolynomial time we can decide whether $N_{S_n}(H)$ is primitive, and if so compute $N_K(H)$. Hence we reduce the question of whether one can solve the normaliser problem in quasipolynomial time to the case where the normaliser is known not to be primitive.