Abstract Group Automorphisms of Two-Step Nilpotent Lie Groups and Partial Automatic Continuity

Abstract

Any abstract (not necessarily continuous) group automorphism of a simple, compact Lie group must be continuous due to Cartan (1930) and van der Waerden (1933). The purpose of this paper is to study a similar question in nilpotent Lie groups. For uncountably many simply connected 2-step nilpotent Lie groups, we show any abstract group automorphism is continuous ``up to discontinuity due to the center and ring automorphisms of $\mathbb{R}$-algebras.'' Such groups include (1) a generic simply connected 2-step nilpotent Lie group $N$ with $\dim [N,N]\geq \frac{\dim N-1}{2}$, (2) all the 12 simply connected 2-step nilpotent Lie groups of dimensions 6 or less, and (3) Iwasawa N-groups of simple Lie groups of rank 1, i.e., ``Heisenberg groups.'' All of the three cases are derived from one key result that gives a sufficient condition for the automorphism group to be of the type described above. Previously, only countably many simply connected nilpotent Lie groups were known to satisfy the same property. We obtain similar results for complex Lie groups as well. We also show if any Lie group $G$ and the group $N$ of (1) or (3) are isomorphic as abstract groups, then they are isomorphic as Lie groups.