Optimal function spaces for continuity of the Hessian determinant as a distribution

Abstract

We establish optimal continuity results for the action of the Hessian determinant on spaces of Besov type into the space of distributions on $\mathbb{R}^N$. In particular, inspired by recent work of Brezis and Nguyen on the distributional Jacobian determinant, we show that the action is continuous on the Besov space of fractional order $B(2-\frac{2}{N},N)$, and that all continuity results in this scale of Besov spaces are consequences of this result. A key ingredient in the argument is the characterization of $B(2-\frac{2}{N},N)$ as the space of traces of functions in the Sobolev space $W^{2,N}(\mathbb{R}^{N+2})$ on the subspace $\mathbb{R}^N$ of codimension $2$. The most delicate and elaborate part of the analysis is the construction of a counterexample to continuity in $B(2-\frac{2}{N},p)$ with $p>N$.