Minimaxity and Efficiency in Exponential Family Regression: From Star-Shaped to Convex Constraints

Abstract

This paper establishes the minimax estimation rate for nonparametric exponential family regression under star-shaped constraints. We consider a parameter space $K$ that is a star-shaped subset of the hypercube $[-M, M]^n$ for a known constant $M > 0$. We operate under the assumption that the underlying exponential family is nonsingular with a twice continuously differentiable log-partition function. Our main result demonstrates that the minimax rate of the $\ell _{2}$ error of such estimation problem is $\epsilon^{*2} \wedge \operatorname{diam}(K)^2$ up to constants exclusively depending on $M$. Here, the critical radius $\epsilon^*$ is defined as \begin{equation*} \epsilon^* = \sup \{\epsilon \left\lvert\right. \epsilon^2 \kappa(M) \le \log N^{\text{loc}}(\epsilon,c)\}, \end{equation*} where $N^{\text{loc}}(\epsilon,c)$ denotes the local metric entropy of $K$, and $\kappa(M) > 0, c>0$ are constants depending only on $M$. Such minimax rate is established by a match between an information-theoretic lower bound and an upper bound implied by a theoretical algorithm. Furthermore, we investigate the computational aspects of this estimation problem. Under mildly stronger assumptions on the constraint set $K$, we propose a computationally efficient, polynomial-time algorithm. We prove that the resulting estimator achieves the minimax optimal rate up to poly-logarithmic factors in the dimension $n$ and the geometric parameters of $K$. Finally, to illustrate the efficacy of our framework, we derive the minimax optimal rates for some concrete examples.