Nonparametric regression with the scale depending on auxiliary variable

Abstract

The paper is devoted to the problem of estimation of a univariate component in a heteroscedastic nonparametric multiple regression under the mean integrated squared error (MISE) criteria. The aim is to understand how the scale function should be used for estimation of the univariate component. It is known that the scale function does not affect the rate of the MISE convergence, and as a result sharp constants are explored. The paper begins with developing a sharp-minimax theory for a pivotal model $Y=f(X)+σ(X,\mathbf{Z})\varepsilon$, where $\varepsilon$ is standard normal and independent of the predictor X and the auxiliary vector-covariate $\mathbf{Z}$. It is shown that if the scale $σ(x,\mathbf{z})$ depends on the auxiliary variable, then a special estimator, which uses the scale (or its estimate), is asymptotically sharp minimax and adaptive to unknown smoothness of f(x). This is an interesting conclusion because if the scale does not depend on the auxiliary covariate $\mathbf{Z}$, then ignoring the heteroscedasticity can yield a sharp minimax estimation. The pivotal model serves as a natural benchmark for a general additive model $Y=f(X)+g(\mathbf{Z})+σ(X,\mathbf{Z})\varepsilon$, where $\varepsilon$ may depend on $(X,\mathbf{Z})$ and have only a finite fourth moment. It is shown that for this model a data-driven estimator can perform as well as for the benchmark. Furthermore, the estimator, suggested for continuous responses, can be also used for the case of discrete responses. Bernoulli and Poisson regressions, that are inherently heteroscedastic, are particular considered examples for which sharp minimax lower bounds are obtained as well. A numerical study shows that the asymptotic theory sheds light on small samples.