Least squares type estimation of the transition density of a particular hidden Markov chain

Abstract

In this paper, we study the following model of hidden Markov chain: $Y_i=X_i+ε_i$, $i=1,...,n+1$ with $(X_i)$ a real-valued stationary Markov chain and $(ε_i)_{1\leq i\leq n+1}$ a noise having a known distribution and independent of the sequence $(X_i)$. We present an estimator of the transition density obtained by minimization of an original contrast that takes advantage of the regressive aspect of the problem. It is selected among a collection of projection estimators with a model selection method. The $L^2$-risk and its rate of convergence are evaluated for ordinary smooth noise and some simulations illustrate the method. We obtain uniform risk bounds over classes of Besov balls. In addition our estimation procedure requires no prior knowledge of the regularity of the true transition. Finally, our estimator permits to avoid the drawbacks of quotient estimators.