Testing Forecast Accuracy of Expectiles and Quantiles with the Extremal Consistent Loss Functions

Abstract

Forecast evaluations aim to choose an accurate forecast for making decisions by using loss functions. However, different loss functions often generate different ranking results for forecasts, which complicates the task of comparisons. In this paper, we develop statistical tests for comparing performances of forecasting expectiles and quantiles of a random variable under consistent loss functions. The test statistics are constructed with the extremal consistent loss functions of Ehm et.al. (2016). The null hypothesis of the tests is that a benchmark forecast at least performs equally well as a competing one under all extremal consistent loss functions. It can be shown that if such a null holds, the benchmark will also perform at least equally well as the competitor under all consistent loss functions. Thus under the null, when different consistent loss functions are used, the result that the competitor does not outperform the benchmark will not be altered. We establish asymptotic properties of the proposed test statistics and propose to use the re-centered bootstrap to construct their empirical distributions. Through simulations, we show the proposed test statistics perform reasonably well. We then apply the proposed method on (1) re-examining abilities of some often-used predictors on forecasting risk premium of the S&P500 index; (2) comparing performances of experts' forecasts on annual growth of U.S. real gross domestic product; (3) evaluating performances of estimated daily value at risk of the S&P500 index.