Gradient COBRA: A kernel-based consensual aggregation for regression

Abstract

In this article, we introduce a kernel-based consensual aggregation method for regression problems. We aim to exibly combine individual regression estimators $r_1, \ldots, r_M$ using a weighted average where the weights are dened based on predicted features given by all the basic estimators and some kernel function. This work extends the context of Biau et al. (2016) to a more general kernel-based framework. We show that this more general conguration also inherits the consistency of the basic consistent estimators, and the same convergence rate as in the classical method is achieved. Moreover, an optimization method based on gradient descent algorithm is proposed to eciently and rapidly estimate the key parameter of the strategy. Various numerical experiments carried out on several simulated and real datasets are also provided to illustrate the eciency and accuracy of the proposed method. Moreover, a domain adaptation-like property of the aggregation strategy is also illustrated on a physics data provided by Commissariat {\`a} l'{\'E}nergie Atomique (CEA).