Gradient COBRA: A kernel-based consensual aggregation for regression

Abstract

In this article, we introduce a kernel-based consensual aggregation method for regression problems. We aim to flexibly combine individual regression estimators r1, ..., rM using a weighted average where the weights are defined based on predicted features given by all the basic estimators and some kernel function. This work extends the context of Biau et al. (2016) to a more general kernel-based framework. We show that this more general configuration also inherits the consistency of the basic consistent estimators, and the same convergence rate as in the classical method is achieved. Moreover, an optimization method based on gradient descent algorithm is proposed to efficiently and rapidly estimate the key parameter of the strategy. Various of numerical experiments carried out on several simulated and real datasets are also provided to illustrate the efficiency and accuracy of the method with the introduction of gradient descent algorithm and smoother kernel functions respectively. Moreover, a domain adaptation-like property of the method is also illustrated on a physics data provided by Commissariat à l'Énergie Atomique (CEA).