Regression using localised functional Bregman divergence

Kanta Naito,Spiridon Penev

doi:10.1214/21-ejs1947

Abstract

This paper is concerned with a unified approach to estimating regression methods based on a certain divergence and its localisation. Some past papers have demonstrated theoretically and numerically that infusing a little localisation in the likelihood-based methods for regression and for density estimation can actually improve the resulting estimators with respect to suitably defined global risk measures. Thus a variety of local likelihood methods have been suggested. We demonstrate that similar effect can also be observed in the general framework discussed in this paper and with respect to robust estimation procedures. Localised versions of robust regression estimation procedures perform better with respect to global risk measures based on minimisation of Bregman divergence measures. An intricate relationship between regression model’s inadequacy and its robustness can be better analysed by using the local approach developed in this paper. We support our claims with a short simulation study.

Highlights

This paper discusses a divergence-based method for estimating a regression function and a local version of this method
We show that choosing a suitable U in the functional Bregman divergence naturally induces a robust version of the resulting localised regression, with a similar risk improvement by the localised estimator when an appropriate choice of G is made
This paper presents a unified way to compose a localised regression inference method by utilising the following triplet (U, G, K): a strictly convex function U for the estimation scheme with the functional Bregman divergence, the link function G in the parametric model utilised, and the kernel function K for localisation

Summary

Introduction

This paper discusses a divergence-based method for estimating a regression function and a local version of this method. We show that choosing a suitable U in the functional Bregman divergence naturally induces a robust version of the resulting localised regression, with a similar risk improvement by the localised estimator when an appropriate choice of G is made. The point-wise application means that in this case d = 1, ∇ means a simple derivative U and we interpret locally, for a fixed t dU (g(t), f (t)) = U (g(t)) − U (f (t)) − U (f (t)){g(t) − f (t)} Using this localised divergence measure at the point t, we define the global (or called functional) Bregman divergence between the densities f and g: DU (g, f ) := dU (g(t), f (t))v(t)dt,

Objectives

Results

Conclusion