Ordered smoothers with exponential weighting

Elena Chernousova,Ekaterina Krymova,Yuri Golubev

doi:10.1214/13-ejs849

Abstract

The main goal in this paper is to propose a new approach to deriving oracle inequalities related to the exponential weighting method. The paper focuses on recovering an unknown vector from noisy data with the help of the family of ordered smoothers [12]. The estimators withing this family are aggregated using the exponential weighting method and the aim is to control the risk of the aggregated estimate. Based on the natural probabilistic properties of the unbiased risk estimate, we derive new oracle inequalities for the mean square risk and show that the exponential weighting permits to improve Kneip’s oracle inequality.

Highlights

Introduction and main resultsThis paper deals with the simple linear modelYi = μi + σξi, i = 1, 2, . . . , n, (1.1)where ξ is a standard white Gaussian noise, i.e. ξi are Gaussian i.i.d. random variables with Eξi = 0 and Eξi2 = 1
For the sake of simplicity it is assumed that the noise level σ > 0 is known
The goal is to estimate an unknown vector μ ∈ Rn based on the data Y = (Y1, . . . , Yn)

Summary

Introduction and main results

The main goal in this paper is to show that for the exponential weighting method one can obtain an oracle inequality with a smaller remainder term than the one in Theorem 1.1, Equation (1.12). For smoothing splines and equidistant design, the set of ordered multipliers is given by (1.8) and this condition follows from the well-known asymptotic formula λk (πk)2m for large k (see [6] for details). To compare the actual remainder terms in (1.17) and (1.12) and to find out what β is good from a practical viewpoint, a numerical experiment has been carried out The goal in this experiment is to compare the exponential weighting methods for β = {0, 1, 2, 4} combined with the cubic smoothing splines for the equidistant design. When rH(μ)/σ2 is not large, the exponential weighting works usually better

Proofs

Auxiliary facts