An exact iterated bootstrap algorithm for small-sample bias reduction

Abstract

It is well known that bootstrap accuracy can be theoretically enhanced by iterating the bootstrap procedure. Monte Carlo approximation to bootstrap iteration incurs prohibitively expensive computational cost, especially when higher levels of resampling are involved. The theoretical gain promised by high-level bootstrap iteration can thus hardly be materialized in practice. By considering bootstrap iteration as a Markov process, we propose an algorithm for its implementation in the context of small-sample bias reduction. The algorithm caters for any number of bootstrap iterations and computes exact bootstrap bias-corrected estimates without the need for extensive Monte Carlo resampling. We discuss the practical value of our algorithm in situations where infinite-level bootstrap iteration yields an unbiased estimate irrespective of the sample size and where our algorithm converges rapidly. Numerical examples are given to illustrate applications to estimates such as functions of sample means, sample quantiles and the Nadaraya–Watson estimate.