Efficient Bootstrap Research Articles

A Bootstrap Algebraic Multilevel Method for Markov Chains

This work concerns the development of an algebraic multilevel method for computing state vectors of Markov chains. We present an efficient bootstrap algebraic multigrid (AMG) method for this task. In our proposed approach, we employ a multilevel eigensolver, with interpolation built using ideas based on compatible relaxation, algebraic distances, and least squares fitting of test vectors. Our adaptive variational strategy for computation of the state vector of a given Markov chain is then a combination of this multilevel eigensolver and an associated additive multilevel preconditioned correction process. We show that the bootstrap AMG eigensolver by itself can efficiently compute accurate approximations to the steady state vector. An additional benefit of the bootstrap approach is that it yields an accurate interpolation operator for many other eigenmodes. This in turn allows for the use of the resulting multigrid hierarchy as a preconditioner to accelerate the generalized minimal residual (GMRES) iteration for computing an additive correction equation for the approximation to the steady state vector. Unlike other existing multilevel methods for Markov chains, our method does not employ any special processing of the coarse-level systems to ensure that stochastic properties of the fine-level system are maintained there. The proposed approach is applied to a range of test problems involving nonsymmetric M-matrices arising from stochastic matrices and showing promising results.

Computationally efficient bootstrap prediction intervals for returns and volatilities in ARCH and GARCH processes

AbstractWe propose a novel, simple, efficient and distribution‐free re‐sampling technique for developing prediction intervals for returns and volatilities following ARCH/GARCH models. In particular, our key idea is to employ a Box–Jenkins linear representation of an ARCH/GARCH equation and then to adapt a sieve bootstrap procedure to the nonlinear GARCH framework. Our simulation studies indicate that the new re‐sampling method provides sharp and well calibrated prediction intervals for both returns and volatilities while reducing computational costs by up to 100 times, compared to other available re‐sampling techniques for ARCH/GARCH models. The proposed procedure is illustrated by an application to Yen/U.S. dollar daily exchange rate data. Copyright © 2010 John Wiley & Sons, Ltd.

Efficient bootstrap with weakly dependent processes

The efficient bootstrap methodology is developed for overidentified moment conditions models with weakly dependent observation. The resulting bootstrap procedure is shown to be asymptotically valid and can be used to approximate the distributions of t-statistics, the J-statistic for overidentifying restrictions, and Wald, Lagrange multiplier and distance statistics for nonlinear hypotheses. The asymptotic validity of the efficient bootstrap based on a computationally less demanding approximate k-step estimator is also shown. The finite sample performance of the proposed bootstrap is assessed using simulations in an intertemporal consumption based asset pricing model.

Weighted bootstrap for neural model selection

This article proposes a weighted bootstrap procedure, which is an efficient bootstrap technique for neural model selection. Our primary interest in reducing computer effort is to not resample (in the original bootstrap procedure) uniformly from the original sample, but to modify this distribution in order to obtain variance reduction. The performance of the weighted bootstrap is demonstrated on two artificial data sets and one real dataset. Experimental results show that the weighted bootstrap procedure permits an approximately 2 to 1 reduction in replication size.

A functional-based distribution diagnostic for a linear model with correlated outcomes

In this paper we present an easy-to-implement graphical distribution diagnostic for linear models with correlated errors. Houseman et al. (2004) constructed quantile-quantile plots for the marginal residuals of such models, suitably transformed. We extend the pointwise asymptotic theory to address the global stochastic behaviour of the corresponding empirical cumulative distribution function, and describe a simulation technique that serves as a computationally efficient parametric bootstrap for generating representatives of its stochastic limit. Thus, continuous functionals of the empirical cumulative distribution function may be used to form global tests of normality. Through the use of projection matrices, we generalised our methods to include tests that are directed at assessing the normality of particular components of the error. Thus, tests proposed by Lange & Ryan (1989) follow as a special case. Our method works well both for models having independent units of sampling and for those in which all observations are correlated.

An efficient shrinkage bootstrap bias estimator for smooth functions of sample means

Since it is not always possible to calculate bootstrap estimators, they are usually approximated by simulation. In this article, we propose a bootstrap bias estimator for smooth functions of sample means that has less mean squared error, due to the simulation process, than the ordinary bootstrap. The estimator is based on shrinking the bootstrap mean towards the original sample mean. It can easily be implemented while demanding almost no additional computational effort.

EFFICIENT BOOTSTRAP RESAMPLING FOR DEPENDENT DATA

ABSTRACT Many authors have discussed a variety of methods for efficient bootstrap simulation. The data used by these methods are assumed to be i.i.d. In this paper we extend the methodology of two of these methods to be applicable to dependent data. Particularly, we modify the first-order balanced method as well as the re-centering method to be applicable to stationary time series. We study their effect on removing or reducing the re-sampling error in the bootstrap estimate of bias. Numerical examples are introduced to illustrate the favorability of the modified methods.

Sequential determination of the number of bootstrap samples

The primary purpose of this paper is that of developing a sequential Monte Carlo approximation to an ideal bootstrap estimate of the parameter of interest. Using the concept of fixed-precision approximation, we construct a sequential stopping rule for determining the number of bootstrap samples to be taken in order to achieve a specified precision of the Monte Carlo approximation. It is shown that the sequential Monte Carlo approximation is asymptotically efficient in the problems of estimation of the bias and standard error of a given statistic. Efficient bootstrap resampling is discussed and a numerical study is carried out for illustrating the obtained theoretical results.

Efficient Bootstrap Tests for the Goodness of Fit in Covariance Structure Analysis

Asymptotic chi-square tests, such as the normal theory likelihood ratio test, are often used to evaluate the goodness-of-fit of a covariance structure analysis model. Another approach is to use the bootstrap test, which is known to have the desired asymptotic level if model restrictions are taken into account in designing a resampling algorithm. The bootstrap test is. however, computationally very tedious and the problem of nonconvergence and improper solutions often arise in bootstrap resampling. In this paper, we propose a bootstrap test which is based on an approximation, by a quadratic form, to the minimum value of a discrepancy function calculated from each bootstrap sample. Hence, the proposed bootstrap test is efficient in the sense of the amount of computing needed and is free from the problem of nonconvergence and improper solutions with resampling. A Monte Carlo experiment is conducted to compare the performance of the proposed method with that of asymptotic chi-square tests for each combination of three distributions and four sample sizes.

Control variates and importance sampling for efficient bootstrap simulations

Importance sampling and control variates have been used as variance reduction techniques for estimating bootstrap tail quantiles and moments, respectively. We adapt each method to apply to both quantiles and moments, and combine the methods to obtain variance reductions by factors from 4 to 30 in simulation examples.

Bootstrap Estimation of Conditional Distributions

Techniques are developed for bootstrap estimation of conditional distributions, with application to confidence intervals and hypothesis tests for one parameter, conditional on the value of an estimator of another. Both Monte Carlo and saddlepoint methods for approximating bootstrap distributions are considered, and empirical methods are suggested for implementing these techniques. For example, in the case of Monte Carlo methods, we suggest empirical techniques for selecting both the smoothing parameter, necessary to define the estimator, and the importance resampling probabilities, required for efficient bootstrap simulation. The smoothing parameter depends critically on the number of Monte Carlo simulations, as well as on the data. Both our theoretical and numerical results indicate that pivoting can substantially improve performance.

A simulation study of balanced and antithetic bootstrap resampling methods

An extensive simulation study is conducted to compare the performance between balanced and antithetic resampling for the bootstrap in estimation of bias, variance, and percentiles when the statistic of interest is the median, the square root of the absolute value of the mean, or the median absolute deviations from the median. Simulation results reveal that balanced resampling provide better efficiencies in most cases; however, antithetic resampling is superior in estimating bias of the median. We also investigate the possibility of combining an existing efficient bootstrap computation of Efron (1990) with balanced or antithetic resampling for percentile estimation. Results indicate that the combination method does indeed offer gains in performance though the gains are much more dramatic for the bootstrap t statistic than for any of the three statistics of interest as described above.

On importance resampling for the bootstrap

SUMMARY We introduce an empirical method of importance resampling, which does not require analytical calculation of the resampling probabilities. Our method can easily be used as part of a general algorithm for Monte Carlo calculation of bootstrap confidence intervals and hypothesis tests. It produces consistent, efficient and unbiased Monte Carlo approximations. We also present a very general but elementary account of importance resampling, which shows that even optimal importance resampling cannot improve on uniform resampling for calculating bootstrap estimates of bias, variance, skewness and related quantities. This result demonstrates a major difference between importance resampling and other approaches to efficient bootstrap simulation, such as balanced resampling and antithetic resampling, which produce significant improvements in efficiency for a wide range of problems involving the bootstrap.

On efficient bootstrap simulation

SUMMARY It is shown that three new, efficient bootstrap algorithms are asymptotically equivalent. This is done in two ways. First, asymptotic formulae for variances and mean squared errors are derived, and shown to be identical. Secondly, it is demonstrated that two of the methods may be viewed as approximations to the third. The three algorithms considered are the balanced bootstrap and the linear approximation method proposed by Davison, Hinkley & Schechtman (1986), and a centring method proposed by Efron in the context of bias estimation. It is shown that each reduces the order of magnitude of mean squared error by the factor n-1, where n is sample size. These results apply to smooth functions of means.

Discussion of the Papers by Hinkley and Diciccio and Romano

Discussion of the Papers by Hinkley and Diciccio and Romano