Efficient Bootstrap Research Articles

Two-sample test with g-modeling and its applications.

Many real data analyses involve two-sample comparisons in location or in distribution. Most existing methods focus on problems where observations are independently and identically distributed in each group. However, in some applications the observed data are not identically distributed but associated with some unobserved parameters which are identically distributed. To address this challenge, we propose a novel two-sample testing procedure as a combination of the -modeling density estimation introduced by Efron and the two-sample Kolmogorov-Smirnov test. We also propose efficient bootstrap algorithms to estimate the statistical significance for such tests. We demonstrate the utility of the proposed approach with two biostatistical applications: the analysis of surgical nodes data with binomial model and differential expression analysis of single-cell RNA sequencing data with zero-inflated Poisson model.

Two-Stage Robust and Sparse Distributed Statistical Inference for Large-Scale Data

In this paper, we address the problem of conducting statistical inference in settings involving large-scale data that may be high-dimensional and contaminated by outliers. The high volume and dimensionality of the data require distributed processing and storage solutions. We propose a two-stage distributed and robust statistical inference procedures coping with high-dimensional models by promoting sparsity. In the first stage, known as model selection, relevant predictors are locally selected by applying robust Lasso estimators to the distinct subsets of data. The variable selections from each computation node are then fused by a voting scheme to find the sparse basis for the complete data set. It identifies the relevant variables in a robust manner. In the second stage, the developed statistically robust and computationally efficient bootstrap methods are employed. The actual inference constructs confidence intervals, finds parameter estimates and quantifies standard deviation. Similar to stage 1, the results of local inference are communicated to the fusion center and combined there. By using analytical methods, we establish the favorable statistical properties of the robust and computationally efficient bootstrap methods including consistency for a fixed number of predictors, and robustness. The proposed two-stage robust and distributed inference procedures demonstrate reliable performance and robustness in variable selection, finding confidence intervals and bootstrap approximations of standard deviations even when data is high-dimensional and contaminated by outliers.

Efficient bootstrap stacking ensemble learning model applied to wind power generation forecasting

The use of wind energy plays a vital role in society owing to its economic and environmental importance. Knowing the wind power generation within a specific time window is useful for facilitating decision making in terms of maintenance, electricity market clearing, and reload sharing. However, the effect of climatic and demographic factors on wind power generation sometimes makes time series forecasting a complex task. Thus, this study evaluates an ensemble learning model that combines bagging and stacking methods applied to time series forecasting with very short-term (10 and 30-minutes) and short-term (60 and 120-minutes) evaluations of wind power generation. Arithmetic and weighted average values were used to integrate the samples from bagging strategy. The weights are defined through multi-objective optimization using a non-dominated sorting genetic algorithm – version II, aiming to enhance the forecasting accuracy and stability simultaneously. To demonstrate the wide applicability of the non-linear ensemble learning model, it is extensively tested with measurement data collected from two wind farms in Bahia State, Brazil. The experimental results show that the proposed ensemble learning model achieves a better forecasting performance than single forecasting models, such as stacking, machine learning, artificial neural networks, and statistical models, with values of approximately 7.63%, 7.58%, 20.8%, and 25%, respectively, in terms of the errors for out-of-sample forecasting reduction. In addition, results with a weighted average are 87.5% superior to those with an arithmetic average for out-of-sample wind power forecasting in the evaluated forecasting horizons. The findings show that the integration of ensemble strategies can provide accurate forecasting results in the renewable energy field.

Enhancing Recommender Systems With a Stimulus-Evoked Curiosity Mechanism

Classical algorithms in recommender systems (RS) mainly emphasis on achieving high accuracy and thus recommend items precisely matching a user's past choices. However, the user may gradually lose interest and crave something more inspiring. In psychology, curiosity is a critical human nature and can be efficient bootstrap exploratory behaviors, thus this phenomenon can be explained as insufficient stimulation to induce curiosity regard to recommended items. Inspired from the above, this work proposes a Curiosity-drive Recommendation Framework (CdRF) which incorporates a highly innovative Stimulus-evoked Curiosity mechanism (SeCM) together with a basic accuracy-oriented algorithm via Borda count. In SeCM, we first estimate the stimulus intensity appearing on each item for each user and then model personalized curiosity among the calculated intensities using Wundt curve. For the target user, the output of CdRF is a ranked list of N N items which are both relevant and highly curiousness. We conduct extensive experiments using four public datasets to evaluate the performance of each specification of SeCM as well as the whole framework CdRF. The results reveal that SeCM can flexibly generate diversified items and CdRF can increase diversity in terms of ILS, Newness and AD while compromising very little Precision. This kind of research also offers a way to understand both individual differences in curiosity and how curiosity contributes to item exploration at the level of RS.

Prediction of Finite Population Proportion When Responses are Misclassified

Abstract We propose a model-based predictive estimator of the finite population proportion of a misclassified binary response, when information on the auxiliary variable(s) is available for all units in the population. Asymptotic properties of the misclassification-adjusted predictive estimator are also explored. We propose a computationally efficient bootstrap variance estimator that exhibits better performance compared to usual analytical variance estimator. The performance of the proposed estimator is compared with other commonly used design-based estimators through extensive simulation studies. The results are supplemented by an empirical study based on literacy data.

Reliability analysis using bootstrap information criterion for small sample size response functions

Statistical model selection and evaluation methods like Akaike information criteria (AIC) and Monte Carlo simulation (MCS) have often established efficient output for reliability analysis with large sample size. Information criterion can provide better model selection and evaluation in small sample sizes setup by considering the well-known measure of bootstrap resampling. Our purpose is to utilize the capabilities of bootstrap resampling in information criterion to check for uncertainty arising from model selection as well as statistics of interest for small sample size using reliability analysis. In this study, therefore, a unique and efficient simulation scheme is proposed which contemplates the best model selection devised from efficient bootstrap simulation or variance reduced bootstrap information criterion to be combined with reliability analysis. It is beneficial to compute the spread of reliability values as against solitary fixed values with desirable statistics of interest for uncertainty analysis. The proposed simulation scheme is verified using a number of sample size focused response functions under repetitions-centred approach with AIC-based reliability analysis for comparison and MCS for accuracy. The results show that the proposed simulation scheme aids the statistics of interest by reducing the spread and hence the uncertainty in sample size-based reliability analysis when compared with conventional methods.

Efficiency balanced designs for bootstrap simulations

This paper considers the balanced resampling design approach of generating bootstrap samples to handle the simulation error problem in bootstrap estimation of mean and variance of a sample statistic of interest. A class of balanced n-arry block design, slightly different from the definition of Tocher (1952), is defined and its necessary conditions for the existence of these designs are derived. In particular, an infinite series of balanced n-arry block designs are found to be efficiency balanced, i.e., every contrast of element (observation) totals has the same loss of information. As an application in bootstrap method, the eccentric class of efficiency balanced n-arry block design (termed Efficient Bootstrap Sample Design (EBSD(n)) can be used to generate resamples of sample size n to achieve an exact bootstrap estimation of mean and variance of statistic “sample average”, i.e., a collection of bootstrap data samples and corresponding bootstrap sample averages where mean and variance of this empirical collection of sample averages match the first two theoretical central moments i.e., moments about the sample average. The EBSD(n) solely depends on the existence of balanced incomplete block design B[ 4n-2, 2n-1, n, n-1] for n ≥ 2. Thus the implementation of computer algorithm for spawning bootstrap samples using EBSD(n) is simple and straightforward for recapitulating sampling characteristics such as standard error of predefined computable symmetric and non symmetric sample statistic of interest.

Design and implementation of a 350-watt BLDC motor driver using an Insulated Gate Bipolar Transistor (IGBT)

IGBT device was not commonly used as a BLDC motor speed controller. This paper described the successful of using IGBTs in a 350Watt BLDC motor speed controller. Triple bridge configuration with PWM (Pulse Width Modulation) wave signal for power tuning was mainly integrated as a main switching method in order to drive the current commutation cycle. Also, an efficient bootstrap IC was designated to perform as the high and low side driver. The data were measured by using a current sensor and a DC voltage sensor to measure the current and the voltage delivered from the power source. Then, IGBT device was activated based on triple bridge commutation cycle and phase based on motor hall effect sensor triggering signal. The performance was also tested by varying the duty cycle input ratio and its PWM frequency. It can conclude that the BLDC motor controller can be made by using an IGBT and added gate drivers for the IGBT. Meanwhile, to adjust the commutation cycle, a high speed processor was used to generate gate activation and PWM signal. Changing the PWM duty cycle starting from 10% to 90% by turning the setting point, observed that BLDC motor rotates successfully without load and maximum power used was 111.20 watts with 2,32 amperes current at 48 volts supply voltage.

Decreasing uncertainty in flood frequency analyses by including historic flood events in an efficient bootstrap approach

Abstract. Flood frequency curves are usually highly uncertain since they are based on short data sets of measured discharges or weather conditions. To decrease the confidence intervals, an efficient bootstrap method is developed in this study. The Rhine river delta is considered as a case study. We use a hydraulic model to normalize historic flood events for anthropogenic and natural changes in the river system. As a result, the data set of measured discharges could be extended by approximately 600 years. The study shows that historic flood events decrease the confidence interval of the flood frequency curve significantly, specifically in the range of large floods. This even applies if the maximum discharges of these historic flood events are highly uncertain themselves.

Applying the rescaling bootstrap under imputation: a simulation study

ABSTRACTResampling methods are a common measure to estimate the variance of a statistic of interest when data consist of nonresponse and imputation is used as compensation. Applying resampling methods usually means that subsamples are drawn from the original sample and that variance estimates are computed based on point estimators of several subsamples. However, newer resampling methods such as the rescaling bootstrap of Chipperfield and Preston [Efficient bootstrap for business surveys. Surv Methodol. 2007;33:167–172] include all elements of the original sample in the computation of its point estimator. Thus, procedures to consider imputation in resampling methods cannot be applied in the ordinary way. For such methods, modifications are necessary. This paper presents an approach applying newer resampling methods for imputed data. The Monte Carlo simulation study conducted in the paper shows that the proposed approach leads to reliable variance estimates in contrast to other modifications.

Uniform asymptotic inference and the bootstrap after model selection

Recently, Tibshirani et al. [J. Amer. Statist. Assoc. 111 (2016) 600–620] proposed a method for making inferences about parameters defined by model selection, in a typical regression setting with normally distributed errors. Here, we study the large sample properties of this method, without assuming normality. We prove that the test statistic of Tibshirani et al. (2016) is asymptotically valid, as the number of samples $n$ grows and the dimension $d$ of the regression problem stays fixed. Our asymptotic result holds uniformly over a wide class of nonnormal error distributions. We also propose an efficient bootstrap version of this test that is provably (asymptotically) conservative, and in practice, often delivers shorter intervals than those from the original normality-based approach. Finally, we prove that the test statistic of Tibshirani et al. (2016) does not enjoy uniform validity in a high-dimensional setting, when the dimension $d$ is allowed grow.

Bootstrapping the out-of-sample predictions for efficient and accurate cross-validation

Cross-Validation (CV), and out-of-sample performance-estimation protocols in general, are often employed both for (a) selecting the optimal combination of algorithms and values of hyper-parameters (called a configuration) for producing the final predictive model, and (b) estimating the predictive performance of the final model. However, the cross-validated performance of the best configuration is optimistically biased. We present an efficient bootstrap method that corrects for the bias, called Bootstrap Bias Corrected CV (BBC-CV). BBC-CV’s main idea is to bootstrap the whole process of selecting the best-performing configuration on the out-of-sample predictions of each configuration, without additional training of models. In comparison to the alternatives, namely the nested cross-validation (Varma and Simon in BMC Bioinform 7(1):91, 2006) and a method by Tibshirani and Tibshirani (Ann Appl Stat 822–829, 2009), BBC-CV is computationally more efficient, has smaller variance and bias, and is applicable to any metric of performance (accuracy, AUC, concordance index, mean squared error). Subsequently, we employ again the idea of bootstrapping the out-of-sample predictions to speed up the CV process. Specifically, using a bootstrap-based statistical criterion we stop training of models on new folds of inferior (with high probability) configurations. We name the method Bootstrap Bias Corrected with Dropping CV (BBCD-CV) that is both efficient and provides accurate performance estimates.

Automation of image categorization with most relevant negatives

Image categorization requires the algorithm to be learned in order to obtain the efficient categorization. The algorithm used for image categorization may misclassify images that are visually similar to the positive ones. Generally, sampling negatives is done at random. In this paper, we have improved Negative Bootstrap in an efficient way to obtain most relevant negatives. To obtain most misclassified visually similar images in a faster way, fast intersection kernel SVM is generalized and used for classification. The accuracy of classified visual concepts is obtained by using the performance metrics. Several different metrics have been used to show the accuracy of relevant negatives. Manual labeling of negatives could be avoided by using the efficient negative bootstrap algorithm.

Efficient bootstrap estimates for tail statistics

Abstract. Bootstrap resamples can be used to investigate the tail of empirical distributions as well as return value estimates from the extremal behaviour of the sample. Specifically, the confidence intervals on return value estimates or bounds on in-sample tail statistics can be obtained using bootstrap techniques. However, non-parametric bootstrapping from the entire sample is expensive. It is shown here that it suffices to bootstrap from a small subset consisting of the highest entries in the sequence to make estimates that are essentially identical to bootstraps from the entire sample. Similarly, bootstrap estimates of confidence intervals of threshold return estimates are found to be well approximated by using a subset consisting of the highest entries. This has practical consequences in fields such as meteorology, oceanography and hydrology where return values are calculated from very large gridded model integrations spanning decades at high temporal resolution or from large ensembles of independent and identically distributed model fields. In such cases the computational savings are substantial.

Power-transformed linear regression on quantile residual life for censored competing risks data

ABSTRACTThis paper proposes a power-transformed linear quantile regression model for the residual lifetime of competing risks data. The proposed model can describe the association between any quantile of a time-to-event distribution among survivors beyond a specific time point and the covariates. Under covariate-dependent censoring, we develop an estimation procedure with two steps, including an unbiased monotone estimating equation for regression parameters and cumulative sum processes for the Box–Cox transformation parameter. The asymptotic properties of the estimators are also derived. We employ an efficient bootstrap method for the estimation of the variance–covariance matrix. The finite-sample performance of the proposed approaches are evaluated through simulation studies and a real example.

Bootstrap inference for instrumental variable models with many weak instruments

This study’s main contribution is to theoretically analyze the application of bootstrap methods to instrumental variable models when the available instruments may be weak and the number of instruments goes to infinity with the sample size. We demonstrate that a standard residual-based bootstrap procedure cannot consistently estimate the distribution of the limited information maximum likelihood estimator or Fuller (1977) estimator under many/many weak instrument sequence. The primary reason is that the standard procedure fails to capture the instrument strength in the sample adequately. In addition, we consider the restricted efficient (RE) bootstrap of Davidson and MacKinnon (2008, 2010, 2014) that generates bootstrap data under the null (restricted) and uses an efficient estimator of the coefficient of the reduced-form equation (efficient). We find that the RE bootstrap is also invalid; however, it effectively mimics more key features in the limiting distributions of interest, and thus, is less distorted in finite samples than the standard bootstrap procedure. Finally, we propose modified bootstrap procedures that provide a valid distributional approximation for the two estimators with many/many weak instruments. A Monte Carlo experiment shows that hypothesis testing based on the asymptotic normal approximation can have severe size distortions in finite samples. Instead, our modified bootstrap procedures greatly reduce these distortions.

Combinatorial approach to inference in partially identified incomplete structural models

We propose a computationally feasible inference method in finite games of complete information. Galichon and Henry, 2011 and Beresteanu, Molchanov, and Molinari, 2011 show that the empirical content in such models is characterized by a collection of moment inequalities whose number increases exponentially with the number of discrete outcomes. We propose an equivalent characterization based on classical combinatorial optimization methods that allows the construction of confidence regions with an efficient bootstrap procedure that runs in linear computing time. The method can be applied to the empirical analysis of cooperative and noncooperative games, instrumental variable models of discrete choice, and revealed preference analysis. We propose an application to the determinants of long term elderly care choices. Incomplete structural models multiple equilibria partial identification sharp bounds confidence regions max‐flow–min‐cut functional quantile bootstrap elderly care C13 C72

Robust, Scalable, and Fast Bootstrap Method for Analyzing Large Scale Data

In this paper we address the problem of performing statistical inference for large scale data sets i.e., Big Data. The volume and dimensionality of the data may be so high that it cannot be processed or stored in a single computing node. We propose a scalable, statistically robust and computationally efficient bootstrap method, compatible with distributed processing and storage systems. Bootstrap resamples are constructed with smaller number of distinct data points on multiple disjoint subsets of data, similarly to the bag of little bootstrap method (BLB) [1]. Then significant savings in computation is achieved by avoiding the re-computation of the estimator for each bootstrap sample. Instead, a computationally efficient fixed-point estimation equation is analytically solved via a smart approximation following the Fast and Robust Bootstrap method (FRB) [2]. Our proposed bootstrap method facilitates the use of highly robust statistical methods in analyzing large scale data sets. The favorable statistical properties of the method are established analytically. Numerical examples demonstrate scalability, low complexity and robust statistical performance of the method in analyzing large data sets.

Robust sparse regression and tuning parameter selection via the efficient bootstrap information criteria

There is currently much discussion about lasso-type regularized regression which is a useful tool for simultaneous estimation and variable selection. Although the lasso-type regularization has several advantages in regression modelling, owing to its sparsity, it suffers from outliers because of using penalized least-squares methods. To overcome this issue, we propose a robust lasso-type estimation procedure that uses the robust criteria as the loss function, imposing L1-type penalty called the elastic net. We also introduce to use the efficient bootstrap information criteria for choosing optimal regularization parameters and a constant in outlier detection. Simulation studies and real data analysis are given to examine the efficiency of the proposed robust sparse regression modelling. We observe that our modelling strategy performs well in the presence of outliers.

Combinatorial Bootstrap Inference in Partially Identified Incomplete Structural Models

We propose a computationally feasible inference method in finite games of complete information. Galichon and Henry (2011) and Beresteanu, Molchanov and Molinari (2011) show that such models are equivalent to a collection of moment inequalities that increases exponentially with the number of discrete outcomes. We propose an equivalent characterization based on classical combinatorial optimization methods that alleviates this computational burden and allows the construction of confidence regions with an efficient combinatorial bootstrap procedure that runs in linear computing time. The method can also be applied to the empirical analysis of cooperative and noncooperative games, instrumental variable models of discrete choice and revealed preference analysis. We propose an application to the determinants of long term elderly care choices.