Comments on: Augmenting the bootstrap to analyze high dimensional genomic data

Abstract

Our first comment is on the optimization of the model parameter τ controlling the amount of noise in the augmented bootstrap method. In a supervised prediction problem, τ can and should be optimized using, e.g., a cross-validation (CV) procedure, as suggested by the authors. If the prediction accuracy is itself evaluated by cross-validation or a related approach, this yields a nested cross-validation procedure involving an inner-loop (in which the parameter is tuned) and an outer-loop (in which the prediction rule with tuned parameter is evaluated), see Statnikov et al (2005) and Boulesteix (2007). Note that different cross-validation schemes can yield different results due to, e.g., the difference in the size of the considered training subsets. In leave-one-out cross-validation, the training subsets have size n−1, whereas they have size n/2 in 2-fold cross-validation. In the case considered here, it is conceivable that cross-validation schemes with many folds (i.e. with large