Limitations on Low Variance k-Fold Cross Validation in Learning Set of Rules Inducers

Abstract

One of the standard methods in a verification of predictive models is a cross validation. In this paper, we examined prediction stability of simple learning set of rules classifier under the k-fold cross validation. We described a class of rules that can pass the k-fold cross validation with zero or a very low variance in accuracy of prediction. The lossless prediction of correct/incorrect assignment distribution theorem, given by the so-called k-fold stable rules, is established, and its implications are discussed and applied in the experiments.