The effects of order on learning classifications by example: Heuristics for finding the optimal order

Abstract

The effects of the order of training examples on the performance of learning models is examined, and shown to be an important variable affecting the number of classification errors which occur. Instances are found where changing the order of examples can cause error rates to vary from 0% to 67%. This level of variation may cause problems in testing or comparing learning programs, where it may be useful to test performance using the best-possible order of training examples. Two heuristics for finding the optimal order of examples are proposed and tested. On 96 variations of two classification problems the heuristics found either optimal or next-to-optimal solutions.