Local discriminant basis neural network ensembles with cross-validation

Abstract

In this paper, the possibility of using an orthonormal basis to train a collection of artificial neural networks (ANNs) in a face recognition task is discussed. This orthonormal basis is selected from a dictionary of orthonormal bases consisting of wavelet packets. Here, a basis is obtained by maximizing a certain discriminant measure (e.g., l/sup 2/) among classes of training images. Once such a basis is selected, its basis vectors are ordered according to their power of discrimination and the first N most local discriminant basis vectors are retained for image decomposition. By projecting all training images onto an individual basis vector of these N most discriminant basis vectors, N versions of the training set at different spatial/scale resolutions are then created. Next, N multilayer feedforward neural networks are trained independently by N different resolution-specific training sets. After networks have been trained, they are combined to form an ensemble of networks. In this paper, the more reliable estimate of the ensemble generalization error is achieved by applying the concept of cross-validation to local discriminant basis neural network ensembles. Finally, the previous and the extended results are given in order to show that the combination of disjoint training sets (cross-validation) and preprocessing (local discriminant basis) methods can improve the generalization error.