Input Variable Selection in Kernel Fisher Discriminant Analysis

Abstract

Variable selection serves a dual purpose in statistical classification problems: it enables one to identify the input variables which separate the groups well, and a classification rule based on these variables frequently has a lower error rate than the rule based on all the input variables. Kernel Fisher discriminant analysis (KFDA) is a recently proposed powerful classification procedure, frequently applied in cases characterized by large numbers of input variables. The important problem of eliminating redundant input variables before implementing KFDA is addressed in this paper. A backward elimination approach is employed, and a criterion which can be used for recursive elimination of input variables is proposed. The merit of the proposal is evaluated in a simulation study and in terms of its performance when applied to two benchmark data sets.