Two-stage multiple kernel learning for supervised dimensionality reduction

Abstract

In supervised dimensionality reduction methods for pattern recognition tasks, the information of the class labels is considered through the process of reducing the input dimensionality, to improve the classification accuracy. Using nonlinear mappings for this purpose makes these models more appropriate for nonlinearly distributed data. In this paper, a new nonlinear supervised dimensionality reduction model is introduced. The dimensionality reduction process in this model is performed through a multiple kernel learning paradigm in two stages. In the first stage, three suitable criteria for supervised dimensionality reduction containing fisher, homoscedasticity, and between-class distance criteria are used to find the kernel weights. With these weights, a linear combination of several valid kernels is computed to make a new suitable kernel function. In the second stage, the kernel discriminant analysis method is employed for nonlinear supervised dimensionality reduction using the kernel computed in the first stage. Many experiments on a variety of real-world datasets including handwritten digits images, objects images, and other datasets, show that the proposed approach among a number of well-known related techniques, results in accurate and fast classifications.