Improved multi-kernel classification machine with Nyström approximation technique and Universum data

Abstract

Universum learning can reflect priori knowledge about application domain and improve classification performances. The kernelized modification of Ho–Kashyap algorithm with squared approximation of the misclassification errors (KMHKS) is an effective learning machine for nonlinearly separable classification problems. While KMHKS only adopts one kernel function, so a multi-kernel classification machine with reduced complexity named Nyström approximation matrix with multiple KMHKSs (NMKMHKS) has been developed. But NMKMHKS has to initialize many parameters and it has not an ability to process noises well. To this end, some scholars propose an improved multi-kernel classification machine with Nyström approximation technique (INMKMHKS). INMKMHKS is based on a new way of generating kernel functions and a new Nyström approximation technique. Related experiments have validated that INMKMHKS possesses five advantages: (1) avoiding the problem of setting too many parameters; (2) keeping comparable space and computational complexities after comparing with NMKMHKS; (3) having a tighter generalization risk bound in terms of Rademacher complexity analysis; (4) possessing an ability to process noises and practical images; (5) a superior recognition can be gotten in a strong correlation between multiple used kernels, which can give a guide advice for choosing kernels. But for traditional multiple kernel learning (MKL), many classification machines including INMKMHKS focus on MKL optimizations, for example, the optimization of model for a MKL. It is difficult for us to find or create a new optimization way now. Indeed, if one pays more attention to data themselves, performance of MKL will also have an improvement. This paper adopts INMKMHKS as a basic learning machine and focuses on data themselves, then it introduces Universum learning into the procedure of INMKMHKS and proposes Universum-based INMKMHKS (Uni-INMKMHKS). The motivation of Uni-INMKMHKS is that it can design a learning machine from data perspective and avoid paying too much attention to MKL optimization which is difficult for scholars to some extent. The contribution of Uni-INMKMHKS is that it has a better recognition than INMKMHKS in average with Universum learning used and inherits the advantages of INMKMHKS simultaneously.