MREKLM: A fast multiple empirical kernel learning machine

Abstract

Multiple Empirical Kernel Learning (MEKL) explicitly maps samples into different empirical feature spaces in which the kernel features of the mapped samples can be directly provided. Thus, MEKL is much easier than the conventional Multiple Kernel Learning (MKL) in terms of processing and analyzing the structure of mapped feature spaces. However, the computational complexity of MEKL with M empirical feature spaces is O(MN3) where N is the number of training samples. The dimensions of the generated empirical feature spaces are approximate to N. When dealing with large-scale problems, MEKL cannot handle them properly due to the severe computation and memory burden. Moreover, most existing MEKL utilizes the gradient decent optimization to learn classifiers, but it is time consuming for training. Therefore, this paper proposes a Multiple Random Empirical Kernel Learning Machine (MREKLM) to overcome these problems. The proposed MREKLM adopts the random projection idea to map samples into multiple low-dimensional empirical feature spaces with lower computational complexity O(MP3), where P(≪N) is the number of the randomly selected samples. After that, MREKLM adopts an analytical optimization approach to directly deal with multi-class problems. The computational complexity of MREKLM is O(M3P3). Experimental results also validate both efficiency and effectiveness of the proposed MREKLM. The contributions of this work are: (1) proposing a fast MEKL algorithm named MREKLM, (2) introducing an efficient random empirical kernel mapping approach, and (3) extending the capability of MEKL to handle large-scale problems.