Optimizing the Gaussian kernel function with the formulated kernel target alignment criterion for two-class pattern classification

Abstract

Nowadays most of the current kernel learning approaches are showing good results in small datasets and fail to scale to large ones. As such, it is necessary to develop faster kernel optimization algorithms that perform better with larger datasets, especially, for the “Big Data” applications. This paper presents a novel fast method to optimize the Gaussian kernel function for two-class pattern classification tasks, where it is desirable for the kernel machines to use an optimized kernel that adapts well to the input data and the learning tasks. We propose to optimize the Gaussian kernel function by using the formulated kernel target alignment criterion. By adopting the Euler–Maclaurin formula and the local and global extremal properties of the approximate kernel separability criterion, the approximate criterion function can be proved to have a determined global minimum point. Thus, when the approximate criterion function is a sufficient approximation of the criterion function, through using a Newton-based algorithm, the proposed optimization is simply solved without being repeated the searching procedure with different starting points to locate the best local minimum. The proposed method is evaluated on thirteen data sets with three Gaussian-kernel-based learning algorithms. The experimental results show that the criterion function has the determined global minimum point for the all thirteen data sets, the proposed method achieves the best high time efficiency performance and the best overall classification performance.