Deep Kernel machines: a survey

Abstract

The emergence of deep learning frameworks paves the way for achieving higher-level data abstractions and possess the potential in consolidating both supervised and unsupervised learning paradigms. Researchers have made many successful explorations in the field of deep learning, with applications in the fields of face recognition, text mining, language translation, image prediction, and action recognition. Kernel machines act as a bridge between the linearity and nonlinearity for many machine learning algorithms such as support vector machines, extreme learning machines, and core vector machines. These Kernel machines play a vital role in mapping the data in the input space to a Kernel-induced high-dimensional feature space to obtain a better distribution of the data. In this Kernel-induced high-dimensional feature space, the distribution of data points will be more amenable to the classification problem under consideration. The Kernel trick facilitates in transforming the machine learning algorithms that require only inner product computations between the data vectors into a Kernel-based approach by selecting an appropriate Kernel function. In Kernel-based approaches, the Kernel functions can thus be utilized for accomplishing the inner product computations between the transformed data vectors in an implicitly defined Kernel-induced feature space. Unlike neural networks, the Kernel machines guarantee structural risk minimization and global optimal solutions. Also, the Kernel machines exhibit capabilities such as theoretical tractability and excellent performance in practical applications. These attempts motivated the researchers towards utilizing the emerging trends of deep learning with Kernel methods for building deep Kernel machines. Researchers integrate Kernel methods and deep learning networks for maintaining their advantages and make up their limitations, then apply the deep Kernel learning approaches for improving the performance of the learning algorithm in different applications. Different ways of building deep Kernel machines by integrating the Kernel methods and deep learning architectures include utilizing Kernel machines as the final classifier of deep learning networks, Kernelization in deep neural networks for better feature enrichment, and building deep Kernel machines by utilizing deep or multiple Kernels in different tasks. This survey attempts to provide an overview of different approaches in building several deep Kernel learning architectures for enhancing the learning algorithm properties and their performance in practical applications.