Deep Multiple Kernel Learning Research Articles

A deep multiple kernel learning-based higher-order fuzzy inference system for identifying DNA N4-methylcytosine sites

N4-methylcytosine (4mC), as a DNA modification, plays a crucial role in epigenetic regulation. However, the existing experimentation methods for accurately identifying 4mC sites are inefficient and highly consumable, making them difficult to implement. Although a variety of new identification methods are continuously being proposed, existing techniques are not yet fully mature. Compared to traditional 4mC site predictors, based on support vector machine or convolutional neural network, we present an alternative computational approach. In this study, we propose a method based on a kernelized higher-order fuzzy inference system (KHFIS) and deep multiple kernel learning, called DMKL-HFIS, to improve the accuracy of 4mC site identification DNA sequences. We use PSTNP to process the benchmark datasets, and then apply KHFIS to obtain multiple fuzzy kernel matrices. A deep neural network is used to fuse multiple fuzzy kernel matrices. Finally, the predicted value is derived from the fused matrix. Our approach was compared with existing mainstream computational methods. On the benchmark datasets (G. subterraneus, D. melanogaster, E. coli, A. thaliana, and C. elegans), the accuracy of our approach exceeded that of a state-of-the-art method by 0.4%, 0.44%, 1.51%, 0.55%, and 0.25%, respectively. Compared to mainstream methods, our approach exhibits a higher level of accuracy and can therefore be considered an effective prediction tool.

Evolving Deep Multiple Kernel Learning Networks Through Genetic Algorithms

Today's Industrial Internet of Things (IIoT) have achieved excellent manufacturing efficiency and automation results by leveraging machine learning (ML) and deep learning (DL). However, trustworthiness of ML/DL brings significant challenges to IIoT. This article proposes an evolving deep multiple kernel learning network through genetic algorithm (KNGA). Our KNGA method uses genetic algorithm (GA) to find the best deep multiple kernel learning structure, including the weights and the topology of the model. Compared with the current well-known models, KNGA has advantages in three aspects: 1) It can achieve good results without using many samples during model training; 2) the model can evolve in the process of training, including self-growth, and self-pruning; and 3) its trustworthiness and reliability can be guaranteed. Moreover, the whole model ensures excellent performance and requires manual adjustment of only a few parameters. Extensive experiments on the UCI, KEEL, Caltech256, and MNIST datasets demonstrate the effectiveness and trustworthiness of the proposed method.

A multi-layer multi-kernel neural network for determining associations between non-coding RNAs and diseases

Identification of associations between non-coding RNAs and diseases plays an important role in the study of pathogenesis, which has been a hot topic in recent research. However, traditional methods are time-consuming to detect the associations between non-coding RNAs and diseases. Recently, associations of non-coding RNAs and diseases can be regarded as bipartite network. In this paper, we propose a novel deep multiple kernel learning method, called the multi-layer multi-kernel deep neural network (MLMKDNN). First, many feature matrices are built by multiple features of non-coding RNAs and diseases. Then, these feature matrices are mapped into kernel space and fused by deep neural network. Finally, combine two fused output of MLMKDNN as the predicted values. Three types of non-coding RNAs (miRNA, circRNA and lncRNA) are used to test the performance of MLMKDNN. Compared with other existing methods, our proposed model has high Area Under Precision Recall (AUPR) value on three types of datasets. Experimental results confirm that our method is an effective predictive tool. It provides a framework that can also be applied to the link prediction of other bipartite networks.

A predictive intelligence system of credit scoring based on deep multiple kernel learning

Banks face the task of improving the accuracy in predicting the behavior of individuals who utilize credit cards, as issuing cards to an appropriate applicant is considered an important matter. Credit card debt that is overdue by at least six months has seen a rapid increase in the past decade in the Chinese credit card industry. The problem of delinquency in credit cards not only affects the development of credit cards but it also influences the sustainability of banks. The use of credit risk assessment is critical in providing a solid foundation upon which the credit card issuer can appropriately approve an applicant for a credit card. In previous studies, a variety of machine learning methods have been proposed to assess credit risk. However, conventional methods are viewed as shallow models and are not good at representing compositional features. Thus, this study applies a deep multiple kernel classifier as a state-of-the-art technique, which is proficient in coping with deep structure and complex data in credit risk assessment. It will support decision-makers issuing credit cards in China appropriately. The results indicate that deep multiple kernel classifier outperforms conventional and ensemble models. Credit card departments with better risk management can avoid possible bad debt, hence benefiting banks’ operations. The applications of predictive intelligence enhance the prediction of human behavior in the credit card industry.

Deep Multiple Kernel Learning for Prediction of MicroRNA Precursors

MicroRNAs are a group of noncoding RNAs that are about 20–24 nucleotides in length. They are involved in the physiological processes of many diseases and regulate transcriptional and post-transcriptional gene expression. Therefore, the prediction of microRNAs is of great significance for basic biological research and disease treatment. MicroRNA precursors are the necessary stage of microRNA formation. RBF kernel support vector machines (RBF-SVMs) and shallow multiple kernel support vector machines (MK-SVMs) are often used in microRNA precursors prediction. However, the RBF-SVMs could not represent the richer sample features, and the MK-SVMs just use a simply convex combination of few base kernels. This paper proposed a localized multiple kernel learning model with a nonlinear synthetic kernel (LMKL-D). The nonlinear synthetic kernel was trained by a three-layer deep multiple kernel learning model. The LMKL-D model was tested on 2241 pre-microRNAs and 8494 pseudo hairpin sequences. The experiments showed that the LMKL-D model achieved 93.06% sensitivity, 99.27% specificity, and 98.03% accuracy on the test set. The results showed that the LMKL-D model can increase the complexity of kernels and better predict microRNA precursors. Our LMKL-D model can better predict microRNA precursors compared with the existing methods in specificity and accuracy. The LMKL-D model provides a reference for further validation of potential microRNA precursors.

Deep kernel supervised hashing for node classification in structural networks

Node classification in structural networks has been proven to be useful in many real world applications. With the development of network embedding, the performance of node classification has been greatly improved. However, nearly all the existing network embedding based methods are hard to capture the actual category features of a node because of the linearly inseparable problem in low-dimensional space; meanwhile they cannot incorporate simultaneously network structure information and node label information into network embedding. To address the above problems, in this paper, we propose a novel Deep Kernel Supervised Hashing (DKSH) method to learn the hashing representations of nodes for node classification. Specifically, a deep multiple kernel learning is first proposed to map nodes into suitable Hilbert space to deal with linearly inseparable problem. Then, instead of only considering structural similarity between two nodes, a novel similarity matrix is designed to merge both network structure information and node label information. Supervised by the similarity matrix, the learned hashing representations of nodes simultaneously preserve the two kinds of information well from the learned Hilbert space. Extensive experiments show that the proposed method significantly outperforms the state-of-the-art baselines over three real world benchmark datasets.

Group-based local adaptive deep multiple kernel learning with lp norm.

The deep multiple kernel Learning (DMKL) method has attracted wide attention due to its better classification performance than shallow multiple kernel learning. However, the existing DMKL methods are hard to find suitable global model parameters to improve classification accuracy in numerous datasets and do not take into account inter-class correlation and intra-class diversity. In this paper, we present a group-based local adaptive deep multiple kernel learning (GLDMKL) method with lp norm. Our GLDMKL method can divide samples into multiple groups according to the multiple kernel k-means clustering algorithm. The learning process in each well-grouped local space is exactly adaptive deep multiple kernel learning. And our structure is adaptive, so there is no fixed number of layers. The learning model in each group is trained independently, so the number of layers of the learning model maybe different. In each local space, adapting the model by optimizing the SVM model parameter α and the local kernel weight β in turn and changing the proportion of the base kernel of the combined kernel in each layer by the local kernel weight, and the local kernel weight is constrained by the lp norm to avoid the sparsity of basic kernel. The hyperparameters of the kernel are optimized by the grid search method. Experiments on UCI and Caltech 256 datasets demonstrate that the proposed method is more accurate in classification accuracy than other deep multiple kernel learning methods, especially for datasets with relatively complex data.

DWS-MKL: Depth-width-scaling multiple kernel learning for data classification

Deep learning technologies have been rapidly developing recently. They have shown excellent performances in many fields. However, deep learning networks have weak adaptability to small sample sizes.Usually, deep learning networks require tens of thousands of samples to avoid overfitting. In contrast, the kernel method can deal with small sample sizes. In this paper, we use only dozens of samples to train a successful classifier. Deep multiple kernel learning (DMKL) has emerged in recent years. It combines the ideas of deep learning and multiple kernel learning. Unfortunately, the DMKL network with a fixed structure cannot adapt to data of different dimensions and sizes. Therefore, in this paper, we propose a novel depth-width-scaling multiple kernel learning (DWS-MKL) algorithm. It has the ability to adjust the architecture according to the input data. We optimize the estimation of leave-one-out error by using the span bound instead of the dual objective function. Finally, we choose a large number of data sets from the UCI benchmark data sets for classification tasks. We also evaluate DWS-MKL on the MNIST data set. The experimental results show that different frameworks have different performances on the same dataset. Our DWS-MKL algorithm obtains better classification results than the state-of-the-art kernel learning algorithms, as determined by a thorough comparison. The encouraging results demonstrate that our method has the potential to improve generalization performance of the model.

A Generic Design of Driver Drowsiness and Stress Recognition Using MOGA Optimized Deep MKL-SVM.

Driver drowsiness and stress are major causes of traffic deaths and injuries, which ultimately wreak havoc on world economic loss. Researchers are in full swing to develop various algorithms for both drowsiness and stress recognition. In contrast to existing works, this paper proposes a generic model using multiple-objective genetic algorithm optimized deep multiple kernel learning support vector machine that is capable to recognize both driver drowsiness and stress. This algorithm simplifies the research formulations and model complexity that one model fits two applications. Results reveal that the proposed algorithm achieves an average sensitivity of 99%, specificity of 98.3% and area under the receiver operating characteristic curve (AUC) of 97.1% for driver drowsiness recognition. For driver stress recognition, the best performance is yielded with average sensitivity of 98.7%, specificity of 98.4% and AUC of 96.9%. Analysis also indicates that the proposed algorithm using multiple-objective genetic algorithm has better performance compared to the grid search method. Multiple kernel learning enhances the performance significantly compared to single typical kernel. Compared with existing works, the proposed algorithm not only achieves higher accuracy but also addressing the typical issues of dataset in simulated environment, no cross-validation and unreliable measurement stability of input signals.

Self-Adaptive Deep Multiple Kernel Learning Based on Rademacher Complexity

The deep multiple kernel learning (DMKL) method has caused widespread concern due to its better results compared with shallow multiple kernel learning. However, existing DMKL methods, which have a fixed number of layers and fixed type of kernels, have poor ability to adapt to different data sets and are difficult to find suitable model parameters to improve the test accuracy. In this paper, we propose a self-adaptive deep multiple kernel learning (SA-DMKL) method. Our SA-DMKL method can adapt the model through optimizing the model parameters of each kernel function with a grid search method and change the numbers and types of kernel function in each layer according to the generalization bound that is evaluated with Rademacher chaos complexity. Experiments on the three datasets of University of California—Irvine (UCI) and image dataset Caltech 256 validate the effectiveness of the proposed method on three aspects.

Energy Commodity Price Forecasting with Deep Multiple Kernel Learning

Oil is an important energy commodity. The difficulties of forecasting oil prices stem from the nonlinearity and non-stationarity of their dynamics. However, the oil prices are closely correlated with global financial markets and economic conditions, which provides us with sufficient information to predict them. Traditional models are linear and parametric, and are not very effective in predicting oil prices. To address these problems, this study developed a new strategy. Deep (or hierarchical) multiple kernel learning (DMKL) was used to predict the oil price time series. Traditional methods from statistics and machine learning usually involve shallow models; however, they are unable to fully represent complex, compositional, and hierarchical data features. This explains why traditional methods fail to track oil price dynamics. This study aimed to solve this problem by combining deep learning and multiple kernel machines using information from oil, gold, and currency markets. DMKL is good at exploiting multiple information sources. It can effectively identify the relevant information and simultaneously select an apposite data representation. The kernels of DMKL were embedded in a directed acyclic graph (DAG), which is a deep model and efficient at representing complex and compositional data features. This provided a solid foundation for extracting the key features of oil price dynamics. By using real data for empirical testing, our new system robustly outperformed traditional models and significantly reduced the forecasting errors.

New Deep Kernel Learning based Models for Image Classification

Deep learning system is used for solving many problems in different domains but it gives an over-fitting risk when richer representations are increased. In this paper, three different models with different deep multiple kernel learning architectures are proposed and evaluated for the breast cancer classification problem. Discrete Wavelet transform and edge histogram descriptor are used to extract the image features. For image classification purpose, support vector machine with the proposed deep multiple kernel models are used. Also, the span bound is employed for optimizing these models over the dual objective function. Furthermore, the comparison between the performance of the traditional support vector machine which uses only single kernel and the introduced models is worked out that show the efficiency of the experimental results of the proposed models.