Multiple Kernel Learning Algorithm Research Articles

A Novel Multi-modal Sentiment Analysis Based on Multiple Kernel Learning with Margin-Dimension Constraint

Most of the sentiment analysis studies focus on the sentimental classification of pictures in the video, ignoring the spatio-temporal information of the sequence of picture frames as well as text and audio information. The multiple kernel learning is a new hotspot in the field of nuclear machine learning, capable of handling multiple modalities. For multiple kernel learning, it is easy to ignore the basic features that are not discriminative, and cannot make full use of the base features of different modes. This paper puts forward a novel multi-modal fusion model for sentiment analysis, in which a multiple kernel learning algorithm based on convolution margin-dimension constraint is proposed for feature fusion. Moreover, the 3D convolutional neural network is used to extract the features of visual information, and the multiple kernel learning algorithm based on margin-dimension constraint is used to fuse visual, text and audio sentiment features. Experiments conducted on the MOUD and IEMOCAP sentiment databases show that the proposed model outperforms existing models in the field of multi-modal sentiment analysis research.

A novel graph-based multiple kernel learning framework for hyperspectral image classification

ABSTRACT Multiple kernel learning (MKL) is an efficient way to improve hyperspectral image classification with few training samples by integrating spectral and spatial features. Nonetheless, presenting a MKL algorithm to capture significant information and eliminate redundancy is indispensable. In this paper, a novel graph-based MKL method (GMKL) is introduced to enhance the performance of the MKL algorithm and achieve high discriminative features, resulting in maximum separability. To do so, an informative low-rank graph is determined via a new algorithm called block diagonal representation that takes into account both local and global data structures. Then, in lieu of a time-consuming search for finding the optimal combination of the basic kernels, unlike the prior works, an optimal projective direction is acquired using graph theory. Various basic kernels generated from superpixels and attribute profiles are projected to produce a discriminative combination of kernels. Eventually, the optimum kernel is given to the support vector machine classifier. Experiments have been conducted on two popular hyperspectral datasets (Indian Pines and Pavia University). For the datasets, the proposed algorithm gained 90.43% and 99.28% overall accuracy, respectively, using only 1% training samples, which is much better than the state-of-the-art MKL algorithm and deep learning-based approaches with lower computational time.

Neural Generalization of Multiple Kernel Learning

Multiple Kernel Learning (MKL) is a conventional way to learn the kernel function in kernel-based methods. MKL algorithms enhance the performance of kernel methods. However, these methods have a lower complexity compared to deep models and are inferior to them regarding recognition accuracy. Deep learning models can learn complex functions by applying nonlinear transformations to data through several layers. In this paper, we show that a typical MKL algorithm can be interpreted as a one-layer neural network with linear activation functions. By this interpretation, we propose a Neural Generalization of Multiple Kernel Learning (NGMKL), which extends the conventional MKL framework to a multi-layer neural network with nonlinear activation functions. Our experiments show that the proposed method, which has a higher complexity than traditional MKL methods, leads to higher recognition accuracy on several benchmarks.

Multi-Kernel Graph Attention Deep Autoencoder for MiRNA-Disease Association Prediction.

Accumulating evidence indicates that microRNAs (miRNAs) can control and coordinate various biological processes. Consequently, abnormal expressions of miRNAs have been linked to various complex diseases. Recognizable proof of miRNA-disease associations (MDAs) will contribute to the diagnosis and treatment of human diseases. Nevertheless, traditional experimental verification of MDAs is laborious and limited to small-scale. Therefore, it is necessary to develop reliable and effective computational methods to predict novel MDAs. In this work, a multi-kernel graph attention deep autoencoder (MGADAE) method is proposed to predict potential MDAs. In detail, MGADAE first employs the multiple kernel learning (MKL) algorithm to construct an integrated miRNA similarity and disease similarity, providing more biological information for further feature learning. Second, MGADAE combines the known MDAs, disease similarity, and miRNA similarity into a heterogeneous network, then learns the representations of miRNAs and diseases through graph convolution operation. After that, an attention mechanism is introduced into MGADAE to integrate the representations from multiple graph convolutional network (GCN) layers. Lastly, the integrated representations of miRNAs and diseases are input into the bilinear decoder to obtain the final predicted association scores. Corresponding experiments prove that the proposed method outperforms existing advanced approaches in MDA prediction. Furthermore, case studies related to two human cancers provide further confirmation of the reliability of MGADAE in practice.

MSGCA: Drug-Disease Associations Prediction Based on Multi-Similarities Graph Convolutional Autoencoder.

Identifying drug-disease associations (DDAs) is critical to the development of drugs. Traditional methods to determine DDAs are expensive and inefficient. Therefore, it is imperative to develop more accurate and effective methods for DDAs prediction. Most current DDAs prediction methods utilize original DDAs matrix directly. However, the original DDAs matrix is sparse, which greatly affects the prediction consequences. Hence, a prediction method based on multi-similarities graph convolutional autoencoder (MSGCA) is proposed for DDAs prediction. First, MSGCA integrates multiple drug similarities and disease similarities using centered kernel alignment-based multiple kernel learning (CKA-MKL) algorithm to form new drug similarity and disease similarity, respectively. Second, the new drug and disease similarities are improved by linear neighborhood, and the DDAs matrix is reconstructed by weighted K nearest neighbor profiles. Next, the reconstructed DDAs and the improved drug and disease similarities are integrated into a heterogeneous network. Finally, the graph convolutional autoencoder with attention mechanism is utilized to predict DDAs. Compared with extant methods, MSGCA shows superior results on three datasets. Furthermore, case studies further demonstrate the reliability of MSGCA.

Predicting Suicide Attempts among Major Depressive Disorder Patients with Structural Neuroimaging: A Machine Learning Approach

IntroductionEvery year at least one million people die by suicide, with major depressive disorder (MDD) being one of the major causes of suicide deaths. Current suicide risk assessments rely on subjective information, are time consuming, low predictive, and poorly reliable. Thus, finding objective biomarkers of suicidality is crucial to move clinical practice towards a precision psychiatry framework, enhancing suicide risk detection and prevention for MDD.ObjectivesThe present study aimed at applying machine learning (ML) algorithms on both grey matter and white-matter voxel-wise data to discriminate MDD suicide attempters (SA) from non-attempters (nSA).Methods91 currently depressed MDD patients (24 SA, 67 nSA) underwent a structural MRI session. T1-weighted images and diffusion tensor imaging scans were respectively pre-processed using Computational Atlas Toolbox 12 (CAT12) and spatial tract-based statistics (TBSS) on FSL, to obtain both voxel-based morphometry (VBM) and fractional anisotropy (FA) measures. Three classification models were built, entering whole-brain VBM and FA maps alone into a Support Vector Machine (SVM) and combining both modalities into a Multiple Kernel Learning (MKL) algorithm. All models were trained through a 5-fold nested cross-validation with subsampling to calculate reliable estimates of balanced accuracy, specificity, sensitivity, and area under the receiver operator curve (AUC).ResultsModels’ performances are summarized in Table 1.Table 1.Models’ performances.Input featuresAlgorithmSpecificitySensitivityBalanced accuracyAUCVBMSVM55.00%50.00%52.50%0.55FASVM72.00%54.00%63.00%0.62VBM and FAMKL68.00%54.00%61.00%0.58Abbreviations: AUC, area under the receiver operator curve; FA, fractional anisotropy; VBM, voxel-based morphometry.ConclusionsOverall, although overcoming the random classification accuracy (i.e., 50%), performances of all models classifying SA and nSA MDD patients were moderate, possibly due to the imbalanced numerosity of classes, with SVM on FA reaching the highest accuracy. Thus, future studies may enlarge the sample and add different features (e.g., functional neuroimaging data) to develop an objective and reliable predictive model to assess and hence prevent suicide risk among MDD patients.Disclosure of InterestNone Declared

A radius-incorporated localized multiple kernel learning algorithm for detecting depression in speech

A radius-incorporated localized multiple kernel learning algorithm for detecting depression in speech

One-Class Classification Using ℓp-Norm Multiple Kernel Fisher Null Approach.

We address the one-class classification (OCC) problem and advocate a one-class MKL (multiple kernel learning) approach for this purpose. To this aim, based on the Fisher null-space OCC principle, we present a multiple kernel learning algorithm where an ℓp-norm regularisation (p ≥ 1) is considered for kernel weight learning. We cast the proposed one-class MKL problem as a min-max saddle point Lagrangian optimisation task and propose an efficient approach to optimise it. An extension of the proposed approach is also considered where several related one-class MKL tasks are learned concurrently by constraining them to share common weights for kernels. An extensive evaluation of the proposed MKL approach on a range of data sets from different application domains confirms its merits against the baseline and several other algorithms.

Identification of drug-side effect association via restricted Boltzmann machines with penalized term.

In the entire life cycle of drug development, the side effect is one of the major failure factors. Severe side effects of drugs that go undetected until the post-marketing stage leads to around two million patient morbidities every year in the United States. Therefore, there is an urgent need for a method to predict side effects of approved drugs and new drugs. Following this need, we present a new predictor for finding side effects of drugs. Firstly, multiple similarity matrices are constructed based on the association profile feature and drug chemical structure information. Secondly, these similarity matrices are integrated by Centered Kernel Alignment-based Multiple Kernel Learning algorithm. Then, Weighted K nearest known neighbors is utilized to complement the adjacency matrix. Next, we construct Restricted Boltzmann machines (RBM) in drug space and side effect space, respectively, and apply a penalized maximum likelihood approach to train model. At last, the average decision rule was adopted to integrate predictions from RBMs. Comparison results and case studies demonstrate, with four benchmark datasets, that our method can give a more accurate and reliable prediction result.

EEG-based Emotion Recognition Using Multiple Kernel Learning

Emotion recognition based on electroencephalography (EEG) has a wide range of applications and has great potential value, so it has received increasing attention from academia and industry in recent years. Meanwhile, multiple kernel learning (MKL) has also been favored by researchers for its data-driven convenience and high accuracy. However, there is little research on MKL in EEG-based emotion recognition. Therefore, this paper is dedicated to exploring the application of MKL methods in the field of EEG emotion recognition and promoting the application of MKL methods in EEG emotion recognition. Thus, we proposed a support vector machine (SVM) classifier based on the MKL algorithm EasyMKL to investigate the feasibility of MKL algorithms in EEG-based emotion recognition problems. We designed two data partition methods, random division to verify the validity of the MKL method and sequential division to simulate practical applications. Then, tri-categorization experiments were performed for neutral, negative and positive emotions based on a commonly used dataset, the Shanghai Jiao Tong University emotional EEG dataset (SEED). The average classification accuracies for random division and sequential division were 92.25% and 74.37%, respectively, which shows better classification performance than the traditional single kernel SVM. The final results show that the MKL method is obviously effective, and the application of MKL in EEG emotion recognition is worthy of further study. Through the analysis of the experimental results, we discovered that the simple mathematical operations of the features on the symmetrical electrodes could not effectively integrate the spatial information of the EEG signals to obtain better performance. It is also confirmed that higher frequency band information is more correlated with emotional state and contributes more to emotion recognition. In summary, this paper explores research on MKL methods in the field of EEG emotion recognition and provides a new way of thinking for EEG-based emotion recognition research.

Myocardial fibrosis detection using kernel methods: preliminary results from a cardiac magnetic resonance study

Abstract Funding Acknowledgements Type of funding sources: None. Background Asserting the presence of myocardial fibrosis from cardiac magnetic resonance (CMR) images is sometimes a complex task, even for experienced cardiac imagers. The application of artificial intelligence models to the evaluation process can be of help for enhancing diagnostic accuracy. Purpose In this work, we tested two different Machine Learning (ML) algorithms, namely kernel methods with Support Vector Machine (SVM) and Convolutional Neural Network (CNN) to a cohort of consecutive CMR studies. The goal was a binary classification task, aimed to identify myocardial scar (present/absent). Methods Dataset consisted of 642 CMR scans, equally divided into healthy and fibrosis-affected. Raw DICOM files were preprocessed through an automated pipeline, in order to retrieve only short-axis post contrast acquisitions. Heart regions were then individuated using a YOLO network, in order to remove the background and focus only on data of interest. Finally, for each subject 10 slices were extracted through interpolation, and all images resized to 128 by 128 pixels. Dataset was divided into training and test sets, with proportions 80%-20%. Results The first analysis was based on state-of-the-art CNN models, pre-trained on the ImageNet dataset. The training of the models was optimized using "reduce learning rate on the plateau", "early stopping", and standard data augmentation techniques. Experiments showed that CNNs-based models led to poor performances. The second attempt was based on kernel methods and SVM. Before feeding the input to the algorithm, dimensionality reduction was implemented using a Principal Component Analysis retaining 99% of the variance. The resulting 335 features were passed as input to an SVM, testing different kernels (e.g. Linear, Gaussian, Cossim). Models were trained and optimized using a Grid Search with a 5-fold Cross-Validation. The best SVM configuration displayed an accuracy of 68% and a sensitivity of 60%. Improved results could be obtained using state-of-the-art Multiple Kernel Learning algorithms, leading to 71% accuracy and sensitivity of 72% (with a 12% increment with respect to the previous algorithm). Conclusions Our preliminary results of this study showed the possibility for a ML system to learn to identify myocardial fibrosis, directly from raw CMR images. In particular, kernel methods were able to achieve good and significant results, even with small amounts of data, suggesting room for improvement. In future works, we plan to furtherly explore kernel methods to improve the results and to build an end-to-end solution for cardiac imagers.

Machine Learning Techniques for the Diagnosis of Schizophrenia Based on Event-Related Potentials.

AntecedentThe event-related potential (ERP) components P300 and mismatch negativity (MMN) have been linked to cognitive deficits in patients with schizophrenia. The diagnosis of schizophrenia could be improved by applying machine learning procedures to these objective neurophysiological biomarkers. Several studies have attempted to achieve this goal, but no study has examined Multiple Kernel Learning (MKL) classifiers. This algorithm finds optimally a combination of kernel functions, integrating them in a meaningful manner, and thus could improve diagnosis.ObjectiveThis study aimed to examine the efficacy of the MKL classifier and the Boruta feature selection method for schizophrenia patients (SZ) and healthy controls (HC) single-subject classification.MethodsA cohort of 54 SZ and 54 HC participants were studied. Three sets of features related to ERP signals were calculated as follows: peak related features, peak to peak related features, and signal related features. The Boruta algorithm was used to evaluate the impact of feature selection on classification performance. An MKL algorithm was applied to address schizophrenia detection.ResultsA classification accuracy of 83% using the whole dataset, and 86% after applying Boruta feature selection was obtained. The variables that contributed most to the classification were mainly related to the latency and amplitude of the auditory P300 paradigm.ConclusionThis study showed that MKL can be useful in distinguishing between schizophrenic patients and controls when using ERP measures. Moreover, the use of the Boruta algorithm provides an improvement in classification accuracy and computational cost.

Fast and interpretable genomic data analysis using multiple approximate kernel learning.

MotivationDataset sizes in computational biology have been increased drastically with the help of improved data collection tools and increasing size of patient cohorts. Previous kernel-based machine learning algorithms proposed for increased interpretability started to fail with large sample sizes, owing to their lack of scalability. To overcome this problem, we proposed a fast and efficient multiple kernel learning (MKL) algorithm to be particularly used with large-scale data that integrates kernel approximation and group Lasso formulations into a conjoint model. Our method extracts significant and meaningful information from the genomic data while conjointly learning a model for out-of-sample prediction. It is scalable with increasing sample size by approximating instead of calculating distinct kernel matrices.ResultsTo test our computational framework, namely, Multiple Approximate Kernel Learning (MAKL), we demonstrated our experiments on three cancer datasets and showed that MAKL is capable to outperform the baseline algorithm while using only a small fraction of the input features. We also reported selection frequencies of approximated kernel matrices associated with feature subsets (i.e. gene sets/pathways), which helps to see their relevance for the given classification task. Our fast and interpretable MKL algorithm producing sparse solutions is promising for computational biology applications considering its scalability and highly correlated structure of genomic datasets, and it can be used to discover new biomarkers and new therapeutic guidelines.Availability and implementationMAKL is available at https://github.com/begumbektas/makl together with the scripts that replicate the reported experiments. MAKL is also available as an R package at https://cran.r-project.org/web/packages/MAKL.Supplementary information Supplementary data are available at Bioinformatics online.

A bi-level formulation for multiple kernel learning via self-paced training

Multiple kernel learning (MKL) is a crucial issue which has been widely researched over the last two decades. Although existing MKL algorithms have achieved satisfactory performance in a broad range of applications, these methods do not adequately consider the adverse effects of unreliable or less reliable instances. To handle this shortcoming, we formulate multiple kernel learning in a bi-level learning paradigm consisting of the kernel combination weight learning (KWL) stage and the self-paced learning (SPL) stage, which alternatively negotiate with each other. The KWL stage dynamically absorbs reliable instances into model learning to accurately capture neighborhood relationships and obtains kernel coefficients via maximizing both global and local kernel alignment in a common schema. The SPL stage automatically evaluates the reliability of training samples via self-paced training. The extensive experiments indicate the robustness and superiority of the presented approach in comparison with existing MKL methods.

Localized multiple kernel learning using graph modularity

• A graph-based heuristic approach for multiple kernel learning (MKL). • Assigns sample-specific kernel weights based on contribution to graph modularity . • Classification performance comparable to state-of-the-art MKL algorithms. • Outperforms other heuristic approaches and has low time complexity. Multiple kernel learning (MKL) algorithms exploit information from multiple feature representations by assigning weights to each representation in the kernel space, and later combining them. However, this ignores the fact that data points exhibit locally varying characteristics. To address this problem, localized MKL algorithms learn locality-specific kernel weights which determine each base kernel’s influence in the locality under consideration. Here, we relate the problem of determining the relevance of base kernels for classification to that of quantifying community structure in graphs. Next, we derive sample-specific kernel weights using graph modularity. Through experiments on publicly available datasets, we show that the proposed approach offers a viable alternative to state-of-the-art MKL approaches while being computationally inexpensive.

FTWSVM-SR: DNA-Binding Proteins Identification via Fuzzy Twin Support Vector Machines on Self-Representation.

Due to the high cost of DNA-binding proteins (DBPs) detection, many machine learning algorithms (ML) have been utilized to large-scale process and detect DBPs. The previous methods took no count of the processing of noise samples. In this study, a fuzzy twin support vector machine (FTWSVM) is employed to detect DBPs. First, multiple types of protein sequence features are formed into kernel matrices; Then, multiple kernel learning (MKL) algorithm is utilized to linear combine multiple kernels; next, self-representation-based membership function is utilized to estimate membership value (weight) of each training sample; finally, we feed the integrated kernel matrix and membership values into the FTWSVM-SR model for training and testing. On comparison with other predictive models, FTWSVM based on SR (FTWSVM-SR) obtains the best performance of Matthew's correlation coefficient (MCC): 0.7410 and 0.5909 on two independent testing sets (PDB186 and PDB2272 datasets), respectively. The results confirm that our method can be an effective DBPs detection tool. Before the biochemical experiment, our model can screen and analyze DBPs on a large scale.

PrognosiT: Pathway/gene set-based tumour volume prediction using multiple kernel learning

BackgroundIdentification of molecular mechanisms that determine tumour progression in cancer patients is a prerequisite for developing new disease treatment guidelines. Even though the predictive performance of current machine learning models is promising, extracting significant and meaningful knowledge from the data simultaneously during the learning process is a difficult task considering the high-dimensional and highly correlated nature of genomic datasets. Thus, there is a need for models that not only predict tumour volume from gene expression data of patients but also use prior information coming from pathway/gene sets during the learning process, to distinguish molecular mechanisms which play crucial role in tumour progression and therefore, disease prognosis.ResultsIn this study, instead of initially choosing several pathways/gene sets from an available set and training a model on this previously chosen subset of genomic features, we built a novel machine learning algorithm, PrognosiT, that accomplishes both tasks together. We tested our algorithm on thyroid carcinoma patients using gene expression profiles and cancer-specific pathways/gene sets. Predictive performance of our novel multiple kernel learning algorithm (PrognosiT) was comparable or even better than random forest (RF) and support vector regression (SVR). It is also notable that, to predict tumour volume, PrognosiT used gene expression features less than one-tenth of what RF and SVR algorithms used.ConclusionsPrognosiT was able to obtain comparable or even better predictive performance than SVR and RF. Moreover, we demonstrated that during the learning process, our algorithm managed to extract relevant and meaningful pathway/gene sets information related to the studied cancer type, which provides insights about its progression and aggressiveness. We also compared gene expressions of the selected genes by our algorithm in tumour and normal tissues, and we then discussed up- and down-regulated genes selected by our algorithm while learning, which could be beneficial for determining new biomarkers.

Efficient Multitask Multiple Kernel Learning With Application to Cancer Research.

Multitask multiple kernel learning (MKL) algorithms combine the capabilities of incorporating different data sources into the prediction model and using the data from one task to improve the accuracy on others. However, these methods do not necessarily produce interpretable results. Restricting the solutions to the set of interpretable solutions increases the computational burden of the learning problem significantly, leading to computationally prohibitive run times for some important biomedical applications. That is why we propose a multitask MKL formulation with a clustering of tasks and develop a highly time-efficient solution approach for it. Our solution method is based on the Benders decomposition and treating the clustering problem as finding a given number of tree structures in a graph; hence, it is called the forest formulation. We use our method to discriminate early-stage and late-stage cancers using genomic data and gene sets and compare our algorithm against two other algorithms. The two other algorithms are based on different approaches for linearization of the problem while all algorithms make use of the cutting-plane method. Our results indicate that as the number of tasks and/or the number of desired clusters increase, the forest formulation becomes increasingly favorable in terms of computational performance.

Brainstem glucose metabolism predicts reward dependence scores in treatment-resistant major depression.

It has been suggested that individual differences in temperament could be involved in the (non-)response to antidepressant (AD) treatment. However, how neurobiological processes such as brain glucose metabolism may relate to personality features in the treatment-resistant depressed (TRD) state remains largely unclear. To examine how brainstem metabolism in the TRD state may predict Cloninger's temperament dimensions Harm Avoidance (HA), Novelty Seeking (NS), and Reward Dependence (RD), we collected 18fluorodeoxyglucose positron emission tomography (18FDG PET) scans in 40 AD-free TRD patients. All participants were assessed with the Temperament and Character Inventory (TCI). We applied a multiple kernel learning (MKL) regression to predict the HA, NS, and RD from brainstem metabolic activity, the origin of respectively serotonergic, dopaminergic, and noradrenergic neurotransmitter (NT) systems. The MKL model was able to significantly predict RD but not HA and NS from the brainstem metabolic activity. The MKL pattern regression model identified increased metabolic activity in the pontine nuclei and locus coeruleus, the medial reticular formation, the dorsal/median raphe, and the ventral tegmental area that contributed to the predictions of RD. The MKL algorithm identified a likely metabolic marker in the brainstem for RD in major depression. Although 18FDG PET does not investigate specific NT systems, the predictive value of brainstem glucose metabolism on RD scores however indicates that this temperament dimension in the TRD state could be mediated by different monoaminergic systems, all involved in higher order reward-related behavior.

A New Convolutional Kernel Classifier for Hyperspectral Image Classification

Multiple Kernel Learning (MKL) algorithms are among the most successful classification methods for hyperspectral data. Nevertheless, these algorithms suffer from two main drawbacks of computational complexity and debility to admit to the end-to-end learning paradigm. This paper proposed a Convolutional Kernel Classifier (CKC) for hyperspectral remote sensing images to address these issues. The CKC uses the Nystrm approximation method to estimate a low-rank approximation of the basis kernels, thus solves the issues associated with the high dimensionality of the basis kernels. The CKC uses deep architecture to learn the optimal combination of the basis kernels and the classification task to enable end-to-end learning. The proposed CKCs architecture is based on a 1D-Convolutional Neural Network (CNN-1D), and it uses kernel dropout to prevent overfitting. It is the first instance of deep-kernel algorithms in the field of remote sensing. The proposed method was compared with several well-known hyperspectral image analysis MKL algorithms, including a multi-kernel variant of the deep kernel machine optimization (M-DKMO), MKL-average, Simple-MKL, and Generalize MKL (GMKL), and state-of-the-art deep learning models, including Vanilla Recurrent Neural Network (VanillaRNN) and CNN-1D in classifying four benchmark hyperspectral datasets. The experimental results show that the CKC consistently outperforms all the competitor methods, and its runtime is lower than its MKL algorithm counterparts on four benchmark hyperspectral datasets. Moreover, the Nystrm approximation solves the high dimensionality of the basis kernels and boosts classification accuracy. The source codes of CKC are available from: https://github.com/MohsenAnsari1373/A-New-Convolutional-Kernel-Classifier-for-Hyperspectral-Image-Classification .