Multiple Kernel Learning Approach Research Articles

Color object classification using multi-channel Zernike moments-based rotation invariant bag-of-visual-words and deep convolutional neural networks

We propose two rotation-invariant approaches for color object classification and recognition using multi-channel Zernike moments (MZMs)-based bag-of-visual-words (BoVWs) and deep convolutional neural networks (DCNN). The first approach, referred to as MZMs-BoVWs, derives the BoVWs descriptors from a color image and its sub-images after forming its hierarchical sub-divisions and computing the MZMs from each of the sub-images to determine similarity among sub-images to facilitate the construction of BoVWs. The second approach, called DCNN-MZMs, derives the MZMs descriptors from the feature maps of a convolutional layer of a DCNN architecture designed in this paper. A fusion of the descriptors, MZMs-BoVWs and CNN-MZMs, along with the high-performing color histograms (CH) descriptor, i.e., CH+MZMs-BoVWs+CNN-MZMs, is also proposed. The process of the object classification is performed using SVM and its multiple-kernel learning approach. Experimental results show that the proposed approaches outperform the existing state-of-the-art approaches under image-rotation and under limited data for training the CNNs.

One-Class Classification Using ℓp-Norm Multiple Kernel Fisher Null Approach.

We address the one-class classification (OCC) problem and advocate a one-class MKL (multiple kernel learning) approach for this purpose. To this aim, based on the Fisher null-space OCC principle, we present a multiple kernel learning algorithm where an ℓp-norm regularisation (p ≥ 1) is considered for kernel weight learning. We cast the proposed one-class MKL problem as a min-max saddle point Lagrangian optimisation task and propose an efficient approach to optimise it. An extension of the proposed approach is also considered where several related one-class MKL tasks are learned concurrently by constraining them to share common weights for kernels. An extensive evaluation of the proposed MKL approach on a range of data sets from different application domains confirms its merits against the baseline and several other algorithms.

Multiple kernel fusion: A novel approach for lake water depth modeling

Multiple kernel fusion (MKF) refers to the task of combining multiple sources of information in the Hilbert space for improved performance. Very often the combined kernel is formed as a linear composition of multiple base kernels where the combination weights are learned from the data. As the first application of an MKF approach in hydrological modeling, lake water depth as one of the pivot factors in the reservoir analysis is simulated by considering different hydro-meteorological variables. The role of each individual input parameter is initially investigated by applying a kernel regression approach. We then illustrate the utility of an MKF formalism which learns kernel combination of weights to yield an optimal composition for kernel regression. A set of 40-year data collected from 27 groundwater and streamflow stations and 7 meteorological stations for precipitation and evaporation parameters in the vicinity of Lake Urmia are utilized for model development. Both visual and quantitative statistical performance criteria illustrate a superior performance for the MKF approach compared to kernel ridge regression (KRR), the support vector regression (SVR), back propagation neural network (BPNN) and auto regressive (AR) models. More specifically, while each individual input parameter fails to provide an accurate prediction for lake water depth modeling, an optimal combination of all input parameters incorporating the groundwater level, streamflow, precipitation and evaporation via a multiple kernel learning approach enhances the predictive performance of the model accuracy in the multiple scenarios. The promising results (RMSE = 0.098 m; R2 = 0.987; NSE = 0.986) may motivate the application of a MKF approach towards solving alternative and complex hydrological problems.

Multiple kernel learning for label relation and class imbalance in multi-label learning

There are two common challenges in multi-label learning (MLL), complex label relation and imbalanced class. Few studies have focused on addressing both problems at the same time. In this paper, we propose a multiple kernel learning (MKL) approach to tackle two challenges, named as Multi-Kernel Multi-Label (MKML) method. MKML contains three kernel modules. The first kernel module adopts the traditional kernel function which contains the global information, and the other two kernel modules are designed for the two problems respectively. The second kernel module learns the inter-label relation by adding supervised information. The third kernel module adjusts the imbalanced decision boundary through multi-layer fusion strategy, which is proved to improve the representation ability of kernels in this paper. Finally, the proposed joint optimization method in this MKL framework achieves good generalization ability. We conduct several related experiments using real-world datasets to evaluate the effectiveness of our method. The results demonstrate that MKML outperforms other state-of-the art methods in MLL task.

Bayesian sparse factor analysis with kernelized observations

Multi-view problems can benefit from latent representations since they find low-dimensional projections that fairly capture the correlations among the multiple views that characterise the data. On the other hand, high-dimensionality and non-linear issues are traditionally handled by kernel methods, inducing a (non)-linear function between the latent projection and the data itself. However, they usually come with exposition to overfitting. Here, we combine Bayesian factor analysis with what we refer to as kernelized observations, in which the proposed model focuses on reconstructing not the data itself, but its relationship with other data points measured by a kernel function. In turn, we extend previous Bayesian FA formulations to be able to model non-linear data relationships by means of kernelized data representations and, at the same time, include additional facilities to obtain compact kernel representations by means of an automatic selection of Bayesian Relevance Vectors (RVs), feature relevance analysis and, even, obtain an automatic multiple kernel learning approach. Besides, this is flexibly included into a modular framework where we can easily adapt the model capabilities to the data needs and, even, combine them with previous FA functionalities such as heterogeneous data representations or semi-supervised learning. Using several public databases, we demonstrate the potential of the approach (and its extensions) w.r.t. common multi-view learning models such as kernel canonical correlation analysis, heterogeneous incomplete – variational autoenconder or manifold relevance determination, where the proposed model shows its ability to outperform the baselines while indistinctly combining model extensions.

Localized multiple kernel learning using graph modularity

• A graph-based heuristic approach for multiple kernel learning (MKL). • Assigns sample-specific kernel weights based on contribution to graph modularity . • Classification performance comparable to state-of-the-art MKL algorithms. • Outperforms other heuristic approaches and has low time complexity. Multiple kernel learning (MKL) algorithms exploit information from multiple feature representations by assigning weights to each representation in the kernel space, and later combining them. However, this ignores the fact that data points exhibit locally varying characteristics. To address this problem, localized MKL algorithms learn locality-specific kernel weights which determine each base kernel’s influence in the locality under consideration. Here, we relate the problem of determining the relevance of base kernels for classification to that of quantifying community structure in graphs. Next, we derive sample-specific kernel weights using graph modularity. Through experiments on publicly available datasets, we show that the proposed approach offers a viable alternative to state-of-the-art MKL approaches while being computationally inexpensive.

Brain oscillatory correlates of visuomotor adaptive learning

Sensorimotor adaptation involves the recalibration of the mapping between motor command and sensory feedback in response to movement errors. Although adaptation operates within individual movements on a trial-to-trial basis, it can also undergo learning when adaptive responses improve over the course of many trials. Brain oscillatory activities related to these “adaptation” and “learning” processes remain unclear. The main reason for this is that previous studies principally focused on the beta band, which confined the outcome message to trial-to-trial adaptation. To provide a wider understanding of adaptive learning, we decoded visuomotor tasks with constant, random or no perturbation from EEG recordings in different bandwidths and brain regions using a multiple kernel learning approach. These different experimental tasks were intended to separate trial-to-trial adaptation from the formation of the new visuomotor mapping across trials. We found changes in EEG power in the post-movement period during the course of the visuomotor-constant rotation task, in particular an increased (i) theta power in prefrontal region, (ii) beta power in supplementary motor area, and (iii) gamma power in motor regions. Classifying the visuomotor task with constant rotation versus those with random or no rotation, we were able to relate power changes in beta band mainly to trial-to-trial adaptation to error while changes in theta band would relate rather to the learning of the new mapping. Altogether, this suggested that there is a tight relationship between modulation of the synchronization of low (theta) and higher (essentially beta) frequency oscillations in prefrontal and sensorimotor regions, respectively, and adaptive learning.

MKL-GRNI: A parallel multiple kernel learning approach for supervised inference of large-scale gene regulatory networks.

High throughput multi-omics data generation coupled with heterogeneous genomic data fusion are defining new ways to build computational inference models. These models are scalable and can support very large genome sizes with the added advantage of exploiting additional biological knowledge from the integration framework. However, the limitation with such an arrangement is the huge computational cost involved when learning from very large datasets in a sequential execution environment. To overcome this issue, we present a multiple kernel learning (MKL) based gene regulatory network (GRN) inference approach wherein multiple heterogeneous datasets are fused using MKL paradigm. We formulate the GRN learning problem as a supervised classification problem, whereby genes regulated by a specific transcription factor are separated from other non-regulated genes. A parallel execution architecture is devised to learn a large scale GRN by decomposing the initial classification problem into a number of subproblems that run as multiple processes on a multi-processor machine. We evaluate the approach in terms of increased speedup and inference potential using genomic data from Escherichia coli, Saccharomyces cerevisiae and Homo sapiens. The results thus obtained demonstrate that the proposed method exhibits better classification accuracy and enhanced speedup compared to other state-of-the-art methods while learning large scale GRNs from multiple and heterogeneous datasets.

Integration of Drug Perturbed Dose Dependent Response Data in Predictive Computational Models of Chemotherapeutic Response Using Multiple‐Kernel‐Learning

Recently there has been a great deal of work to understand the dynamics of genome expression under small molecule perturbation. This has been demonstrated by the recent expansion of the Connectivity Map (CMAP) ( www.broadinstitute.org/connectivity‐map‐cmap) with roughly 5000 small molecule signatures. While this has proven to be invaluable resource it is still limited due to many drugs being constrained to a handful of cancer cell genomes, because of this, it is difficult to leverage these datasets to make an accurate and robust predictive model. Outside CMAP there is limited data that systematically characterize small molecule induced genomic changes, however, recently Monks et al. (Cancer Res 2018 DEC 15) has made available data in the NCI60 cell line panel for treated and untreated cell lines at 2, 6, and 24hr and two different dosing concentrations for 15 anti‐cancer agents. In our work we explore how each dataset can be optimally leveraged to build a predictive model of cell IC50 within each of the 15 drugs using a multiple kernel learning approach. With this method we can reach correlations as high as 0.88 (p ~1e−6) between predicted and measured IC50 with an average correlation of 0.57 on an independent test set. Additionally, perturbation with higher drug dose yields better individual models with average correlation around 0.451 compared to non‐treated data of 0.391 suggesting drug induced genomic changes are an important predictor of drug performance. Furthermore, we present a novel method that derives signatures present in perturbed data that can be captured in basal expression data. This approach to selecting features yields model performance that is equal or better than a signature derived basally alone. Our work demonstrates the importance of both basal and dynamic gene expression in modeling and understanding of drug response. We hope this work motivates further experiments to characterize drug induced gene dynamics for both modeling applications and understanding the underlying genomic influences in drug response.Support or Funding InformationShipley Chair in Comparative Oncology

P.305 Inflammation and neuroimaging differentiate bipolar and unipolar depression: a multiple kernel learning approach

P.305 Inflammation and neuroimaging differentiate bipolar and unipolar depression: a multiple kernel learning approach

Analyzing Group-Level Emotion with Global Alignment Kernel based Approach

From the perspective of social science, understanding group emotion has become increasingly important for teams to considerably accomplish organizational work. Currently, automatically analyzing the perceived affect of a group of people has been received increasingly interest in affective computing community. The variability in group size makes difficulty for group-level emotion recognition to straightforwardly measure the feature distance of two group-level images. Recent works attempted to resolve the preceding problem by using feature encoding. However, the early works lack of efficiency. To alleviate this problem, this article aims to design a new method to effectively analyze the group behavior from a group-level image. Motivated by time-series kernel approaches explored in dynamic facial expression classification, this article mainly concentrates on global alignment kernel and design support vector machine with the combined global alignment kernels (SVM-CGAK) to better recognize group-level emotion. Specifically, we first propose to use global alignment kernel to explicitly measure the distance of two group-level images. For improving the performance of global alignment kernel, we use the global weight sort scheme based on their spatial relation information to sort the faces from group-level image, making an efficient data structure to the global alignment kernel. With this new global alignment kernel, we construct the backbone of SVM-CGAK, namely, support vector machine with global alignment kernel. Furthermore, considering the challenging environment, we construct two global alignment kernels based on Reisz-based Volume Local Binary Pattern and deep convolutional neural network features, respectively. Lastly, to make the robustness of group-level emotion recognition, we propose SVM-CGAK combining both global alignment kernels with multiple kernel learning approach. It can enhance the discriminative ability of each global alignment kernel. Intensive experiments are conducted on three challenging group-level emotion databases. The experimental results demonstrate that the proposed approach achieves promising performance for group-level emotion recognition compared with the recent state-of-the-art methods.

Effects of Compression on Remote Sensing Image Classification Based on Fractal Analysis

Image compression is essential for remote sensing due to the large volume of produced remote sensing imagery and system’s limited transmission or storage capacity. As one of the most important applications, classification might be affected due to the introduced distortion during compression. Hence, we perform a quantitative study on the effects of compression on remote sensing image classification and propose a method to estimate the remote sensing image classification accuracy based on fractal analysis. Multiscale feature extraction is performed and a multiple kernel learning approach is proposed accordingly. The experimental results on our established database indicate that the classification accuracy predicted by our method exhibits high consistency with the ground truth and our method shows its superiority when compared with other classical reference algorithms.

Learning Distance Metric for Support Vector Machine: A Multiple Kernel Learning Approach

Recent work in distance metric learning has significantly improved the performance in k-nearest neighbor classification. However, the learned metric with these methods cannot adapt to the support vector machines (SVM), which are amongst the most popular classification algorithms using distance metrics to compare samples. In order to investigate the possibility to develop a novel model for joint learning distance metric and kernel classifier, in this paper, we provide a new parameterization scheme for incorporating the squared Mahalanobis distance into the Gaussian RBF kernel, and formulate kernel learning into a generalized multiple kernel learning framework, gearing towards SVM classification. We demonstrate the effectiveness of the proposed algorithm on the UCI machine learning datasets of varying sizes and difficulties and two real-world datasets. Experimental results show that the proposed model achieves competitive classification accuracies and comparable execution time by using spectral projected gradient descent optimizer compared with state-of-the-art methods.

Inner Product Optimization for Effective Multiple Kernel Learning

This paper proposes a novel and effective method for multiple kernel learning (MKL) that differs from standard MKL approaches. The new algorithm, named effective multiple kernel learning (EMKL), proposes a learn function space generated by multiple kernels with a group of parameters, as well as constructs a new inner-product with an optimized symmetric positive-definite matrix for the solution space of learning problem. We also describe the proposed EMKL algorithm within a general-purpose image classification framework. The EMKL algorithm has a closed-form solution, thus enabling optimization without the need for the additional iterative operations that are common in standard MKL algorithms. Hence, our proposed EMKL has a lower computational cost than that of conventional MKL techniques. The results of experiments on two benchmark datasets show the effectiveness of the proposed algorithm.

Geometrically invariant color, shape and texture features for object recognition using multiple kernel learning classification approach

The geometrically invariant image descriptors are very important for recognizing objects of arbitrary shapes and orientations. We propose a framework for the fusion of the geometrically invariant descriptors representing color, shape, and texture for the recognition of color objects using multiple kernel learning (MKL) approach. To describe texture of color images, we propose an effective rotation invariant texture descriptor which is based on the Zernike moments (ZMs) of the gradient of the color images, referred to as the GZMs. For the shape features, we use the ZMs of the intensity component of a color image and also use multi-channel ZMs (MZMs) which have proven to be superior in performance than the quaternion ZMs (QZMs). For comparative performance analysis, rotation invariants of the QZMs (RQZMs) are also considered. Since the color histograms (CH) are known to be very effective color descriptors, we consider them for representing color. The five sets of features – CH, ZMs, GZMs, MZMs, and RQZMs are invariant to translation, rotation, and scale. The fusion of color, shape and texture features in different combinations using the MKL approach is shown to provide very high recognition rates on PASCAL VOC 2005, Soccer, SIMPLIcity, Flower, and Caltech-101 datasets.

Localized Multiple Kernel learning for Anomaly Detection: One-class Classification

Multi-kernel learning has been well explored in the recent past and has exhibited promising outcomes for multi-class classification and regression tasks. In this paper, we present a multiple kernel learning approach for the One-class Classification (OCC) task and employ it for anomaly detection. Recently, the basic multi-kernel approach has been proposed to solve the OCC problem, which is simply a convex combination of different kernels with equal weights. This paper proposes a Localized Multiple Kernel learning approach for Anomaly Detection (LMKAD) using OCC, where the weight for each kernel is assigned locally. Proposed LMKAD approach adapts the weight for each kernel using a gating function. The parameters of the gating function and one-class classifier are optimized simultaneously through a two-step optimization process. We present the empirical results of the performance of LMKAD on 25 benchmark datasets from various disciplines. This performance is evaluated against existing Multi Kernel Anomaly Detection (MKAD) algorithm, and four other existing kernel-based one-class classifiers to showcase the credibility of our approach. LMKAD achieves significantly better Gmean scores while using a lesser number of support vectors compared to MKAD. Friedman test is also performed to verify the statistical significance of the results claimed in this paper.

Pain-Related Fear-Dissociable Neural Sources of Different Fear Constructs.

Fear of pain demonstrates significant prognostic value regarding the development of persistent musculoskeletal pain and disability. Its assessment often relies on self-report measures of pain-related fear by a variety of questionnaires. However, based either on “fear of movement/(re)injury/kinesiophobia,” “fear avoidance beliefs,” or “pain anxiety,” pain-related fear constructs plausibly differ while it is unclear how specific the questionnaires are in assessing these different constructs. Furthermore, the relationship of pain-related fear to other anxiety measures such as state or trait anxiety remains ambiguous. Advances in neuroimaging such as machine learning on brain activity patterns recorded by functional magnetic resonance imaging might help to dissect commonalities or differences across pain-related fear constructs. We applied a pattern regression approach in 20 human patients with nonspecific chronic low back pain to reveal predictive relationships between fear-related neural pattern information and different pain-related fear questionnaires. More specifically, the applied multiple kernel learning approach allowed the generation of models to predict the questionnaire scores based on a hierarchical ranking of fear-related neural patterns induced by viewing videos of activities potentially harmful for the back. We sought to find evidence for or against overlapping pain-related fear constructs by comparing the questionnaire prediction models according to their predictive abilities and associated neural contributors. By demonstrating evidence of nonoverlapping neural predictors within fear-processing regions, the results underpin the diversity of pain-related fear constructs. This neuroscientific approach might ultimately help to further understand and dissect psychological pain-related fear constructs.

Energy Sustainability in Smart Cities: Artificial Intelligence, Smart Monitoring, and Optimization of Energy Consumption

Energy sustainability is one of the key questions that drive the debate on cities’ and urban areas development. In parallel, artificial intelligence and cognitive computing have emerged as catalysts in the process aimed at designing and optimizing smart services’ supply and utilization in urban space. The latter are paramount in the domain of energy provision and consumption. This paper offers an insight into pilot systems and prototypes that showcase in which ways artificial intelligence can offer critical support in the process of attaining energy sustainability in smart cities. To this end, this paper examines smart metering and non-intrusive load monitoring (NILM) to make a case for the latter’s value added in context of profiling electric appliances’ electricity consumption. By employing the findings in context of smart cities research, the paper then adds to the debate on energy sustainability in urban space. Existing research tends to be limited by data granularity (not in high frequency) and consideration of about six kinds of appliances. In this paper, a hybrid genetic algorithm support vector machine multiple kernel learning approach (GA-SVM-MKL) is proposed for NILM, with consideration of 20 kinds of appliance. Genetic algorithm helps to solve the multi-objective optimization problem and design the optimal kernel function based on various kernel properties. The performance indicators are sensitivity (Se), specificity (Sp) and overall accuracy (OA) of the classifier. First, the performance evaluation of proposed GA-SVM-MKL achieves Se of 92.1%, Sp of 91.5% and OA of 91.8%. Second, the percentage improvement of performance indicators using proposed method is more than 21% compared with traditional kernel. Third, results reveal that by keeping different modes of electric appliance as identical class label, the performance indicators can increase to about 15%. Forth, tunable modes of GA-SVM-MKL classifier are proposed to further enhance the performance indicators up to 7%. Overall, this paper is a bold and novel contribution to the debate on energy utilization and sustainability in urban spaces as it integrates insights from artificial intelligence, IoT, and big data analytics and queries them in a context defined by energy sustainability in smart cities.

A Multiple Kernel Learning Approach for Air Quality Prediction

Air quality prediction is an important research issue due to the increasing impact of air pollution on the urban environment. However, existing methods often fail to forecast high-polluting air conditions, which is precisely what should be highlighted. In this paper, a novel multiple kernel learning (MKL) model that embodies the characteristics of ensemble learning, kernel learning, and representative learning is proposed to forecast the near future air quality (AQ). The centered alignment approach is used for learning kernels, and a boosting approach is used to determine the proper number of kernels. To demonstrate the performance of the proposed MKL model, its performance is compared to that of classical autoregressive integrated moving average (ARIMA) model; widely used parametric models like random forest (RF) and support vector machine (SVM); popular neural network models like multiple layer perceptron (MLP); and long short-term memory neural network. Datasets acquired from a coastal city Hong Kong and an inland city Beijing are used to train and validate all the models. Experiments show that the MKL model outperforms the other models. Moreover, the MKL model has better forecast ability for high health risk category AQ.

Scuba: scalable kernel-based gene prioritization

BackgroundThe uncovering of genes linked to human diseases is a pressing challenge in molecular biology and precision medicine. This task is often hindered by the large number of candidate genes and by the heterogeneity of the available information. Computational methods for the prioritization of candidate genes can help to cope with these problems. In particular, kernel-based methods are a powerful resource for the integration of heterogeneous biological knowledge, however, their practical implementation is often precluded by their limited scalability.ResultsWe propose Scuba, a scalable kernel-based method for gene prioritization. It implements a novel multiple kernel learning approach, based on a semi-supervised perspective and on the optimization of the margin distribution. Scuba is optimized to cope with strongly unbalanced settings where known disease genes are few and large scale predictions are required. Importantly, it is able to efficiently deal both with a large amount of candidate genes and with an arbitrary number of data sources. As a direct consequence of scalability, Scuba integrates also a new efficient strategy to select optimal kernel parameters for each data source. We performed cross-validation experiments and simulated a realistic usage setting, showing that Scuba outperforms a wide range of state-of-the-art methods.ConclusionsScuba achieves state-of-the-art performance and has enhanced scalability compared to existing kernel-based approaches for genomic data. This method can be useful to prioritize candidate genes, particularly when their number is large or when input data is highly heterogeneous. The code is freely available at https://github.com/gzampieri/Scuba.