Kernel-based Machine Learning Methods Research Articles

Biological Sequence Classification with Multivariate String Kernels

String kernel-based machine learning methods have yielded great success in practical tasks of structured/sequential data analysis. They often exhibit state-of-the-art performance on many practical tasks of sequence analysis such as biological sequence classification, remote homology detection, or protein superfamily and fold prediction. However, typical string kernel methods rely on the analysis of discrete 1D string data (e.g., DNA or amino acid sequences). In this paper, we address the multiclass biological sequence classification problems using multivariate representations in the form of sequences of features vectors (as in biological sequence profiles, or sequences of individual amino acid physicochemical descriptors) and a class of multivariate string kernels that exploit these representations. On three protein sequence classification tasks, the proposed multivariate representations and kernels show significant 15-20 percent improvements compared to existing state-of-the-art sequence classification methods.

A novel partial least squares weighting Gaussian process algorithm and its application to near infrared spectroscopy data mining problems

A new partial least squares (PLS) weighting Gaussian process (PWGP) algorithm is proposed to improve the regression performance of Gaussian process (GP), an outstanding kernel-based machine learning method, on high dimensional data with small sample size, especially near-infrared (NIR) spectroscopy. Important indexes of original variables are firstly calculated according to their contributions to the PLS regression model. After being weighted by these important indexes, new values of the observations are input into GP algorithm for further regression analysis. Relying on the PLS based weighting technique, important variables could be highlighted by their relatively large index values. Consequently “information saturation” phenomenon could be successfully overcome. Most importantly, unlike other weighting methods, there is no need to have prior knowledge in order to optimize any factors or parameters, thus the PWGP method is especially suitable for regression problems of “black-box” systems. Applications of the proposed method on three NIR spectroscopy dataset, which are widely used as test data, strongly confirmed that the predictive performance of PWGP is superior to other approaches.

Kernel learning for ligand-based virtual screening: discovery of a new PPARγ agonist

We demonstrate the theoretical and practical application of modern kernel-based machine learning methods to ligand-based virtual screening by successful prospective screening for novel agonists of the peroxisome proliferator-activated receptor γ (PPARγ) [1]. PPARγ is a nuclear receptor involved in lipid and glucose metabolism, and related to type-2 diabetes and dyslipidemia. Applied methods included a graph designed for molecular similarity analysis [2], principle component analysis [3], multiple learning [4], and, Gaussian process regression [5]. In the machine learning approach to ligand-based virtual screening, one uses the similarity principle [6] to identify potentially active compounds based on their similarity to known reference ligands. Kernel-based machine learning [7] uses the kernel trick, a systematic approach to the derivation of non-linear versions of linear algorithms like separating hyperplanes and regression. Prerequisites for learning are similarity measures with the mathematical property of positive semidefiniteness (kernels). The iterative similarity optimal assignment graph (ISOAK) [2] is defined directly on the annotated structure graph, and was designed specifically for the comparison of small molecules. In our virtual screening study, its use improved results, e.g., in principle component analysis-based visualization and Gaussian process regression. Following a thorough retrospective validation using a data set of 176 published PPARγ agonists [8], we screened a vendor library for novel agonists. Subsequent testing of 15 compounds in a cell-based transactivation assay [9] yielded four active compounds. The most interesting hit, a natural product derivative with cyclobutane scaffold, is a full selective PPARγ agonist (EC50 = 10 ± 0.2 μM, inactive on PPARα and PPARβ/δ at 10 μM). We demonstrate how the interplay of several modern kernel-based machine learning approaches can successfully improve ligand-based virtual screening results.

From Machine Learning to Natural Product Derivatives that Selectively Activate Transcription Factor PPARγ

Advanced kernel-based machine learning methods enable the identification of innovative bioactive compounds with minimal experimental effort. Comparative virtual screening revealed that nonlinear models of the underlying structure–activity relationship are necessary for successful compound picking. In a proof-of-concept study a novel truxillic acid derivative was found to selectively activate transcription factor PPARγ. Detailed facts of importance to specialist readers are published as ”Supporting Information”. Such documents are peer-reviewed, but not copy-edited or typeset. They are made available as submitted by the authors. Please note: The publisher is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.

Gaussian Processes for Short-Term Traffic Volume Forecasting

The accurate modeling and forecasting of traffic flow data such as volume and travel time are critical to intelligent transportation systems. Many forecasting models have been developed for this purpose since the 1970s. Recently kernel-based machine learning methods such as support vector machines (SVMs) have gained special attention in traffic flow modeling and other time series analyses because of their outstanding generalization capability and superior nonlinear approximation. In this study, a novel kernel-based machine learning method, the Gaussian processes (GPs) model, was proposed to perform short-term traffic flow forecasting. This GP model was evaluated and compared with SVMs and autoregressive integrated moving average (ARIMA) models based on four sets of traffic volume data collected from three interstate highways in Seattle, Washington. The comparative results showed that the GP and SVM models consistently outperformed the ARIMA model. This study also showed that because the GP model is formulated in a full Bayesian framework, it can allow for explicit probabilistic interpretation of forecasting outputs. This capacity gives the GP an advantage over SVMs to model and forecast traffic flow.

Kernel-based machine learning for fast text mining in R

Recent advances in the field of kernel-based machine learning methods allow fast processing of text using string kernels utilizing suffix arrays. kernlab provides both kernel methods’ infrastructure and a large collection of already implemented algorithms and includes an implementation of suffix-array-based string kernels. Along with the use of the text mining infrastructure provided by tm these packages provide R with functionality in processing, visualizing and grouping large collections of text data using kernel methods. The emphasis is on the performance of various types of string kernels at these tasks.

Protein–protein interaction extraction by leveraging multiple kernels and parsers

Protein–protein interaction (PPI) extraction is an important and widely researched task in the biomedical natural language processing (BioNLP) field. Kernel-based machine learning methods have been used widely to extract PPI automatically, and several kernels focusing on different parts of sentence structure have been published for the PPI task. In this paper, we propose a method to combine kernels based on several syntactic parsers, in order to retrieve the widest possible range of important information from a given sentence. We evaluate the method using a support vector machine (SVM), and we achieve better results than other state-of-the-art PPI systems on four out of five corpora. Further, we analyze the compatibility of the five corpora from the viewpoint of PPI extraction, and we see that some of them have small incompatibilities, but they can still be combined with a little effort.

Kernel-based online machine learning and support vector reduction

We apply kernel-based machine learning methods to online learning situations, and look at the related requirement of reducing the complexity of the learnt classifier. Online methods are particularly useful in situations which involve streaming data, such as medical or financial applications. We show that the concept of span of support vectors can be used to build a classifier that performs reasonably well while satisfying given space and time constraints, thus making it potentially suitable for such online situations. The span-based heuristic is observed to be effective under stringent memory limits (that is when the number of support vectors a machine can hold is very small).

Mapping of environmental data using kernel-based methods

Recently, kernel-based Machine Learning methods have gained great popularity in many data analysis and data mining fields: pattern recognition, biocomputing, speech and vision, engineering, remote sensing etc. The paper describes the use of kernel methods to approach the processing of large datasets from environmental monitoring networks. Several typical problems of the environmental sciences and their solutions provided by kernel-based methods are considered: classification of categorical data (soil type classification), mapping of environmental and pollution continuous information (pollution of soil by radionuclides), mapping with auxiliary information (climatic data from Aral Sea region). The promising developments, such as automatic emergency hot spot detection and monitoring network optimization are discussed as well.

Kernlab- AnS4Package for Kernel Methods inR

kernlab is an extensible package for kernel-based machine learning methods in R. It takes advantage of R's new S4 ob ject model and provides a framework for creating and using kernel-based algorithms. The package contains dot product primitives (kernels), implementations of support vector machines and the relevance vector machine, Gaussian processes, a ranking algorithm, kernel PCA, kernel CCA, and a spectral clustering algorithm. Moreover it provides a general purpose quadratic programming solver, and an incomplete Cholesky decomposition method.