Regularized Least Squares Classifier Research Articles

A Novel Doubly Reweighting Multisource Transfer Learning Framework

In this paper, to effectively use the decision knowledge from multiple source domains to predict the labels of samples in the target domain, a novel doubly reweighting multisource transfer learning called DRMTL framework is proposed. DRMTL aims to simultaneously optimize the structural risk function, domain reweighting adaptation, pointwise reweighting adaptation and manifold consistency. The merits of DRMTL include the following: 1) The importance of every source domain can be evaluated using the proposed novel flexible weighting index; 2) The loss between an unknown label prediction and its prediction by some source decision function for a target sample can be reweighted using a novel domain separator; and 3) The manifold structure of the target domain is effectively used in this framework. Finally, a specific learning algorithm, i.e., a doubly reweighting multisource transfer learning using the regularized least-squares classifier called DRM-RLS, is proposed using the DRMTL framework and the classical regularized least-squares classifier, and its convergence is also proven. Our experimental results from several real-world datasets reveal that the proposed approach outperforms several state-of-the-art transfer learning algorithms.

Object-based cloud detection of multitemporal high-resolution stationary satellite images

Satellite remote sensing that utilizes highly accurate cloud detection is important for monitoring natural disasters. The GaoFen-4, China’s first high-resolution stationary satellite, was recently launched and acquires imagery at a spatial resolution of 50 m and a high temporal resolution (up to 10 min). An object-based cloud detection method was conducted for a time series of GaoFen-4 images. The cloudy objects were obtained from the individual images, and the outlier detection of multiple temporal objects was further processed for refinement. In the initial cloud detection, the objects were segmented by the mean-shift algorithm, and their morphological features were extracted by extended attribute profiles. The threshold-detected cloudy objects were trained according to spectral and morphological features, and the initial objects were classified as cloudy or clear by a regularized least-squares classifier. Furthermore, the medians and standard deviations of the classified cloudy and clear objects were calculated and subsequently refined by the outlier detection of multiple temporal images. The clear object features deviated more than a multiple of standard deviations from the medians of the clear objects that were classified as cloudy objects. Additionally, the refined clear objects were obtained by a similar outlier detection method. Flood event monitoring using GaoFen-4 images showed that the average overall accuracy of the initial cloud detection was 83.4% and increased to 93.3% after refinement. This object-based cloud detection method was insensitive to variations in land objects and can effectively improve cloud detection within small or thin areas, which can be helpful for the monitoring of natural disasters.

Generalization improvement for regularized least squares classification

In the past decades, regularized least squares classification (RLSC) is a commonly used supervised classification method in the machine learning filed because it can be easily resolved through the simple matrix analysis and achieve a close-form solution. Recently, some studies conjecture that the margin distribution is more crucial to the generalization performance. Moreover, from the view of margin distribution, RLSC only considers the first-order statistics (i.e., margin mean) and does not consider the actual higher-order statistics of margin distribution. In this paper, we propose a novel RLSC which takes into account the actual second-order (i.e., variance) information of margin distribution. It is intuitively expected that small margin variance will improve the generalization performance of RLSC from a geometric view. We incorporate the margin variance into the objective function of RLSC and achieve the optimal classifier by minimizing the margin variance. To evaluate the performance of our algorithm, we conduct a series of experiments on several benchmark datasets in comparison with RLSC, kernel minimum squared error, support vector machine and large margin distribution machine. And the empirical results verify the effectiveness of our algorithm and indicate that the margin distribution is helpful to improve the classification performance.

A kernel matrix dimension reduction method for predicting drug-target interaction

The prediction of drug-target interactions plays an important role in the drug discovery process, which serves to identify new drugs or novel targets for existing drugs. However, experimental methods for predicting drug-target interactions are expensive and time-consuming. Therefore, the in silico prediction of drug-target interactions has recently attracted increasing attention. In this study, we proposed a kernel matrix dimension reduction method (KMDR) for predicting drug-target interactions, and in order to facilitate benchmark comparisons, two other representative algorithms, the Regularized Least Squares classifier (RLS) and the semi-supervised link prediction classifier (SLP), were also used to predict drug-target interactions on a same dataset. The results show that the kernel matrix reduction dimension method could improve the performance on drug-target interaction prediction; in particular, KMDR could significantly improve performance on low degree drug target interaction prediction. We further show that, in theory, the formulations of above three algorithms have a unified form, which could be seen as a kernel matrix transformation based on eigenvalue. This finding could provide us a research direction -- to design better algorithms for predicting drug-target interaction by optimize kernel matrix transformation based on eigenvalue.

Classification of Remotely Sensed Algal Blooms along the Coast of India using Support Vector Machines and Regularized Least Squares

Background/Objectives: The recent times have observed an inflation in frequency of occurrence of Algal Blooms (ABs). In this work, seven most commonly occurring species (i.e., Trichodesmium erythraeum, Noctiluca scintillans/miliaris, Cocholodinium ploykrikoides, Chattonella marina, Karenia mikimotoi and Protoperidinium species) that have contributed to major ABs along the coastline of India in the years 2002 to 2015 are classified. Methods/Statistical Analysis: Processing the data procured by MODIS Aqua sensor, classification of seven species of algae is performed based only on the feature of Remote Sensing Reflectance (R rs ). In contrast to the existing algorithms like band-ratio and interpretation of water discoloration, classification of blooms is based on Support Vector Machine (SVM) and Regularized Least Squares (RLS) algorithms. Findings: Classification is executed using LIBSVM and GURLS library for fast and efficient performance. The classification accuracies achieved using both the classifiers are comparable; the overall accuracy using SVM classifier is 88.37%, whereas that obtained with RLS classifier is 89.98%. Applications/Improvements: These results reveal that the above mentioned algorithms are capable of effectively detecting these ABs which is of immense interest in fisheries and healthcare industries. The algorithms can be further trained with bloom parameters based on in-situ datasets from additional occurrences.

Feature Selection for Ridge Regression with Provable Guarantees.

We introduce single-set spectral sparsification as a deterministic sampling-based feature selection technique for regularized least-squares classification, which is the classification analog to ridge regression. The method is unsupervised and gives worst-case guarantees of the generalization power of the classification function after feature selection with respect to the classification function obtained using all features. We also introduce leverage-score sampling as an unsupervised randomized feature selection method for ridge regression. We provide risk bounds for both single-set spectral sparsification and leverage-score sampling on ridge regression in the fixed design setting and show that the risk in the sampled space is comparable to the risk in the full-feature space. We perform experiments on synthetic and real-world data sets; a subset of TechTC-300 data sets, to support our theory. Experimental results indicate that the proposed methods perform better than the existing feature selection methods.

An eigenvalue transformation technique for predicting drug-target interaction.

The prediction of drug-target interactions is a key step in the drug discovery process, which serves to identify new drugs or novel targets for existing drugs. However, experimental methods for predicting drug-target interactions are expensive and time-consuming. Therefore, the in silico prediction of drug-target interactions has recently attracted increasing attention. In this study, we propose an eigenvalue transformation technique and apply this technique to two representative algorithms, the Regularized Least Squares classifier (RLS) and the semi-supervised link prediction classifier (SLP), that have been used to predict drug-target interaction. The results of computational experiments with these techniques show that algorithms including eigenvalue transformation achieved better performance on drug-target interaction prediction than did the original algorithms. These findings show that eigenvalue transformation is an efficient technique for improving the performance of methods for predicting drug-target interactions. We further show that, in theory, eigenvalue transformation can be viewed as a feature transformation on the kernel matrix. Accordingly, although we only apply this technique to two algorithms in the current study, eigenvalue transformation also has the potential to be applied to other algorithms based on kernels.

On Unsupervised Training of Multi-Class Regularized Least-Squares Classifiers

In this work we present the first efficient algorithm for unsupervised training of multi-class regularized least-squares classifiers. The approach is closely related to the unsupervised extension of the support vector machine classifier known as maximum margin clustering, which recently has received considerable attention, though mostly considering the binary classification case. We present a combinatorial search scheme that combines steepest descent strategies with powerful meta-heuristics for avoiding bad local optima. The regularized least-squares based formulation of the problem allows us to use matrix algebraic optimization enabling constant time checks for the intermediate candidate solutions during the search. Our experimental evaluation indicates the potential of the novel method and demonstrates its superior clustering performance over a variety of competing methods on real world datasets. Both time complexity analysis and experimental comparisons show that the method can scale well to practical sized problems.

Efficient recurrent local search strategies for semi- and unsupervised regularized least-squares classification

Binary classification tasks are among the most important ones in the field of machine learning. One prominent approach to address such tasks are support vector machines which aim at finding a hyperplane separating two classes well such that the induced distance between the hyperplane and the patterns is maximized. In general, sufficient labeled data is needed for such classification settings to obtain reasonable models. However, labeled data is often rare in real-world learning scenarios while unlabeled data can be obtained easily. For this reason, the concept of support vector machines has also been extended to semi- and unsupervised settings: in the unsupervised case, one aims at finding a partition of the data into two classes such that a subsequent application of a support vector machine leads to the best overall result. Similarly, given both a labeled and an unlabeled part, semi-supervised support vector machines favor decision hyperplanes that lie in a low density area induced by the unlabeled training patterns, while still considering the labeled part of the data. The associated optimization problems for both the semi- and unsupervised case, however, are of combinatorial nature and, hence, difficult to solve. In this work, we present efficient implementations of simple local search strategies for (variants of) the both cases that are based on matrix update schemes for the intermediate candidate solutions. We evaluate the performances of the resulting approaches on a variety of artificial and real-world data sets. The results indicate that our approaches can successfully incorporate unlabeled data. (The unsupervised case was originally proposed by Gieseke F, Pahikkala et al. (2009). The derivations presented in this work are new and comprehend the old ones (for the unsupervised setting) as a special case.)

Bioinformatic Challenges of Big Data in Non-Coding RNA Research

Bioinformatic Challenges of Big Data in Non-Coding RNA Research

A machine learning approach for the identification of odorant binding proteins from sequence-derived properties.

BackgroundOdorant binding proteins (OBPs) are believed to shuttle odorants from the environment to the underlying odorant receptors, for which they could potentially serve as odorant presenters. Although several sequence based search methods have been exploited for protein family prediction, less effort has been devoted to the prediction of OBPs from sequence data and this area is more challenging due to poor sequence identity between these proteins.ResultsIn this paper, we propose a new algorithm that uses Regularized Least Squares Classifier (RLSC) in conjunction with multiple physicochemical properties of amino acids to predict odorant-binding proteins. The algorithm was applied to the dataset derived from Pfam and GenDiS database and we obtained overall prediction accuracy of 97.7% (94.5% and 98.4% for positive and negative classes respectively).ConclusionOur study suggests that RLSC is potentially useful for predicting the odorant binding proteins from sequence-derived properties irrespective of sequence similarity. Our method predicts 92.8% of 56 odorant binding proteins non-homologous to any protein in the swissprot database and 97.1% of the 414 independent dataset proteins, suggesting the usefulness of RLSC method for facilitating the prediction of odorant binding proteins from sequence information.

Regularized Least Squares Cancer Classifiers from DNA microarray data

BackgroundThe advent of the technology of DNA microarrays constitutes an epochal change in the classification and discovery of different types of cancer because the information provided by DNA microarrays allows an approach to the problem of cancer analysis from a quantitative rather than qualitative point of view. Cancer classification requires well founded mathematical methods which are able to predict the status of new specimens with high significance levels starting from a limited number of data. In this paper we assess the performances of Regularized Least Squares (RLS) classifiers, originally proposed in regularization theory, by comparing them with Support Vector Machines (SVM), the state-of-the-art supervised learning technique for cancer classification by DNA microarray data. The performances of both approaches have been also investigated with respect to the number of selected genes and different gene selection strategies.ResultsWe show that RLS classifiers have performances comparable to those of SVM classifiers as the Leave-One-Out (LOO) error evaluated on three different data sets shows. The main advantage of RLS machines is that for solving a classification problem they use a linear system of order equal to either the number of features or the number of training examples. Moreover, RLS machines allow to get an exact measure of the LOO error with just one training.ConclusionRLS classifiers are a valuable alternative to SVM classifiers for the problem of cancer classification by gene expression data, due to their simplicity and low computational complexity. Moreover, RLS classifiers show generalization ability comparable to the ones of SVM classifiers also in the case the classification of new specimens involves very few gene expression levels.