Multi-task Support Vector Machine Research Articles

Multi-task Support Vector Machine Classifier with Generalized Huber Loss

Multi-task Support Vector Machine Classifier with Generalized Huber Loss

Machine learning prediction of malaria vaccine efficacy based on antibody profiles.

Immunization through repeated direct venous inoculation of Plasmodium falciparum (Pf) sporozoites (PfSPZ) under chloroquine chemoprophylaxis, using the PfSPZ Chemoprophylaxis Vaccine (PfSPZ-CVac), induces high-level protection against controlled human malaria infection (CHMI). Humoral and cellular immunity contribute to vaccine efficacy but only limited information about the implicated Pf-specific antigens is available. Here, we examined Pf-specific antibody profiles, measured by protein arrays representing the full Pf proteome, of 40 placebo- and PfSPZ-immunized malaria-naïve volunteers from an earlier published PfSPZ-CVac dose-escalation trial. For this purpose, we both utilized and adapted supervised machine learning methods to identify predictive antibody profiles at two different time points: after immunization and before CHMI. We developed an adapted multitask support vector machine (SVM) approach and compared it to standard methods, i.e. single-task SVM, regularized logistic regression and random forests. Our results show, that the multitask SVM approach improved the classification performance to discriminate the protection status based on the underlying antibody-profiles while combining time- and dose-dependent data in the prediction model. Additionally, we developed the new fEature diStance exPlainabilitY (ESPY) method to quantify the impact of single antigens on the non-linear multitask SVM model and make it more interpretable. In conclusion, our multitask SVM model outperforms the studied standard approaches in regard of classification performance. Moreover, with our new explanation method ESPY, we were able to interpret the impact of Pf-specific antigen antibody responses that predict sterile protective immunity against CHMI after immunization. The identified Pf-specific antigens may contribute to a better understanding of immunity against human malaria and may foster vaccine development.

Semi-supervised Multi-task Learning with Auxiliary data

Compared with single-task learning, multi-tasks can obtain better classifiers by the information provided by each task. In the process of multi-task data collection, we always focus on the target task data in the training process, and ignore the non-target task data and unlabeled data that may be contained in the target task. In response to this issue, this paper introduces auxiliary or Universum into semi-supervised multi-task problem, and proposes a multi-task support vector machine (SU-MTLSVM) method based on semi-supervised learning to handle the case where each task contains the labeled, unlabeled, and Universum samples in the training set. This method introduces Universum as prior knowledge and provides high-dimensional information for semi-supervised learning, and builds a unique classifier from a large amount of unlabeled data. We then use KKT conditions and Lagrangian method to optimize the formulation of the model, and get the model parameters. Finally, we collect different data sets in the experiment part, and compare the performance of multiple baselines with the proposed method. Experiments prove that the method proposed in this paper is more effective for multi-task applications.

Multi-task support vector machine with pinball loss

With the boom in machine learning, support vector machine (SVM) is widely employed in pattern recognition. However, most of SVM models concentrate on single-task learning, multi-task learning has been largely neglected. Compared with single-task learning, multi-task learning can achieve a good performance for each task by mining the shared information among tasks. In addition, loss function also plays an important role in the accuracy of SVM. Inspired by multi-task learning and the SVM with pinball loss (pin-SVM), we propose two novel multi-task support vector machines with pinball loss for binary classification, named as MTL-pin-SVM I and MTL-pin-SVM II. Both methods maximize the quantile distance for each task, which realizes less sensitive to noise and more stable for re-sampling. Moreover, MTL-pin-SVM II can use different combinations of kernel functions for different tasks, which can get better performance than other multi-task models by choosing the suitable combinations of kernel functions for different tasks. And they include the multi-task SVM with hinge loss as their special cases, which are denoted as MTL-C-SVM I and MTL-C-SVM II. The extensive experiments on multi-task datasets fully validate the validity of the proposed models.

A new multi-task learning method with universum data

Multi-task learning (MTL) obtains a better classifier than single-task learning (STL) by sharing information between tasks within the multi-task models. Most existing multi-task learning models only focus on the data of the target tasks during training, and ignore the data of non-target tasks that may be contained in the target tasks. In this way, Universum data can be added to classifier training as prior knowledge, and these data do not belong to any indicated categories. In this paper, we address the problem of multi-task learning with Universum data, which improves utilization of non-target task data. We introduce Universum learning to make non-target task data act as prior knowledge and propose a novel multi-task support vector machine with Universum data (U-MTLSVM). Based on the characteristics of MTL, each task have corresponding Universum data to provide prior knowledge. We then utilize the Lagrange method to solve the optimization problem so as to obtain the multi-task classifiers. Then, conduct experiments to compare the performance of the proposed method with several baslines on different data sets. The experimental results demonstrate the effectiveness of the proposed methods for multi-task classification.

Safe sample screening for regularized multi-task learning

As a machine learning paradigm, multi-task learning (MTL) attracts increasing attention recently. It can improve the overall performance by exploiting the correlation among different tasks. It is especially helpful in dealing with small sample learning problems. As a classic multi-task learner, regularized multi-task learning (RMTL) inspired lots of multi-task learning researches in the past. Massive researches have shown the performance of RMTL when compared to single-task learners, i.e., support vector machine. However, the training complexity will be considerably large when training large datasets. To tackle such a problem, we propose safe screening rules for an improved regularized multi-task support vector machine (IRMTL). By statically detecting and removing inactive samples from multiple tasks simultaneously before solving the reduced optimization problem, both rules reduce the training time significantly without incurring performance degradation of the proposed method. The experimental results on 13 benchmark datasets and an image dataset also clearly demonstrate the effectiveness of safe screening rules for IRMTL.

Multi-task $$\nu$$-twin support vector machines

Twin support vector machine (TWSVM) is proved to be better than support vector machine (SVM) in most cases, since it only deals with two smaller quadratic programming problems, which leads to high computational efficiency. It is proposed to solve a single-task learning problem, just like many other machine learning algorithms. However, a learning task may have relationships with other tasks in many practical problems. Training those tasks independently may neglect the underlying information among all tasks, while such information may be useful to improve the overall performance. Inspired by the multi-task learning theory, we propose two novel multi-task $$\nu$$ -TWSVMs. Both models inherit the merits of multi-task learning and $$\nu$$ -TWSVM. Meanwhile, they overcome the shortcomings of other multi-task SVMs and multi-task TWSVMs. Experimental results on three benchmark datasets and two popular image datasets also clearly demonstrate the effectiveness of our methods.

Efficient multi-task chemogenomics for drug specificity prediction.

Adverse drug reactions, also called side effects, range from mild to fatal clinical events and significantly affect the quality of care. Among other causes, side effects occur when drugs bind to proteins other than their intended target. As experimentally testing drug specificity against the entire proteome is out of reach, we investigate the application of chemogenomics approaches. We formulate the study of drug specificity as a problem of predicting interactions between drugs and proteins at the proteome scale. We build several benchmark datasets, and propose NN-MT, a multi-task Support Vector Machine (SVM) algorithm that is trained on a limited number of data points, in order to solve the computational issues or proteome-wide SVM for chemogenomics. We compare NN-MT to different state-of-the-art methods, and show that its prediction performances are similar or better, at an efficient calculation cost. Compared to its competitors, the proposed method is particularly efficient to predict (protein, ligand) interactions in the difficult double-orphan case, i.e. when no interactions are previously known for the protein nor for the ligand. The NN-MT algorithm appears to be a good default method providing state-of-the-art or better performances, in a wide range of prediction scenario that are considered in the present study: proteome-wide prediction, protein family prediction, test (protein, ligand) pairs dissimilar to pairs in the train set, and orphan cases.

Multi-Task Support Vector Machine for Data Classification

Multi-task Learning (MTL) algorithms aim to improve the performance of several learning methods through shared information among all tasks. One particularly successful instance of multi-task learning is its adaptation to support vector machine (SVM). Recently advances in large-margin learning have shown that their solutions may be misled by the spread of data and preferentially separate classes along large spread directions. In this paper, we propose a novel formulation for multi-task learning by extending the recently published relative margin machine algorithm to the multi-task learning paradigm. The new method is an extension of support vector machine for single task learning. The objective of our algorithm is to obtain a different predictor for each task while taking into account the fact that the tasks are related as well as the spread of the data. We test the proposed method experimentally using real data. The experiments show that the proposed method performs better than existing multi-task leaning with SVM and single-task leaning with SVM.

Multi-task support vector machines for feature selection with shared knowledge discovery

Feature selection is an effective way to reduce computational cost and improve feature quality for the large-scale multimedia analysis system. In this paper, we propose a novel feature selection method in which the hinge loss function with a ℓ2,1-norm regularization term is used to learn a sparse feature selection matrix for each learning task. Meanwhile, shared information exploiting across multiple tasks has been also taken into account by imposing a constraint which globally limits the combined feature selection matrices to be low-rank. A convex optimization method is proposed to use in the framework by minimizing the trace norm of a matrix instead of minimizing the rank of a matrix directly. Afterwards, gradient descent is applied to find the global optimum. Extensive experiments have been conducted across eight datasets for different multimedia applications, including action recognition, face recognition, object recognition and scene recognition. Experimental results demonstrate that the proposed method performs better than other compared approaches. Especially, when the shared information across multiple tasks is very beneficial to the multi-task learning, obvious improvements can be observed.

Multitask multiclass support vector machines: Model and experiments

Multitask learning or learning multiple related tasks simultaneously has shown a better performance than learning these tasks independently. Most approaches to multitask multiclass problems decompose them into multiple multitask binary problems, and thus cannot effectively capture inherent correlations between classes. Although very elegant, traditional multitask support vector machines are restricted by the fact that different learning tasks have to share the same set of classes. In this paper, we present an approach to multitask multiclass support vector machines based on the minimization of regularization functionals. We cast multitask multiclass problems into a constrained optimization problem with a quadratic objective function. Therefore, our approach can learn multitask multiclass problems directly and effectively. This approach can learn in two different scenarios: label-compatible and label-incompatible multitask learning. We can easily generalize the linear multitask learning method to the non-linear case using kernels. A number of experiments, including comparisons with other multitask learning methods, indicate that our approach for multitask multiclass problems is very encouraging.