Safe sample screening for regularized multi-task learning

Abstract

As a machine learning paradigm, multi-task learning (MTL) attracts increasing attention recently. It can improve the overall performance by exploiting the correlation among different tasks. It is especially helpful in dealing with small sample learning problems. As a classic multi-task learner, regularized multi-task learning (RMTL) inspired lots of multi-task learning researches in the past. Massive researches have shown the performance of RMTL when compared to single-task learners, i.e., support vector machine. However, the training complexity will be considerably large when training large datasets. To tackle such a problem, we propose safe screening rules for an improved regularized multi-task support vector machine (IRMTL). By statically detecting and removing inactive samples from multiple tasks simultaneously before solving the reduced optimization problem, both rules reduce the training time significantly without incurring performance degradation of the proposed method. The experimental results on 13 benchmark datasets and an image dataset also clearly demonstrate the effectiveness of safe screening rules for IRMTL.