L<sup>3</sup>-SVM: a lifelong learning method for SVM

Abstract

We propose a lifelong learning method for Support Vector Machine (SVM) training called L3-SVM. It focuses on the large-scale and endless stream data which are usually the characteristics of lifelong learning. The batch SVM cannot handle large-scale data for the huge space and time requirements and also cannot learn incrementally. In order to solve these two problems, we introduce a Prototype Support Layer (PSL) before SVM training, which is maintained by a Learning Prototype Network (LPN). The LPN learns representative prototypes from the input data in an online way and the PSL records the learned prototypes based on which the SVM is trained. As the size of the prototypes in the PSL is small in comparison with original data, L3-SVM is very fast. Another challenge of lifelong learning is the “open-ended” environment, i.e. data from novel classes or new distributions may occur during learning, upon which the classifiers should be retrained or updated. Many versions of incremental SVM only retain the Support Vectors after training, losing much potentially useful information of the original data, which leads to a declined performance as new data arrives. The PSL, on the other hand, works as a Long Term Memory (LTM) system. It records the representative prototypes of all previous data. Experiments demonstrate that the SVM trained upon these representative prototypes is as accurate as the state-of-the-art SVM, and much faster and can handle much larger data set than existing methods. Moreover, due to the incremental and self-adaptive properties of LPN, L3-SVM is able to work efficiently and effectively when novel class data come.