Domain-incremental learning without forgetting based on random vector functional link networks

Abstract

Incremental learning is a paradigm that extends knowledge by learning from new data, often used to add new classes to an existing model or to learn a new domain. It imposes strict limitations on the model’s access to data from previous tasks, making it similar to the human learning process. The main challenge of incremental learning is catastrophic forgetting, where previous knowledge is severely forgotten while learning new tasks. In this work, we propose a novel approach for domain-incremental learning. Inspired by the Normal Equation, we accumulate the Gram Matrix from each task’s hidden layer output to update a simplified RVFL model. This algorithm achieves performance comparable to joint training while strictly adhering to privacy restrictions. With issues such as forgetting, storage requirements and privacy protection be addressed, this algorithm has the potential to play a crucial role in the field of edge computing and other related fields.