Abstract
AbstractIn the Class Incremental Learning (CIL) setup, a learning model must have the ability to incrementally update its knowledge to recognize newly appeared classes (plasticity) while maintaining the knowledge to recognize the classes it has already learned (stability). Such conflicting requirements are known as the stability-plasticity dilemma, and most existing studies attempt to achieve a good balance between them by stability improvements. Unlike those attempts, we focus on the generality of representations. The basic idea is that a model does not need to change if it has already learned such general representations that they contain enough information to recognize new classes. However, the general representations are not optimal for recognizing the classes a model has already learned because the representations must contain unrelated and noisy information for recognizing them. To acquire representations suitable for recognizing known classes while leveraging general representations, in this paper, we propose a new CIL framework that learns general representations and transforms them into suitable ones for the target classification tasks. In our framework, we achieve the acquisition of general representations and their transformation by self-supervised learning and attention techniques, respectively. In addition, we introduce a novel knowledge distillation loss to make the transformation mechanism stable. Using benchmark datasets, we empirically confirm that our framework can improve the average incremental accuracy of four types of CIL methods that employ knowledge distillation in the CIL setting.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.