Abstract
One key objective of artificial intelligence involves the continuous adaptation of machine learning models to new tasks. This branch of continual learning is also referred to as lifelong learning (LL), where a major challenge is to minimize catastrophic forgetting, or forgetting previously learned tasks. While previous work on catastrophic forgetting has been focused on vision problems; this work targets time-series data. In addition to choosing an architecture appropriate for time-series sequences, our work addresses limitations in previous work, including the handling of distribution shifts in class labels. We present multi-objective learning with three loss functions to minimize catastrophic forgetting, prediction error, and errors in generalizing across label shifts, simultaneously. We build a multi-task autoencoder network with a hierarchical convolutional recurrent architecture. The proposed method is capable of learning multiple time-series tasks simultaneously. For cases where the model needs to learn multiple new tasks, we propose sequential learning, starting with tasks that have the best individual performances. This solution was evaluated on four benchmark human activity recognition datasets collected from mobile sensing devices. A wide set of baseline comparisons is performed, and an ablation analysis is run to evaluate the impact of the different losses in the proposed multi-objective method. The results demonstrate an up to 4% performance improvement in catastrophic forgetting compared to the use of loss functions in state-of-the-art solutions while demonstrating minimal losses compared to upper bound methods of traditional fine-tuning (FT) and multi-task learning (MTL).
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.