Abstract Manufacturing processes undergo continuous changes in order to meet various requirements, such as process/product changes and variations in tool/workpiece conditions, leading to mixed, heterogenous, or anomalous data. As a result, a quality prediction model trained from previous data may not perform well when new tasks emerge. In order to achieve in-time and accurate product quality prediction, it is crucial to develop a predictive method that adapt to variations in the manufacturing system, capable of learning from new tasks without forgetting previous ones and detecting unknown tasks. This study proposes a deep learning method integrated with continual learning for in-situ quality prediction that is capable of learning from new tasks without forgetting previous ones. To demonstrate this idea, deep CNNs are designed to analyze in-process sensor data, which consist of shared layers to capture the common underlying features across all tasks, and task-specific layers that capture specific characteristics of each individual task. In order to identify the task to which incoming product belongs, a task prediction approach based on task relevancy using filter subspace distance is proposed. When new data come in, the model first identifies the task, followed by predicting the quality of the current product. The proposed method is demonstrated on two case studies, including quality prediction of the workpiece using acoustic emissions during the laser-induced plasma micro-machining process, and quality prediction of the product through thermal images during the hot stamping process.