Catastrophic Forgetting Research Articles

Utilizing Inherent Bias for Memory Efficient Continual Learning: A Simple and Robust Baseline

Learning from continuously evolving data is critical in real-world applications. This type of learning, known as Continual Learning (CL), aims to assimilate new information without compromising performance on prior knowledge. However, learning new information leads to a bias in the network towards recent observations, resulting in a phenomenon known as catastrophic forgetting. The complexity increases in Online Continual Learning (OCL) scenarios where models are allowed only a single pass over data. Existing OCL approaches that rely on replaying exemplar sets are not only memory-intensive when it comes to large-scale datasets but also raise security concerns. While recent dynamic network models address memory concerns, they often present computationally demanding, over-parameterized solutions with limited generalizability. To address this longstanding problem, we propose a novel OCL approach termed “Bias Robust online Continual Learning (BRCL).” BRCL retains all intermediate models generated. These models inherently exhibit a preference for recently learned classes. To leverage this property for enhanced performance, we devise a strategy we describe as ‘utilizing bias to counteract bias.’ This method involves the development of an Inference function that capitalizes on the inherent biases of each model towards the recent tasks. Furthermore, we integrate a model consolidation technique that aligns the first layers of these models, particularly focusing on similar feature representations. This process effectively reduces the memory requirement, ensuring a low memory footprint. Despite the simplicity of the methodology to guarantee expandability to various frameworks, extensive experiments reveal a notable performance edge over leading methods on key benchmarks, getting continual learning closer to matching offline training. (Source code will be made publicly available upon the publication of this paper.)

Learning incremental audio–visual representation for continual multimodal understanding

Deep learning methods have demonstrated remarkable success in processing static datasets for various video tasks. However, when confronted with continuous data streams, these approaches often encounter the challenge of catastrophic forgetting. This phenomenon leads to a significant decline in overall performance when learning new classes incrementally. Moreover, existing methods tend to overlook the correlation between audio and visual modalities in video incremental learning, despite their joint significance in scene comprehension. How to continuously learn from new classes while maintaining the knowledge of old videos with limited storage and computing resources is becoming imperative in the field of multimodal learning. In this paper, we introduce CavRL, a pioneering benchmark for audio–visual representation learning under class incremental scenarios. To mitigate catastrophic forgetting, we propose a rehearsal-based training approach that leverages a small exemplar set from previous classes. Our approach constrains the memory buffer within strict storage limits, optimizing exemplar selection by learning correlative audio–visual representations. Additionally, we employ a distillation method to mitigate forgetting in a self-supervised manner. Evaluations on two prevalent multimodal tasks: audio–visual event classification and audio–visual speaker recognition, which demonstrate that CavRL outperforms existing state-of-the-art incremental learning methods across various settings. We anticipate that CavRL will significantly advance research in continual multimodal learning.

Rapid context inference in a thalamocortical model using recurrent neural networks

Cognitive flexibility is a fundamental ability that enables humans and animals to exhibit appropriate behaviors in various contexts. The thalamocortical interactions between the prefrontal cortex (PFC) and the mediodorsal thalamus (MD) have been identified as crucial for inferring temporal context, a critical component of cognitive flexibility. However, the neural mechanism responsible for context inference remains unknown. To address this issue, we propose a PFC-MD neural circuit model that utilizes a Hebbian plasticity rule to support rapid, online context inference. Specifically, the model MD thalamus can infer temporal contexts from prefrontal inputs within a few trials. This is achieved through the use of PFC-to-MD synaptic plasticity with pre-synaptic traces and adaptive thresholding, along with winner-take-all normalization in the MD. Furthermore, our model thalamus gates context-irrelevant neurons in the PFC, thus facilitating continual learning. We evaluate our model performance by having it sequentially learn various cognitive tasks. Incorporating an MD-like component alleviates catastrophic forgetting of previously learned contexts and demonstrates the transfer of knowledge to future contexts. Our work provides insight into how biological properties of thalamocortical circuits can be leveraged to achieve rapid context inference and continual learning.

Federated Incremental Learning algorithm based on Topological Data Analysis

Federated learning is a distributed learning approach aimed at preserving user’s data privacy, while incremental learning is an adaptive machine learning method that enables continuous learning of new data. The combination of these two approaches into Federated Incremental Learning (FIL) algorithms brings together their respective advantages. However, the existing federated incremental learning still faces two main challenges: (1) catastrophic forgetting of previous knowledge by local models when adapting to incremental tasks, and (2) limited capability of the server to capture critical features during federated aggregation. To address these challenges, this paper proposes a federated incremental learning algorithm based on Topological Data Analysis (TDA). Firstly, the algorithm extracts topological features from the input information and designs a Topological Stability Loss (TSL) to mitigate catastrophic forgetting of previous knowledge by local models. Secondly, a feature attention mechanism is utilized to select appropriate attention weights for each local model, enhancing the recognition performance of the global model. The experimental results on publicly available datasets, namely CIFAR10, CIFAR100, and ImageNet, demonstrate that the proposed approach in this paper achieves global accuracies of 67.23%, 65.75%, and 62.41% respectively for the federated incremental learning model. These accuracies surpass the existing Icarl incremental algorithm by improvements of 3.21%, 2.87%, and 1.34% respectively. Therefore, the algorithm presented in this paper achieves better performance, indicating its superiority over existing methods.

Continuously Learning Prediction Models for Smart Domestic Hot Water Management

Domestic hot water (DHW) consumption represents a significant portion of household energy usage, prompting the exploration of smart heat pump technology to efficiently meet DHW demands while minimizing energy waste. This paper proposes an innovative investigation of models using deep learning and continual learning algorithms to personalize DHW predictions of household occupants’ behavior. Such models, alongside a control system that decides when to heat, enable the development of a heat-pumped-based smart DHW production system, which can heat water only when needed and avoid energy loss due to the storage of hot water. Deep learning models, and attention-based models particularly, can be used to predict time series efficiently. However, they suffer from catastrophic forgetting, meaning that when they dynamically learn new patterns, older ones tend to be quickly forgotten. In this work, the continuous learning of DHW consumption prediction has been addressed by benchmarking proven continual learning methods on both real dwelling and synthetic DHW consumption data. Task-per-task analysis reveals, among the data from real dwellings that do not present explicit distribution changes, a gain compared to the non-evolutive model. Our experiment with synthetic data confirms that continual learning methods improve prediction performance.

Privacy-Enhanced Prototype-Based Federated Cross-Modal Hashing for Cross-Modal Retrieval

Cross-modal hashing is widely used for efficient similarity searches, improving data processing efficiency, and reducing storage costs. Existing cross-modal hashing methods primarily focus on centralized training scenarios, where fixed-scale and fixed-category multi-modal data is collected beforehand. However, these methods often face challenges associated with the potential risk of privacy breaches and high data communication costs during data transmission in real-world multimedia retrieval tasks. To tackle these challenges, in this article, we propose an efficient privacy-enhanced prototype-based federated cross-modal hashing (PEPFCH). In PEPFCH, we integrate local and global prototypes in order to effectively capture the distinctive traits of individual clients, while also harnessing the collective intelligence of the entire federated learning system. Moreover, to ensure the security of prototype information and prevent its disclosure during the aggregation process, we use a prototype encryption transmission mechanism to encrypt the prototype information before transmission, making it challenging for attackers to gain access to sensitive data. Additionally, to facilitate personalized federated learning and alleviate the issue of parametric catastrophic forgetting, we establish the image and text hyper-networks for each client and adopt a hyper-network extension strategy to selectively preserve and update previously acquired knowledge when acquiring new concepts or categories. Comprehensive experiments highlight the efficiency and superiority of our proposed method. To enhance research and accessibility, we have publicly released our source codes at: https://github.com/vindahi/PEPFCH .

Class incremental learning with analytic learning for hyperspectral image classification

Hyperspectral image classification (HSIC) is crucial for applications in agriculture and environmental monitoring. As ground objects evolve and remote sensing technology progresses, there is a growing need for HSIC models that can adapt to new data classes without requiring to be retrained from scratch. Throughout the continual learning procedure, the model is expected to not only effectively extract spatial–spectral features from the hyperspectral image but also alleviate the issue of catastrophic forgetting, i.e., the model forgets the learned classes’ knowledge when accessing novel classes during the training process. In this paper, for the HSIC task, we propose a class incremental learning method (HSI-CIL) that is based on analytic learning, a technique that converts network training into linear problems. Specifically, The HSI-CIL model consists of a lightweight feature extractor, a Feature Processing Module (FPM) and an Analytic Linear Classifier (ALC). This model does not need data storage for old and new classes and has only one epoch in the incremental learning stage, so it has lower consumption of resources and training time than several attempts have been proposed for addressing catastrophic forgetting. We perform abundant experiments with the proposed HSI-CIL on three publicly available hyperspectral datasets including Indian Pines, Pavia University, and Salinas. The experiments demonstrate that our HSI-CIL exceeds the state-of-the-art class incremental learning (CIL) techniques applied in HSIC with a certain gap.

Continual knowledge graph embedding enhancement for joint interaction-based next click recommendation

Knowledge Graph Embedding (KGE) based deep neural networks contribute to recommender systems in diverse application scenarios. However, Catastrophic Forgetting (CForg) significantly degrades their performance. Although exemplar replay is commonly adopted as a possible remedy to alleviate the intensity of CForg, a trade-off between performance and complexity occurs in this process. Therefore, in this work, we introduce Continual Knowledge graph embedding enhancement for joint Interaction-based Next click recommendation (CKIN) to defy the CForg and assuage the complexity. Typically, we introduce the Semantic Relevance Estimation (SRE) technique to ensure information relevance by filtering out irrelevant-data and reducing the space complexity. We introduce the SRE-enhanced deep probabilistic technique to probably replay the most relevant exemplars to defy the CForg and reduce the time complexity. Moreover, we introduce the integration of locality-preserving loss into the KGE framework to optimize the loss. In substantial experiments on real-world datasets, CKIN outperforms the baseline methods by effectively meeting the highlighted challenges.

Learning to learn for few-shot continual active learning

Continual learning strives to ensure stability in solving previously seen tasks while demonstrating plasticity in a novel domain. Recent advances in continual learning are mostly confined to a supervised learning setting, especially in NLP domain. In this work, we consider a few-shot continual active learning setting where labeled data are inadequate, and unlabeled data are abundant but with a limited annotation budget. We exploit meta-learning and propose a method, called Meta-Continual Active Learning. This method sequentially queries the most informative examples from a pool of unlabeled data for annotation to enhance task-specific performance and tackles continual learning problems through a meta-objective. Specifically, we employ meta-learning and experience replay to address inter-task confusion and catastrophic forgetting. We further incorporate textual augmentations to avoid memory over-fitting caused by experience replay and sample queries, thereby ensuring generalization. We conduct extensive experiments on benchmark text classification datasets from diverse domains to validate the feasibility and effectiveness of meta-continual active learning. We also analyze the impact of different active learning strategies on various meta continual learning models. The experimental results demonstrate that introducing randomness into sample selection is the best default strategy for maintaining generalization in meta-continual learning framework.

Latent Relations at Steady-state with Associative Nets.

Models of word meaning that exploit patterns of word usage across large text corpora to capture semantic relations, like the topic model and word2vec, condense word-by-context co-occurrence statistics to induce representations that organize words along semantically relevant dimensions (e.g., synonymy, antonymy, hyponymy, etc.). However, their reliance on latent representations leaves them vulnerable to interference, makes them slow learners, and commits to a dual-systems account of episodic and semantic memory. We show how it is possible to construct the meaning of words online during retrieval to avoid these limitations. We implement a spreading activation account of word meaning in an associative net, a one-layer highly recurrent network of associations, called a Dynamic-Eigen-Net, that we developed to address the limitations of earlier variants of associative nets when scaling up to deal with unstructured input domains like natural language text. We show that spreading activation using a one-hot coded Dynamic-Eigen-Net outperforms the topic model and reaches similar levels of performance as word2vec when predicting human free associations and word similarity ratings. Latent Semantic Analysis vectors reached similar levels of performance when constructed by applying dimensionality reduction to the Shifted Positive Pointwise Mutual Information but showed poorer predictability for free associations when using an entropy-based normalization. An analysis of the rate at which the Dynamic-Eigen-Net reaches asymptotic performance shows that it learns faster than word2vec. We argue in favor of the Dynamic-Eigen-Net as a fast learner, with a single-store, that is not subject to catastrophic interference. We present it as an alternative to instance models when delegating the induction of latent relationships to process assumptions instead of assumptions about representation.

Artificial intelligence-driven real-world battery diagnostics

Addressing real-world challenges in battery diagnostics, particularly under incomplete or inconsistent boundary conditions, has proven difficult with traditional methodologies such as first-principles and atomistic calculations. Despite advances in data assimilation techniques, the overwhelming volume and diversity of data, coupled with the lack of universally accepted models, underscore the limitations of these traditional approaches. Recently, deep learning has emerged as a highly effective tool in overcoming persistent issues in battery diagnostics by adeptly managing expansive design spaces and discerning intricate, multidimensional correlations. This approach resolves challenges previously deemed insurmountable, especially with lost, irregular, or noisy data through the design of specialized network architectures that adhere to physical invariants. However, gaps remain between academic advancements and their practical applications, including challenges in explainability and the computational costs associated with AI-driven solutions. Emerging technologies such as explainable artificial intelligence (XAI), AI for IT operations (AIOps), lifelong machine learning to mitigate catastrophic forgetting, and cloud-based digital twins open new opportunities for intelligent battery life-cycle assessment. In this perspective, we outline these challenges and opportunities, emphasizing the potential of innovative technologies to transform battery diagnostics, as demonstrated by our recent practice and the progress made in the field. This includes promising achievements in both academic and industry field demonstrations in modeling and forecasting the dynamics of multiphysics and multiscale battery systems. These systems feature inhomogeneous cascades of scales, informed by our physical, electrochemical, observational, empirical, and/or mathematical understanding of the battery system. Through data assimilation efforts, meticulous craftsmanship, and elaborate implementations—and by considering the wealth and spatio-temporal heterogeneity of available data—such AI-based intelligent learning philosophies have great potential to achieve better accuracy, faster training, and improved generalization.

Class incremental learning with self-supervised pre-training and prototype learning

Deep Neural Networks (DNNs) have achieved great success on classification tasks of closed class sets. However, new classes, like new categories of social media topics, are continual added to the real world, making it necessary to learn incrementally. This is hard for DNNs because they tend to overfit new classes while ignoring old classes, a phenomenon known as catastrophic forgetting. State-of-the-art (SOTA) methods rely on knowledge distillation and data replay techniques but still have limitations. In this work, we analyze the causes of catastrophic forgetting in class incremental learning, which refers to representation drift, representation confusion, and classifier distortion. Based on this view, we propose a two-stage learning framework with a fixed encoder and an incrementally updated prototype classifier. The encoder generates a feature space with high intrinsic dimensionality while the prototype classifier preserves the decision boundary. Experimental results on public image datasets show that our non-exemplar-based method significantly outperforms SOTA exemplar-based methods.

Comparative analysis of optimization algorithm-based training algorithms for a few shot continual learning

Incremental learning relies on the availability of ample training data for novel classes, a requirement that is often unfeasible in various application scenarios, particularly when new classes are rare groups that are pricey or challenging to attain. The main focus of incremental learning is on the tricky task of continuously learning to classify new classes in incoming data with no erasing knowledge of old classes. The research intends to develop a comparative analysis of optimization algorithms in training few-shot continual learning models to conquer catastrophic forgetting. The presented mechanism integrates various steps: pre-processing and classification. Images are initially pre-processed through contrast enhancement to elevate their quality. Pre-processed outputs are then classified by employing Continually Evolved Classifiers, generated to address a matter of catastrophic forgetting. Furthermore, to further enhance performance, Serial Exponential Sand Cat Swarm optimization algorithm (SE-SCSO) is employed and compared against ten other algorithms, containing Grey Wolf Optimization (GWO) algorithm, Moth flame optimization (MFO), cuckoo Search Optimization Algorithm (CSOA), Elephant Search Algorithm (ESA), Whale Optimization Algorithm (WOA), Artificial Algae Algorithm (AAA), Cat Swarm Optimization (CSO), Fish Swarm Algorithm (FSA), Genetic Bee Colony (GBC) Algorithm, and Particle swarm optimization (PSO). From the experiment results, SE-SCSO had attained the maximum performance with an accuracy of 89.6%, specificity of 86%, precision of 83%, recall of 92.3% and f-measure of 87.4%.

FIDWATCH: Federated incremental distillation for continuous monitoring of IoT security threats

The fast evolutions of Internet of Things (IoT) technologies have been accelerating their applicability in different sectors of life and becoming a pillar for sustainable development. However, this revolutionary expansion led to a substantial increase in attack surface, raising many concerns about security threats and their possible consequences. Machine learning has significantly contributed to designing intrusion detection systems (IDS) but suffers from critical limitations such as data privacy and sovereignty, data imbalance, concept drift, and catastrophic forgetting. This collectively makes existing IDSs an improper choice for securing IoT environments. This paper presents a federated learning approach called FIDWATCH to continuously monitor and detect a broad range of IoT security threats. The local side of FIDWATCH introduces contrastive focal loss to enhance the ability of the local model (teacher) to discriminate between diverse types of IoT security threats while putting an increased emphasis on hard-to-classify samples. A fine-grained Knowledge Distillation (KD) is introduced to allow the client to distill the required teacher's knowledge into a lighter, more compact model termed the pupil model. This greatly assists the competence and flexibility of the model in resource-constrained scenarios. Furthermore, an adaptive incremental updating method is introduced in FIDWATCH to allow the global model to exploit the distilled knowledge and refine the shared dataset. This helps generate global anchors for improving the robustness of the mode against the distributional shift, thereby improving model alignment and compliance with the dynamics of IoT security threats. Proof-of-concept simulations are performed on data from two public datasets (BoT-IoT and ToN-IoT), demonstrating the superiority of FIDWATCH over cutting-edge performance with an average f1-score of 97.07% and 95.63%, respectively.

APM: Adaptive parameter multiplexing for class incremental learning

In recent developments within the domain of image classification, deep neural networks (DNNs) have attracted considerable scholarly interest and have been extensively trained using data in closed environments. Such training methodologies contrast sharply with the inherently open, progressive, and adaptive processes of the natural visual system, leading to emergent challenges. Among these, catastrophic forgetting is notable, where the network acquisition of new class information precipitates the erosion of previously established knowledge. Additionally, the network encounters the stability-plasticity dilemma, necessitating a delicate equilibrium between assimilating novel classes and retaining existing ones. To address these issues, we propose a novel incremental learning model, termed Adaptive Parameter Multiplexing (APM), which incorporates a cross-class parameter adaptive incremental strategy. Central to our methodology is the conceptualization of parameter multiplexing or incremental as a learnable optimization problem, enabling the model to autonomously evaluate and decide on the necessity for parameter adjustment throughout its training lifecycle. This framework is designed to enhance the ability of the network to extract features for new class categories effectively through incremental parameters while simultaneously employing parameter multiplexing to augment storage optimization. Our model is underpinned by a dual strategy of coarse-grained and fine-grained parameter multiplexing, guided by a learnable score that dynamically assesses the appropriateness of parameter multiplexing versus incremental updates, facilitating an optimized balance for incremental model performance and storage. In addition, we have integrated a novel regularization loss mechanism for the learnable score to optimize storage efficiency. The effectiveness of APM is empirically validated through rigorous testing on benchmark datasets, including ImageNet100, CIFAR100, CIFAR10, and CUB200. The experimental outcomes indicate that, with a trace amount of parameter increase, our model achieves significant enhancements in classification performance across both new and previously established classes, thereby surpassing existing benchmarks set by state-of-the-art algorithms in the field.

Optimal Knowledge Distillation through Non-Heuristic Control of Dark Knowledge

In this paper, a method is introduced to control the dark knowledge values also known as soft targets, with the purpose of improving the training by knowledge distillation for multi-class classification tasks. Knowledge distillation effectively transfers knowledge from a larger model to a smaller model to achieve efficient, fast, and generalizable performance while retaining much of the original accuracy. The majority of deep neural models used for classification tasks append a SoftMax layer to generate output probabilities and it is usual to take the highest score and consider it the inference of the model, while the rest of the probability values are generally ignored. The focus is on those probabilities as carriers of dark knowledge and our aim is to quantify the relevance of dark knowledge, not heuristically as provided in the literature so far, but with an inductive proof on the SoftMax operational limits. These limits are further pushed by using an incremental decision tree with information gain split. The user can set a desired precision and an accuracy level to obtain a maximal temperature setting for a continual classification process. Moreover, by fitting both the hard targets and the soft targets, one obtains an optimal knowledge distillation effect that mitigates better catastrophic forgetting. The strengths of our method come from the possibility of controlling the amount of distillation transferred non-heuristically and the agnostic application of this model-independent study.

Advancing continual lifelong learning in neural information retrieval: Definition, dataset, framework, and empirical evaluation

Continual learning refers to the capability of a machine learning model to learn and adapt to new information, without compromising its performance on previously learned tasks. Although several studies have investigated continual learning methods for neural information retrieval (NIR) tasks, a well-defined task definition is still lacking, and it is unclear how typical learning strategies perform in this context. To address this challenge, a systematic task definition of continual NIR is presented, along with a multiple-topic dataset that simulates continuous information retrieval. A comprehensive continual neural information retrieval framework consisting of typical retrieval models and continual learning strategies is then proposed. Empirical evaluations illustrate that the proposed framework can successfully prevent catastrophic forgetting in neural information retrieval and enhance performance on previously learned tasks. The results also indicate that embedding-based retrieval models experience a decline in their continual learning performance as the topic shift distance and dataset volume of new tasks increase. In contrast, pretraining-based models do not show any such correlation. Adopting suitable learning strategies can mitigate the effects of topic shift and data augmentation in continual neural information retrieval.

Continual learning for seizure prediction via memory projection strategy

Despite extensive algorithms for epilepsy prediction via machine learning, most models are tailored for offline scenarios and cannot handle actual scenarios where data changes over time. Catastrophic forgetting(CF) for learned electroencephalogram(EEG) data occurs when EEG changes dynamically in the clinical setting. This paper implements a continual learning(CL) strategy Memory Projection(MP) for epilepsy prediction, which can be combined with other algorithms to avoid CF. Such a strategy enables the model to learn EEG data from each patient in dynamic subspaces with weak correlation layer by layer to minimize interference and promote knowledge transfer. Regularization Loss Reconstruction Algorithm and Matrix Dimensionality Reduction Algorithm are introduced into the core of MP. Experimental results show that MP exhibits excellent performance and low forgetting rates in sequential learning of seizure prediction. The forgetting rate of accuracy and sensitivity under multiple experiments are below 5%. When learning from multi-center datasets, the forgetting rates for accuracy and sensitivity decrease to 0.65% and 1.86%, making it comparable to state-of-the-art CL strategies. Through ablation experiments, we have analyzed that MP can operate with minimal storage and computational cost, which demonstrates practical potential for seizure prediction in clinical scenarios.

Hebbian Descent: A Unified View on Log-Likelihood Learning.

This study discusses the negative impact of the derivative of the activation functions in the output layer of artificial neural networks, in particular in continual learning. We propose Hebbian descent as a theoretical framework to overcome this limitation, which is implemented through an alternative loss function for gradient descent we refer to as Hebbian descent loss. This loss is effectively the generalized log-likelihood loss and corresponds to an alternative weight update rule for the output layer wherein the derivative of the activation function is disregarded. We show how this update avoids vanishing error signals during backpropagation in saturated regions of the activation functions, which is particularly helpful in training shallow neural networks and deep neural networks where saturating activation functions are only used in the output layer. In combination with centering, Hebbian descent leads to better continual learning capabilities. It provides a unifying perspective on Hebbian learning, gradient descent, and generalized linear models, for all of which we discuss the advantages and disadvantages. Given activation functions with strictly positive derivative (as often the case in practice), Hebbian descent inherits the convergence properties of regular gradient descent. While established pairings of loss and output layer activation function (e.g., mean squared error with linear or cross-entropy with sigmoid/softmax) are subsumed by Hebbian descent, we provide general insights for designing arbitrary loss activation function combinations that benefit from Hebbian descent. For shallow networks, we show that Hebbian descent outperforms Hebbian learning, has a performance similar to regular gradient descent, and has a much better performance than all other tested update rules in continual learning. In combination with centering, Hebbian descent implements a forgetting mechanism that prevents catastrophic interference notably better than the other tested update rules. When training deep neural networks, our experimental results suggest that Hebbian descent has better or similar performance as gradient descent.

Class Incremental Deep Learning: A Computational Scheme to Avoid Catastrophic Forgetting in Domain Generation Algorithm Multiclass Classification

Domain Generation Algorithms (DGAs) are algorithms present in most malware used by botnets and advanced persistent threats. These algorithms dynamically generate domain names to maintain and obfuscate communication between the infected device and the attacker’s command and control server. Since DGAs are used by many threats, it is extremely important to classify a given DGA according to the threat it is related to. In addition, as new threats emerge daily, classifier models tend to become obsolete over time. Deep neural networks tend to lose their classification ability when retrained with a dataset that is significantly different from the initial one, a phenomenon known as catastrophic forgetting. This work presents a computational scheme composed of a deep learning model based on CNN and natural language processing and an incremental learning technique for class increment through transfer learning to classify 60 DGA families and include a new family to the classifier model, training the model incrementally using some examples from known families, avoiding catastrophic forgetting and maintaining metric levels. The proposed methodology achieved an average precision of 86.75%, an average recall of 83.06%, and an average F1 score of 83.78% with the full dataset, and suffered minimal losses when applying the class increment.