Continuous Learning Framework Research Articles

Personalized Federated Continual Learning for Task-Incremental Biometrics

In the age of Internet of Things where information is explosively growing, people pay more attention on personal privacy. In the real-world task-incremental scenario for biometrics, every edge device faces continuous task flows of private data without communication with others. Security and Performance are the primary concerns in identity authentication, and Federated Continual Learning (FCL) is a promising solution. In this paper, we design a personalized federated continual learning framework to solve the problem of sequentially identification in every distributed device. For each client, we create an adaptive continual meta-learning model called cTD-αMAML, aiming to align the gradients of previous and new tasks and to make the learning-rate model learnable. For central aggregation, the server gathers the meta-initialization from every local update and allocates the updated global meta-initialization to clients. We propose an extension of FedAvg to locally reserve the learnable learning-rate network to realize the personalization of clients. Results prove that in Continual Learning, our cTD-αMAML can learn to learn the seen tasks and avoid catastrophic forgetting. And in Federated Continual Learning, our personalized method realizes the knowledge transferring across clients, meanwhile improving the local performance and reducing the communication cost. In this way, the proposed personalized federated continual learning framework can obtain a biometric template that is able to learn the expression space for new tasks with rapid adaption.

Federated Continuous Learning Based on Stacked Broad Learning System Assisted by Digital Twin Networks: An Incremental Learning Approach for Intrusion Detection in UAV Networks

The edge of the Internet of Things (IoT), which consists of Unmanned Aerial Vehicles (UAVs), is vulnerable to network intrusion because software and wireless connections are used extensively in the IoT. Designing an efficient Intrusion Detection System (IDS) model is imperative. However, when creating IDS models with distributed data collected by UAVs, it is necessary to take precautions to protect the data’s security and privacy. Furthermore, most of the IDS models are focused on one-time learning but not on continuous learning. To this end, we propose a Federated Continuous learning framework with a Stacked Broad Learning System (FCL-SBLS) based on Digital Twin Network (DTN), which can learn and train the IDS model on new data quickly and continuously. In order to improve the efficiency and quality of the IDS model when training and aggregation, we employ an asynchronous federated learning architecture, and a Deep Deterministic Policy Gradient (DDPG)-based UAV selection scheme assisted by DTN is proposed to help the global IDS model aggregation. The presented algorithm is validated using the CIC-IDS2017 dataset, and the simulation results reveal that our algorithm achieves higher efficiency and accuracy than the existing FL scheme.

Self-driven continual learning for class-added motor fault diagnosis based on unseen fault detector and propensity distillation

Continual learning is a high-potential technique that enables intelligent motor fault diagnosis models to extend new diagnosable fault classes without costly training from scratch. However, existing continual learning methods have the following limitations. (1) They manually detect new faults, which is labor-intensive, untimely, and more importantly, may lead to mistaken diagnosis results. (2) They adopt the traditional knowledge distillation to align the absolute responses of old and new models, which alleviates catastrophic forgetting but restricts flexible learning from incremental datasets. To overcome the above limitations, this paper proposes a novel self-driven continual learning framework for class-added motor fault diagnosis, which can spontaneously detect unseen faults and perform more flexible continual learning from incremental datasets. For the automatic detection of unseen faults, after collecting online samples, adversarial training with exemplars of each seen class is conducted to measure the class separability. The truth fault classes that are unseen for diagnosis models can be clearly distinguished from all seen classes, and correspondingly missed diagnosis or misdiagnosing can be avoided effectively and incremental samples with new fault types can be collected quickly. For the flexible continual learning strategy, a more flexible knowledge distillation is proposed to preserve the prediction propensity rather than the absolute response. This strategy not only keeps the recognition performance of old classes but also loosens unnecessary constraints and increases the diagnosis model plasticity to learn new knowledge from incremental datasets, thus improving the accuracy of motor fault diagnosis during continual learning. The effectiveness of the proposed method is verified by conducting fault simulation experiments of three-phase motors and its superiority is also demonstrated by comparing it with some state-of-the-art diagnosis methods.

Continual learning approaches for single cell RNA sequencing data

Single-cell RNA sequencing data is among the most interesting and impactful data of today and the sizes of the available datasets are increasing drastically. There is a substantial need for learning from large datasets, causing nontrivial challenges, especially in hardware. Loading even a single dataset into the memory of an ordinary, off-the-shelf computer can be infeasible, and using computing servers might not always be an option. This paper presents continual learning as a solution to such hardware bottlenecks. The findings of cell-type classification demonstrate that XGBoost and Catboost algorithms, when implemented in a continual learning framework, exhibit superior performance compared to the best-performing static classifier. We achieved up to 10% higher median F1 scores than the state-of-the-art on the most challenging datasets. On the other hand, these algorithms can suffer from variations in data characteristics across diverse datasets, pointing out indications of the catastrophic forgetting problem.

CRNet: A Fast Continual Learning Framework With Random Theory.

Artificial neural networks are prone to suffer from catastrophic forgetting. Networks trained on something new tend to rapidly forget what was learned previously, a common phenomenon within connectionist models. In this work, we propose an effective and efficient continual learning framework using random theory, together with Bayes' rule, to equip a single model with the ability to learn streaming data. The core idea of our framework is to preserve the performance of old tasks by guiding output weights to stay in a region of low error while encountering new tasks. In contrast to the existing continual learning approaches, our main contributions concern (1) closed-formed solutions with detailed theoretical analysis; (2) training continual learners by one-pass observation of samples; (3) remarkable advantages in terms of easy implementation, efficient parameters, fast convergence, and strong task-order robustness. Comprehensive experiments under popular image classification benchmarks, FashionMNIST, CIFAR-100, and ImageNet, demonstrate that our methods predominately outperform the extensive state-of-the-art methods on training speed while maintaining superior accuracy and the number of parameters, in the class incremental learning scenario. Code is available at https://github.com/toil2sweet/CRNet.

Continual Learning for Cuffless Blood Pressure Measurement using PPG and ECG Signals.

Although numerous studies have been conducted on cuffless blood pressure (BP) estimation using machine learning methods, most of the data-driven models are static, with model parameters fixed after training is complete. However, BP is dynamic and the performance would degrade for a static model when the to-be predicted BP distribution deviates from the training BP distribution. In this paper, we propose a continual learning (CL) framework in which deep learning models are developed to learn dynamically and continuously for arterial BP (ABP) estimation with photoplethysmography (PPG) and electrocardiogram (ECG) waveforms. The effectiveness of the CL model is validated on UCI Repository and MIMIC-III database with a total of 132 individual samples, and compared with conventional training method. It was found that the CL model improved the ABP estimation accuracy in terms of mean absolute error (MAE) by 17.47% on average compared with conventional training model. Furthermore, the improvement increased with the variability of ABP. These results demonstrate that CL model has potential to estimate dynamic ABP, which has been challenging with conventional training.

A multi-label Continual Learning framework to scale deep learning approaches for packaging equipment monitoring

Continual Learning aims to learn from a stream of tasks, being able to remember at the same time both new and old tasks. We propose a scenario that holds immense appeal for various real-world applications, where a model adapts to handle a stream of machines with distribution shifts Tests on real packaging data proved the feasibility of Continual Learning for addressing such problems. Our study uncovers the limitations of previous algorithms in the Domain Incremental Learning. Our research presents a novel approach for tackling multi-label tasks in Continual Learning, achieving superior performance compared to existing approaches found in the literature. Our method not only achieves optimal performance but also has logarithmic complexity, significantly reducing computation times.

A continual learning framework for adaptive defect classification and inspection

Machine-vision-based defect classification techniques have been widely adopted for automatic quality inspection in manufacturing processes. This article describes a general framework for classifying defects from high volume data batches with efficient inspection of unlabeled samples. The concept is to construct a detector to identify new defect types, send them to the inspection station for labeling, and dynamically update the classifier in an efficient manner that reduces both storage and computational needs imposed by data samples of previously observed batches. Both a simulation study on image classification and a case study on surface defect detection via 3D point clouds are performed to demonstrate the effectiveness of the proposed method.

Mutual mentor: Online contrastive distillation network for general continual learning

The goal of General Continual Learning (GCL) is to preserve learned knowledge and learn new knowledge with constant memory from infinite data stream where task boundaries are blurry. Distilling the model’s response of reserved samples between the old and new models is an effective way to achieve promising performance on GCL. However, it accumulates the inherent old model’s response bias and is not robust to model changes. To this end, we propose a Mutual Mentor General Continual Learning (MMGCL) framework to tackle these problems, which explores a training process in which the student and teacher models mentor each other. Concretely, the student model consolidates the learned knowledge by respectively aligning the relation and adaptive responses with those of the teacher model while the teacher model updates its parameters by integrating the parameters of the student model to accumulate new knowledge. To further improve the effectiveness of the mutual mentor, we integrate the inter-instance knowledge to optimize the outputs of the teacher model, which can not only supervise the student model but also indirectly optimize the teacher model. Extensive experiments on six benchmark datasets demonstrate that our MMGCL significantly outperforms state-of-the-art approaches under diverse continual learning settings with various buffer sizes.

Rate of penetration estimation downhole with machine learning for drilling position control

The development of fully autonomous downhole directional drilling systems has been hindered by the lack of rate of penetration (ROP) information downhole. This information is critical for determining bit position and steering dynamics, which currently can only be determined at the rig. Consequently, drilling position controllers must be placed on the surface, which leads to significant performance limitations due to communication delays with the downhole drilling tool. This study investigates a new approach that enables downhole implementation of position control: real-time estimation of ROP downhole. Motivated by results from the field of drilling optimization, estimation is performed using machine learning. In contrast to prior work, we limit model features to only measurements acquired downhole, and we focus on minimizing model size given the constrained computing environment downhole. We use a continuous learning framework with incremental training data availability and derive relationships that reveal how prediction error can impact position measurement and controller dynamics. Additionally, we develop novel performance metrics and introduce a modified form of cross-validation for model evaluation. Using data acquired from three directional wells, experiments are conducted to compare the performance of several machine learning models and to assess the effect of varying retraining frequency and training data quantity. The results indicate that the proposed method achieves comparable accuracy to models implemented on the surface despite being constrained in size and model inputs. Additionally, the results show that increasing the retraining frequency almost always reduces test error, and that the training data can be downsampled substantially without a significant drop in accuracy.

Bayesian continual learning via spiking neural networks.

Among the main features of biological intelligence are energy efficiency, capacity for continual adaptation, and risk management via uncertainty quantification. Neuromorphic engineering has been thus far mostly driven by the goal of implementing energy-efficient machines that take inspiration from the time-based computing paradigm of biological brains. In this paper, we take steps toward the design of neuromorphic systems that are capable of adaptation to changing learning tasks, while producing well-calibrated uncertainty quantification estimates. To this end, we derive online learning rules for spiking neural networks (SNNs) within a Bayesian continual learning framework. In it, each synaptic weight is represented by parameters that quantify the current epistemic uncertainty resulting from prior knowledge and observed data. The proposed online rules update the distribution parameters in a streaming fashion as data are observed. We instantiate the proposed approach for both real-valued and binary synaptic weights. Experimental results using Intel's Lava platform show the merits of Bayesian over frequentist learning in terms of capacity for adaptation and uncertainty quantification.

Tf-GCZSL: Task-free generalized continual zero-shot learning

Learning continually from a stream of training data or tasks with an ability to learn the unseen classes using a zero-shot learning framework is gaining attention in the literature. It is referred to as continual zero-shot learning (CZSL). Existing CZSL requires clear task-boundary information during training which is not practically feasible. This paper proposes a task-free generalized CZSL (Tf-GCZSL) method with short-term/long-term memory to overcome the requirement of task-boundary in training. A variational autoencoder (VAE) handles the fundamental ZSL tasks. The short-term and long-term memory help to overcome the condition of the task boundary in the CZSL framework. Further, the proposed Tf-GCZSL method combines the concept of experience replay with dark knowledge distillation and regularization to overcome the catastrophic forgetting issues in a continual learning framework. Finally, the Tf-GCZSL uses a fully connected classifier developed using the synthetic features generated at the latent space of the VAE. The performance of the proposed Tf-GCZSL is evaluated in the existing task-agnostic prediction setting and the proposed task-free setting for the generalized CZSL over the five ZSL benchmark datasets. The results clearly indicate that the proposed Tf-GCZSL improves the prediction at least by 12%, 1%, 3%, 4%, and 3% over existing state-of-the-art and baseline methods for CUB, aPY, AWA1, AWA2, and SUN datasets, respectively in both settings (task-agnostic prediction and task-free learning). The source code is available at https://github.com/Chandan-IITI/Tf-GCZSL.

The Continuous Learning Framework: Applying accountability for widespread organisational change

This article introduces the Continuous Learning Framework for quality within higher education and other internal evaluation contexts. The framework maps the terrain of this special issue, and consists of four elements: accountability, improvement, performance and investment. The article briefly describes the evolution of the framework including its application to a government reform package, before focussing on its practical application within a specific university context to stimulate widespread organisational change and improve the use of data in academic decision-making. For internal evaluators, the framework offers a structured way to step into strategic decision-making conversations with executives and a structured approach to embed quality or internal evaluation within policy, protocols and practice of governments, organisations, teams and individuals. Moreover, it facilitates capacity building and creates an environment conducive to continuous improvement leading to continuous learning. The framework and its application directly align with Deming’s work from the 1980s.

Improving Pedestrian Prediction Models With Self-Supervised Continual Learning

Autonomous mobile robots require accurate human motion predictions to safely and efficiently navigate among pedestrians, whose behavior may adapt to environmental changes. This letter introduces a self-supervised continual learning framework to improve data-driven pedestrian prediction models online across various scenarios continuously. In particular, we exploit online streams of pedestrian data, commonly available from the robot’s detection and tracking pipeline, to refine the prediction model and its performance in unseen scenarios. To avoid the forgetting of previously learned concepts, a problem known as catastrophic forgetting, our framework includes a regularization loss to penalize changes of model parameters that are important for previous scenarios and retrains on a set of previous examples to retain past knowledge. Experimental results on real and simulation data show that our approach can improve prediction performance in unseen scenarios while retaining knowledge from seen scenarios when compared to naively training the prediction model online.

ACCESSIBLE, COMPREHENSIVE, CONTINUOUS, EXPERIENTIAL, AND REFLECTIVE (ACCER) STUDENT LEARNING FRAMEWORK OF PARANAQUE CITY COLLEGE

Paranaque City College (PCC) in the Philippines has no existing Student Learning Framework that will enhance the teaching and learning process and the quality of student learning. This study discusses about the significance of the Student Learning Framework called ACCER or Accessible, Comprehensive, Continuous, Experiential and Reflective. ACCER is an acronym for Accessible, Comprehensive, Continuous, Experiential and Reflective Student Learning Framework. This study was conducted to determine the current status of the quality of teaching and learning at Paranaque City College. Specifically, it aimed to: (1) determine the demographic profiles of both educators and students in terms of age, gender, employment status/year level, length of service/student status; (2) find out the level of assessment in the current learning framework in PCC as perceived both by educators and students in terms of the following descriptions: a. accessible b. comprehensive c. continuous d. experiential e. reflective. This study utilized the quantitative method of research. Researchers use mathematical frameworks and ideas that relate to the amount under consideration in this technique of inquiry. Objective, detailed, and in many cases investigative, quantitative research frameworks are characteristics of the discipline. This style of investigation yields logical, statistical, and unbiased results. Structured data collection was carried out utilizing samples that represent the 10% of the population. There were 187 student participants and 30 faculty members in this quantitative investigation. The ACCER Student Learning Framework’s criteria, such as level of agreeableness, were found to be helpful in determining the current status of educators and students. Educators may find that students’ self-reflection is at its maximum point when using a holistic approach. Faculty members, on the other hand, must work together to improve their methods of teaching. According to the findings of the study, educators must update their teaching materials to make resources more readily available. ACCER Student Learning Framework, developed by PCC’s current President/College Administrator, has crucial indicators that can help all of PCC’s faculty members think more deeply on their teaching methods. This study is limited only to the student learning framework to improve the teaching and learning process at Paranaque City College. The Student Learning Framework was originally developed by the main author and its uniqueness will improve the teaching and learning process and student performance of Paranaque City College students.

Admitting the addressee detection faultiness of voice assistants to improve the activation performance using a continuous learning framework

The main promise of voice assistants is their ability to correctly interpret and learn from user input as well as the ability to utilize this knowledge to achieve specific goals and tasks. These systems need predetermined activation actions to start a conversation. Unfortunately, the typically used solution, wake-words, force an unnatural interaction. Furthermore, this method can also confuse when the wake-word, or a phonetically similar phrase, has been said but no interaction with the system is intended by the user. Thereby, the system not only lacks the adequacy of interpersonal interaction, it moreover suffers from an addressee detection faultiness. Although various aspects have already been investigated in this field of acoustic addressee detection research, we demonstrated that the test data used so far rely on ideal conditions: The dialog complexity between human–human and human–device interactions is essentially different while in reality, the behavior of each individual addressing either another human or a device is of large variation. Thus the problem of addressee detection is simplified too much. Our approach works with a specifically designed dataset comprising of human–human and human–computer interactions of similar dialog complexity. Our proposed addressee detection faultiness framework actively communicates the system’s uncertainty that may arise. In connection with a continuous learning framework, this enables a voice assistant system to adapt itself to the users’ individual addressee behavior. This approach achieves significantly improved classification rates of 85.77%, which gives an absolute improvement of 32.22% in comparison to similar experiments employing human annotations as ground truth.

Effective collection construction for information retrieval evaluation and optimization

The availability of test collections in Cranfield paradigm has significantly benefited the development of models, methods and tools in information retrieval. Such test collections typically consist of a set of topics, a document collection and a set of relevance assessments. Constructing these test collections requires effort of various perspectives such as topic selection, document selection, relevance assessment, and relevance label aggregation etc. The work in the thesis provides a fundamental way of constructing and utilizing test collections in information retrieval in an effective, efficient and reliable manner. To that end, we have focused on four aspects. We first study the document selection issue when building test collections. We devise an active sampling method for efficient large-scale evaluation [Li and Kanoulas, 2017]. Different from past sampling-based approaches, we account for the fact that some systems are of higher quality than others, and we design the sampling distribution to over-sample documents from these systems. At the same time, the estimated evaluation measures are unbiased, and assessments can be used to evaluate new, novel systems without introducing any systematic error. Then a natural further step is determining when to stop the document selection and assessment procedure. This is an important but understudied problem in the construction of test collections. We consider both the gain of identifying relevant documents and the cost of assessing documents as the optimization goals. We handle the problem under the continuous active learning framework by jointly training a ranking model to rank documents, and estimating the total number of relevant documents in the collection using a "greedy" sampling method [Li and Kanoulas, 2020]. The next stage of constructing a test collection is assessing relevance. We study how to denoise relevance assessments by aggregating from multiple crowd annotation sources to obtain high-quality relevance assessments. This helps to boost the quality of relevance assessments acquired in a crowdsourcing manner. We assume a Gaussian process prior on query-document pairs to model their correlation. The proposed model shows good performance in terms of interring true relevance labels. Besides, it allows predicting relevance labels for new tasks that has no crowd annotations, which is a new functionality of CrowdGP. Ablation studies demonstrate that the effectiveness is attributed to the modelling of task correlation based on the axillary information of tasks and the prior relevance information of documents to queries. After a test collection is constructed, it can be used to either evaluate retrieval systems or train a ranking model. We propose to use it to optimize the configuration of retrieval systems. We use Bayesian optimization approach to model the effect of a δ -step in the configuration space to the effectiveness of the retrieval system, by suggesting to use different similarity functions (covariance functions) for continuous and categorical values, and examine their ability to effectively and efficiently guide the search in the configuration space [Li and Kanoulas, 2018]. Beyond the algorithmic and empirical contributions, work done as part of this thesis also contributed to the research community as the CLEF Technology Assisted Reviews in Empirical Medicine Tracks in 2017, 2018, and 2019 [Kanoulas et al., 2017, 2018, 2019]. Awarded by: University of Amsterdam, Amsterdam, The Netherlands. Supervised by: Evangelos Kanoulas. Available at: https://dare.uva.nl/search?identifier=3438a2b6-9271-4f2c-add5-3c811cc48d42.

When to Stop Reviewing in Technology-Assisted Reviews

Technology-Assisted Reviews (TAR) aim to expedite document reviewing (e.g., medical articles or legal documents) by iteratively incorporating machine learning algorithms and human feedback on document relevance. Continuous Active Learning (CAL) algorithms have demonstrated superior performance compared to other methods in efficiently identifying relevant documents. One of the key challenges for CAL algorithms is deciding when to stop displaying documents to reviewers. Existing work either lacks transparency—it provides an ad-hoc stopping point, without indicating how many relevant documents are still not found, or lacks efficiency by paying an extra cost to estimate the total number of relevant documents in the collection prior to the actual review. In this article, we handle the problem of deciding the stopping point of TAR under the continuous active learning framework by jointly training a ranking model to rank documents, and by conducting a “greedy” sampling to estimate the total number of relevant documents in the collection. We prove the unbiasedness of the proposed estimators under a with-replacement sampling design, while experimental results demonstrate that the proposed approach, similar to CAL, effectively retrieves relevant documents; but it also provides a transparent, accurate, and effective stopping point.

Bi-Objective Continual Learning: Learning ‘New’ While Consolidating ‘Known’

In this paper, we propose a novel single-task continual learning framework named Bi-Objective Continual Learning (BOCL). BOCL aims at both consolidating historical knowledge and learning from new data. On one hand, we propose to preserve the old knowledge using a small set of pillars, and develop the pillar consolidation (PLC) loss to preserve the old knowledge and to alleviate the catastrophic forgetting problem. On the other hand, we develop the contrastive pillar (CPL) loss term to improve the classification performance, and examine several data sampling strategies for efficient onsite learning from ‘new’ with a reasonable amount of computational resources. Comprehensive experiments on CIFAR10/100, CORe50 and a subset of ImageNet validate the BOCL framework. We also reveal the performance accuracy of different sampling strategies when used to finetune a given CNN model. The code will be released.

Continuous reinforcement learning to adapt multi-objective optimization online for robot motion

This article introduces a continuous reinforcement learning framework to enable online adaptation of multi-objective optimization functions for guiding a mobile robot to move in changing dynamic environments. The robot with this framework can continuously learn from multiple or changing environments where it encounters different numbers of obstacles moving in unknown ways at different times. Using both planned trajectories from a real-time motion planner and already executed trajectories as feedback observations, our reinforcement learning agent enables the robot to adapt motion behaviors to environmental changes. The agent contains a Q network connected to a long short-term memory network. The proposed framework is tested in both simulations and real robot experiments over various, dynamically varied task environments. The results show the efficacy of online continuous reinforcement learning for quick adaption to different, unknown, and dynamic environments.