Incremental Learning Algorithm Research Articles

An Incremental Naive Bayes Learner for Real-time Health Prediction

Healthcare monitoring systems have improved with the Internet of Things and machine learning prediction models. Traditional batch machine-learning approaches cannot generate an effective model since most data will be continuous and real-time. Real-time medical data processing is challenging since the entire data is unavailable during prediction. Here, a continuous model adaptation based on incremental learning is demanded. The development of such a system can significantly improve patient outcomes and reduce healthcare costs. This paper proposes an Incremental Naive Bayes Learner that can handle concept drifts in data. The algorithm keeps a sliding window of data constantly updated as new data is added. The adaptive window size determines how much data is used to train the model at any given time. After processing each chunk of data, the algorithm calculates the model’s accuracy. The system detects a concept drift and dynamically updates the training set if the accuracy drops below a predefined threshold. We conducted a comparative evaluation of state-of-the-art batch and incremental learning algorithms on different medical datasets, demonstrating the impact of incremental learning. The results demonstrate the effectiveness of our approach: the Agrawal dataset achieved the highest accuracy at 66.6%, followed by Dialysis at 61.6%, while Liver and HCC achieved 50% and 58.3% accuracy, respectively. This approach ensures a sustained high level of accuracy in healthcare monitoring systems over time, even amidst concept drift.

Global Model Selection via Solution Paths for Robust Support Vector Machine.

Robust support vector machine (RSVM) using ramp loss provides a better generalization performance than traditional support vector machine (SVM) using hinge loss. However, the good performance of RSVM heavily depends on the proper values of regularization parameter and ramp parameter. Traditional model selection technique with gird search has extremely high computational cost especially for fine-grained search. To address this challenging problem, in this paper, we first propose solution paths of RSVM (SPRSVM) based on the concave-convex procedure (CCCP) which can track the solutions of the non-convex RSVM with respect to regularization parameter and ramp parameter respectively. Specifically, we use incremental and decremental learning algorithms to deal with the Karush-Khun-Tucker violating samples in the process of tracking the solutions. Based on the solution paths of RSVM and the piecewise linearity of model function, we can compute the error paths of RSVM and find the values of regularization parameter and ramp parameter, respectively, which corresponds to the minimum cross validation error. We prove the finite convergence of SPRSVM and analyze the computational complexity of SPRSVM. Experimental results on a variety of benchmark datasets not only verify that our SPRSVM can globally search the regularization and ramp parameters respectively, but also show a huge reduction of computational time compared with the grid search approach.

When Broad Learning System Meets Label Noise Learning: A Reweighting Learning Framework.

Broad learning system (BLS) is a novel neural network with efficient learning and expansion capacity, but it is sensitive to noise. Accordingly, the existing robust broad models try to suppress noise by assigning each sample an appropriate scalar weight to tune down the contribution of noisy samples in network training. However, they disregard the useful information of the noncorrupted elements hidden in the noisy samples, leading to unsatisfactory performance. To this end, a novel BLS with adaptive reweighting (BLS-AR) strategy is proposed in this article for the classification of data with label noise. Different from the previous works, the BLS-AR learns for each sample a weight vector rather than a scalar weight to indicate the noise degree of each element in the sample, which extends the reweighting strategy from sample level to element level. This enables the proposed network to precisely identify noisy elements and thus highlight the contribution of informative ones to train a more accurate representation model. Thanks to the separability of the model, the proposed network can be divided into several subnetworks, each of which can be trained efficiently. In addition, three corresponding incremental learning algorithms of the BLS-AR are developed for adding new samples or expanding the network. Substantial experiments are conducted to explicate the effectiveness and robustness of the proposed BLS-AR model.

A Memory-Efficient Federated Kernel Support Vector Machine for Edge Devices.

A federated learning (FL) scheme (denoted as Fed-KSVM) is designed to train kernel support vector machines (SVMs) over multiple edge devices with low memory consumption. To decompose the training process of kernel SVM, each edge device first constructs high-dimensional random feature vectors of its local data, and then trains a local SVM model over the random feature vectors. To reduce the memory consumption on each edge device, the optimization problem of the local model is divided into several subproblems. Each subproblem only optimizes a subset of the model parameters over a block of random feature vectors with a low dimension. To achieve the same optimal solution to the original optimization problem, an incremental learning algorithm called block boosting is designed to solve these subproblems sequentially. After training of the local models, the central server constructs a global SVM model by averaging the model parameters of these local models. Fed-KSVM only increases the iterations of training the local SVM models to save the memory consumption, while the communication rounds between the edge devices and the central server are not affected. Theoretical analysis shows that the kernel SVM model trained by Fed-KSVM converges to the optimal model with a linear convergence rate. Because of such a fast convergence rate, Fed-KSVM reduces the communication cost during training by up to 99% compared with the centralized training method. The experimental results also show that Fed-KSVM reduces the memory consumption on the edge devices by nearly 90% while achieving the highest test accuracy, compared with the state-of-the-art schemes.

Identification of optimal locations of adaptive traffic signal control using heuristic methods

Adaptive Traffic Signal Control (ATSC) adjusts signal timings to real-time traffic measurements, increasing operational efficiency within a network. However, ATSC is both expensive to install and operate making it infeasible to deploy at all signalized intersections within a network. This study presents a bi-level optimization framework that applies heuristic methods to identify a limited set of locations for ATSC deployment within an urban network. At the upper-level, the Population Based Incremental Learning (PBIL) algorithm is employed to generate, evaluate, learn, and update different ATSC configurations. The lower-level uses the delay-based Max-Pressure algorithm to simulate the ATSC configuration within a microsimulation platform. The study proposes improvements to the PBIL algorithm by considering constraints on the maximum number of intersections for ATSC deployment and incorporates prior information about the intersection performance (i.e., informed search). Simulation results on the traffic network of State College, PA reveal that the proposed PBIL algorithm consistently outperforms baseline methods that select locations only based on queue-lengths or delays in terms of reducing overall network travel times. The study also reveals that intersections experiencing the highest delays or longest queues are not always the best candidates for ATSC. Moreover, applying ATSC at all intersections does not always provide the best performance; in fact, ATSC applied to some locations could increase travel times by contributing additional congestion downstream. Additionally, the modified PBIL algorithm with the informed search strategy is more efficient at identifying promising solutions suggesting it can be readily applied to more generalized optimization problems.

Decentralized and collaborative machine learning framework for IoT

Decentralized machine learning has recently been proposed as a potential solution to the security issues of the canonical federated learning approach. In this paper, we propose a decentralized and collaborative machine learning framework specially oriented to resource-constrained devices, usual in IoT deployments. With this aim we propose the following construction blocks. First, an incremental learning algorithm based on prototypes that was specifically implemented to work in low-performance computing elements. Second, two random-based protocols to exchange the local models among the computing elements in the network. Finally, two algorithmics approaches for prediction and prototype creation. This proposal was compared to a typical centralized incremental learning approach in terms of accuracy, training time and robustness with very promising results.

Adaptive hierarchical positioning error compensation for long-term service of industrial robots based on incremental learning with fixed-length memory window and incremental model reconstruction

Industrial robots have been extensively used in industry, however, geometric errors mainly caused by connecting rod parameter error and non-geometric errors caused by deflection and friction, etc., limit its application in high-accuracy machining. Aiming at addressing these two types of errors, parametric methods for error compensation based on the kinematic model and non-parametric methods of directly establishing the mapping relationship between the actual and target poses of the robot end-effector are investigated and proposed. Currently both types of methods are mainly offline and will be no longer applicable when the pose of the end-effector in the workspace changes dramatically or the working performance of the robot degrades. Thus, to compensate the positioning error of an industrial robot during long-term operation, this research proposes an adaptive hierarchical compensation method based on fixed-length memory window incremental learning and incremental model reconstruction. Firstly, the correlation between positioning errors and robot poses is studied, a calibration sample library is created, and thus the actively evaluating mechanism of the pose mapping model is established to overcome the problem of the robot’ workspace having a differential distribution of error levels. Then, an incremental learning algorithm with fixed-length memory window and an incremental model reconstruction algorithm are designed to optimize the pose mapping model in terms of its parameters and architecture and overcome the problem that the performance degradation of the robot exacerbates the positioning error and affects the applicability of the pose mapping model, ensuring that the pose mapping model runs stably above the target accuracy level. Finally, the proposed method is applied to the long-term compensation case of a Stäubli industrial robot and a UR robot, and compared to state-of-art methods. Verification results show the proposed method reduces the position error of the Stäubli robot from 0.85mm to 0.13mm and orientation error from 0.68° to 0.07°, as well as reduces the position error of the UR robot from 2.11mm to 0.17mm, demonstrating that the proposed method works in real world scenarios and outperforms similar methods.

Mutual dimensionless improved bearing fault diagnosis based on Bp-increment broad learning system in computer vision

Efficient and accurate diagnosis of rotating machinery in the petrochemical industry is crucial for ensuring normal machinery operation. However, the nonlinear and non-stationary vibration signals generated by machinery in harsh environments pose significant challenges in distinguishing fault signals from normal ones. Although several fault diagnosis methods based on mutual dimensionless indicators (MDI) have been proposed, they often fail to achieve effective and accurate health monitoring. Hence, this paper proposes a BP-Incremental Broad Learning System (BP-INBLS) model based on the combined synergistic of two modules, to address the existing challenges. Firstly, a new mutual dimensionless indicator (VMDI) with high sensitivity and low overlap is refactored. Secondly, leveraging the advantages of incremental learning algorithms, a novel Broad Learning System (BLS) model for quickly identifying different fault types is constructed. Finally, the proposed method is validated using multiple datasets and verified through a comparative analysis with a published method based on dimensionless indicators (DI). The results demonstrate the effectiveness of the proposed method in fault diagnosis.

Transformer-BLS: An efficient learning algorithm based on multi-head attention mechanism and incremental learning algorithms

Due to its efficient model calibration given by unique incremental learning capability, broad learning system (BLS) has made impressive progress in image analytical tasks such as image classification and object detection. Inspired by this incremental remodel success, we proposed a novel transformer-BLS network to achieve a trade-off between model training speed and accuracy. Specially, we developed sub-BLS layers with the multi-head attention mechanism and combining these layers to construct a transformer-BLS network. In particular, our proposed transformer-BLS network provides four different incremental learning algorithms that enable the proposed model can realize the increments of its feature nodes, enhancement nodes, input data and sub-BLS layers, respectively, without the need of the full-weight update in this model. Furthermore, we validated the performance of our transformer-BLS network and its four incremental learning algorithms on a variety of image classification datasets. The results demonstrated that the proposed transformer-BLS maintains classification performance on both the MNIST and Fashion-MNIST datasets, while saving 2/3 of the training time. These findings imply that the proposed method has the potential in significant reducing model training complexity with this incremental remodel system, while simultaneously improving the increment learning performance of the original BLS within such contexts, especially in the classification task of some datasets.

Continual learning classification method with human-in-the-loop

The classification problem is essential to machine learning, often used in fault detection, condition monitoring, and behavior recognition. In recent years, due to the rapid development of incremental learning, reinforcement learning, transfer learning, and continual learning algorithms, the contradiction between the classification model and new data has been alleviated. However, due to the lack of feedback, most classification algorithms take long to search and may deviate from the correct results. Because of this, we propose a continual learning classification method with human-in-the-loop (HCLCM) based on the artificial immune system. HCLCM draws lessons from the mechanism that humans can enhance immune response through various intervention technologies and brings humans into the test learning process in a supervisory role. The human experience is integrated into the test phase, and the parameters corresponding to the error identification data are adjusted online. It enables it to converge to an accurate prediction model at the lowest cost and to learn new data categories without retraining the classifier.•All necessary steps and formulas of HCLCM are provided.•HCLCM adds manual intervention to improve the classification ability of the model.•HCLCM can recognize new types of data.

Incremental Online Machine Learning for Detecting Malicious Nodes in Vehicular Communications Using Real-Time Monitoring

Detecting malicious activities in Vehicular Ad hoc Networks (VANETs) is an important research field as it can prevent serious damage within the network and enhance security and privacy. In this regard, a number of approaches based on machine learning (ML) algorithms have been proposed. However, they encounter several challenges due to data being constantly generated over time; this can impact the performance of models trained on fixed datasets as well as cause the need for real-time data analysis to obtain timely responses to potential threats in the network. Therefore, it is crucial for machine learning models to learn and improve their predictions or decisions in real time as new data become available. In this paper, we propose a new approach for attack detection in VANETs based on incremental online machine learning. This approach uses data collected from the monitoring of the VANET nodes’ behavior in real time and trains an online model using incremental online learning algorithms. More specifically, this research addresses the detection of black hole attacks that pose a significant threat to the Ad hoc On Demand Distance Vector (AODV) routing protocol. The data used for attack detection are gathered from simulating realistic VANET scenarios using the well-known simulators Simulation of Urban Mobility (SUMO) and Network Simulator (NS-3). Further, key features which are relevant in capturing the behavior of VANET nodes under black hole attack are monitored over time. The performance of two online incremental classifiers, Adaptive Random Forest (ARF) and K-Nearest Neighbors (KNN), are assessed in terms of Accuracy, Recall, Precision, and F1-score metrics, as well as training and testing time. The results show that ARF can be successfully applied to classify and detect black hole nodes in VANETs. ARF outperformed KNN in all performance measures but required more time to train and test compared to KNN. Our findings indicate that incremental online learning, which enables continuous and real-time learning, can be a potential method for identifying attacks in VANETs.

An optimized backpropagation neural network models for the prediction of nanomaterials concentration for purification industrial wastewater

In this paper, an optimized backpropagation neural network (BPNN) prediction models have been proposed to estimate the concentration of titanium dioxide (TiO2) nanomaterials required for the removal of Acid Black 24 dye, a type of textile dye, from industrial wastewater. The research gap addressed in this study lies in the investigation of various optimization algorithms for BPNN models and their profound impact on the accuracy of nanomaterial concentration predictions, thus contributing to the advancement of nanotechnology-based wastewater treatment research and its practical applications in industrial settings. Although several studies have predicted the amount of nanomaterial needed to degrade the dyes in industrial wastewater using BPNN, few have investigated the effects of optimized BPNN and how greatly the amount of degradation can vary. Therefore, six different optimization algorithms, including genetic algorithms (GAs), ant colony optimization (ACO), particle swarm optimization (PSO), evaluation strategy (ES), biogeography-based optimization (BBO), and population-based incremental learning algorithm (PBIL), are applied to optimize and evaluate the BPNN models. Experimental data obtained from chemical laboratory experiments are used to train and test the models, with different experiments conducted for various factors like TiO2, H2SO4, NaIO4, K2S2O8, dye, and silver nitrate. The performance of the optimized BPNN models is assessed using metrics such as mean squared error (MSE), root means squared error (RMSE), and R-squared (R2). The results demonstrate the effectiveness of the proposed models, with ACO yielding the best outcomes across all models. The R2 values for the TiO2, H2SO4, NaIO4, and K2S2O8 models after optimization indicate perfect predictions, whereas the R2 values before optimization ranged from 0.888 to 0.984. PSO performs well for most models except for K2S2O8, GA improves most models except for the silver nitrate model, ES enhances only the Dye model, and PBIL does not affect the TiO2 model. Overall, the optimized BPNN models exhibit impressive accuracy in estimating the TiO2 concentration required for removing Acid Black 24 dye from industrial wastewater.

An Incremental Broad-Learning-System-Based Approach for Tremor Attenuation for Robot Tele-Operation.

The existence of the physiological tremor of the human hand significantly affects the application of tele-operation systems in performing high-precision tasks, such as tele-surgery, and currently, the process of effectively eliminating the physiological tremor has been an important yet challenging research topic in the tele-operation robot field. Some scholars propose using deep learning algorithms to solve this problem, but a large number of hyperparameters lead to a slow training speed. Later, the support-vector-machine-based methods have been applied to solve the problem, thereby effectively canceling tremors. However, these methods may lose the prediction accuracy, because learning energy cannot be accurately assigned. Therefore, in this paper, we propose a broad-learning-system-based tremor filter, which integrates a series of incremental learning algorithms to achieve fast remodeling and reach the desired performance. Note that the broad-learning-system-based filter has a fast learning rate while ensuring the accuracy due to its simple and novel network structure. Unlike other algorithms, it uses incremental learning algorithms to constantly update network parameters during training, and it stops learning when the error converges to zero. By focusing on the control performance of the slave robot, a sliding mode control approach has been used to improve the performance of closed-loop systems. In simulation experiments, the results demonstrated the feasibility of our proposed method.

Incremental Machine Learning for Soft Pneumatic Actuators with Symmetrical Chambers

Soft robotics is a specialized field of robotics that focuses on the design, manufacture, and control of robots made of soft materials, as opposed to those made of rigid links. One of the primary challenges for the future use of continuous or hyper-redundant robotics systems in industrial and medical technology is the development of suitable modeling and control approaches. Due to the complex non-linear behavior of soft materials and the unpredictable motion of actuators, the task of modeling complex soft actuators is very time-consuming. As a result, earlier studies have undertaken research into model-free methods for controlling soft actuators. In recent years, machine learning (ML) methods have become widely popular in research. The adaptability of an ML model to a non-linear soft drive system alongside the varying actuation behavior of soft drives over time as a result of material characteristics and performance requirements is the key rationale for including an ML model. The system requires the online updating of the ML model in order to work with the non-linear system. Sequential data collected from the test bench and converted into a hypothesis are used to perform incremental learning. These methods are called lifelong learning and progressive learning. Real-time data flow training is combined with incremental learning (IL), and a neural network model is tuned sequentially for each data input. In this article, a method for the intelligent control of soft pneumatic actuators based on an incremental learning algorithm is proposed. A soft pneumatic actuator was subjected to three distinct test conditions in a controlled test environment for a specified duration of data gathering. Additionally, data were collected through finite element method simulations. The collected data were used to incrementally train a neural network, and the resulting model was analyzed for errors with both training and test data. The training and testing errors were compared for different incremental learning (IL) algorithms, including K-nearest neighbors, a decision tree, linear regression, and a neural network. The feasibility of the modulo-free intelligent control of soft pneumatic actuators based on an incremental learning algorithm was verified, solving the problem of the control of software actuators.

Visual Analytics on Network Forgetting for Task‐Incremental Learning

AbstractTask‐incremental learning (Task‐IL) aims to enable an intelligent agent to continuously accumulate knowledge from new learning tasks without catastrophically forgetting what it has learned in the past. It has drawn increasing attention in recent years, with many algorithms being proposed to mitigate neural network forgetting. However, none of the existing strategies is able to completely eliminate the issues. Moreover, explaining and fully understanding what knowledge and how it is being forgotten during the incremental learning process still remains under‐explored. In this paper, we propose KnowledgeDrift, a visual analytics framework, to interpret the network forgetting with three objectives: (1) to identify when the network fails to memorize the past knowledge, (2) to visualize what information has been forgotten, and (3) to diagnose how knowledge attained in the new model interferes with the one learned in the past. Our analytical framework first identifies the occurrence of forgetting by tracking the task performance under the incremental learning process and then provides in‐depth inspections of drifted information via various levels of data granularity. KnowledgeDrift allows analysts and model developers to enhance their understanding of network forgetting and compare the performance of different incremental learning algorithms. Three case studies are conducted in the paper to further provide insights and guidance for users to effectively diagnose catastrophic forgetting over time.

Development and research of a neural network alternate incremental learning algorithm

In this paper, the relevance of developing methods and algorithms for neural network incremental learning is shown. Families of incremental learning techniques are presented. A possibility of using the extreme learning machine for incremental learning is assessed. Experiments show that the extreme learning machine is suitable for incremental learning, but as the number of training examples increases, the neural network becomes unsuitable for further learning. To solve this problem, we propose a neural network incremental learning algorithm that alternately uses the extreme learning machine to correct the only output layer network weights (operation mode) and the backpropagation method (deep learning) to correct all network weights (sleep mode). During the operation mode, the neural network is assumed to produce results or learn from new tasks, optimizing its weights in the sleep mode. The proposed algorithm features the ability for real-time adaption to changing external conditions in the operation mode. The effectiveness of the proposed algorithm is shown by an example of solving the approximation problem. Approximation results after each step of the algorithm are presented. A comparison of the mean square error values when using the extreme learning machine for incremental learning and the developed algorithm of neural network alternate incremental learning is made.

DeepWE: A Deep Bayesian Active Learning Waypoint Estimator for Indoor Walkers

Waypoint Estimation (WE) has a wide range of applications for indoor walkers, such as fire rescue, and navigation to find exit doors, lifts, or stairs as examples of waypoints, etc. Data-driven waypoint estimation has been on the rise with advancements in deep learning algorithms. The current waypoint estimation methods, however, face two challenges. On one hand, most waypoint detection approaches rely on visual sensors, hence, their estimation performance is limited by light when collecting visual data. On the other hand, data-driven methods necessitate a large number of labeled data to train a WE model, which significantly increases the time spent manually marking labels. Targeting the above two challenges, our work first proposes a novel Deep Bayesian Active Learning Waypoint Estimator for indoor walkers (DeepWE) based on Human Activity Recognition (HAR). This estimates six indoor waypoints through walkers daily activities due to the strong correlation between human activities and waypoints. Firstly, an initial DeepWE model is developed using a Bayesian ensembled Convolutional Neural Network (B-CNN) using the accelerometer and gyroscope data. Then, active learning is employed to query the most formative samples from pool points with four acquisition functions, and only these queried samples are labeled manually. Finally, the initial DeepWE model is updated from this labeled data using an incremental learning algorithm. Empirical results on two publicly available USC-HAD and OPPORTUNITY datasets show DeepWE performs a considerable accuracy boost for waypoint estimation, with a substantial amount of acquired pool points reduction (more than 40%)

Self-Paced Broad Learning System.

Broad learning system (BLS), an efficient neural network with a flat structure, has received a lot of attention due to its advantages in training speed and network extensibility. However, the conventional BLS adopts the least square loss, which treats each sample equally and thus is sensitivity to noise and outliers. To address this concern, in this article we propose a self-paced BLS (SPBLS) model by incorporating the novel self-paced learning (SPL) strategy into the network for noisy data regression. With the assistance of the SPL criterion, the model output is used as feedback to learn appropriate priority weight to readjust the importance of each sample. Such a reweighting strategy can help SPBLS to distinguish samples from "easy" to "difficult" in model training, equipping the model robust to noise and outliers while maintaining the characteristics of the original system. Moreover, two incremental learning algorithms associated to SPBLS have also been developed, with which the system can be updated quickly and flexibly without retraining the entire model when new training samples are added or the network needs to be expanded. Experiments conducted on various datasets demonstrate that the proposed SPBLS can achieve satisfying performance for noisy data regression.

Efficient and Effective Cardinality Estimation for Skyline Family

Cardinality estimation, predicting the query result size, is a fundamental problem in databases. Existing skyline cardinality estimation methods are computationally infeasible for massive skyline queries over the large-scale database. In this paper, we introduce a unified skyline family w.r.t. various skyline variants. We propose an efficient and effective skyline family cardinality estimation model, named EECE, in an end-to-end manner. EECE consists of two modules, unsupervised data distribution learning (DDL) and supervised monotonic cardinality estimation (MCE). DDL leverages the mixture data guided transformer to learn the distribution of database and query parameters for model pre-training. MCE further incorporates supervised learning and parameter clamping to enhance the estimation under monotonicity guarantees. We develop an efficient incremental learning algorithm for EECE to adapt the database and query logs update. Extensive experiments on several real-world and synthetic datasets demonstrate that, EECE speeds up the cardinality estimation by six orders of magnitude, with more than 39% accuracy gain, compared to the state-of-the-art approaches.

SoC-based real-time SVM classification with integrated training using HLS and PYNQ

Support Vector Machines (SVM) are widely used techniques in the field of classification problems because of their ability to effectively deal with datasets that have complex non-linear structures and a high dimensionality. The compute-intensive training algorithm associated with SVM makes it challenging to keep an up-to-date model that accurately reflects the characteristics of newly arriving data points in real-time systems. This paper proposes a novel training algorithm for incremental learning from large datasets, based on a variant of Sequential Minimal Optimization (SMO). High-Level Synthesis (HLS) was used for implementing the Field Programmable Gate Array (FPGA) based Intellectual Property (IP) Core, which includes the computationally intensive kernel computation portion of the training algorithm. In addition to the kernel computation, the inference phase of the SVM classifier is built into the IP core, and its use can be switched on the fly. The computational latency and memory bandwidth of an IP core are optimized using loop pipelining and DMA burst data transfer. With the help of hardware/software co-design, the IP core is integrated into the design of a flexible and re-usable System on Chip (SoC) called PYNQ Overlay. The experiments show that the overlay outperforms the embedded processor, multiple hardware SVM classifiers, and hardware accelerated Convolutional Neural Networks (CNN) in terms of real-time efficiency. The Overlay makes much less use of the resources available on the chip in comparison to the majority of the CNN accelerators. The overlay achieves an average classification accuracy that is only 1% lower than that of an ARM Cortex-A9 processor, according to experimental results on six datasets. Furthermore, it can increase training speed by an average of 31.82x and inference speed by an average of 31.74x. In addition, the proposed Overlay design achieves a 2.3x improvement in average training speed, as measured in Mega bits per second, compared to existing SVM training implementations, along with incremental learning and multi-class classification support.