Incremental Support Vector Machine Research Articles

Support Vector Machines With Uncertainty Option and Incremental Sampling for Kriging

ABSTRACTThis paper presents a novel approach to pollution assessment by investigating support vector machines (SVM) with an uncertainty option to overcome the limitations of traditional kriging. While kriging is a major tool for geostatistical modelling, allowing to estimate the distribution of contaminants in a region from a small set of samples, it does not allow to extract also the uncertainty map. An uncertainty map is of great interest, as it allows to identify regions of high uncertainty where one should sample in order to reduce high level of uncertainties. In this paper, we propose two variants of the SVM with an uncertainty option, each using a different hinge loss to improve the accuracy and efficiency. These losses allow to estimate different levels of contaminations, as well as uncertainty, such as the three levels: positive, uncertain and negative, namely for pollution estimation: high‐pollution, uncertain and low‐pollution. In addition to the exploration of SVM variants, we propose an innovative active sample selection strategy based on the uncertainty criterion. This strategy is designed to systematically reduce uncertainties in pollution assessment, thus providing adaptability to dynamic environmental changes. An incremental SVM with an uncertainty option is introduced to further optimise the sample selection process. Furthermore, the decision‐making process is refined through the introduction of a novel three‐hinge loss. The corresponding optimization problem and its resolution allow for a more nuanced contamination assessment with multiple levels of estimation, providing a valuable tool for characterising contamination levels with increased granularity. Extensive experiments on synthetic and real data validate the proposed methodology. Synthetic data simulations assess the quality of the approach, while real data from a two‐dimensional porosity measurement demonstrate practical applicability. This research contributes to the advancement of pollution assessment methodologies, providing an adaptable solution for environmental monitoring.

Feature Extraction and Classification of Power Quality Disturbances Using Optimized Tunable-Q Wavelet Transform and Incremental Support Vector Machine

The widespread integration of renewable energy sources (RESs) into power systems using power electronics-based interface devices has led to a substantial rise in power quality (PQ) issues. There is an immediate requirement for effective monitoring, detection, and classification of power quality disturbances (PQDs) that is needed to take remedial measures and design planning of the system architecture. This study presents a hybrid approach with an objective for the feature extraction and classification of PQDs. The proposed hybrid approach is comprised of an optimized tunable-Q wavelet transform (OTQWT) for the feature extraction and incremental support vector machine (ISVM). A four-stage approach is suggested for the PQ detection and classification in this study. In the first stage, the various data are retrieved both in the form of synthetic data by mathematical formulations and real-time data with prototype design setup. In the second stage, regardless of the specified wavelet function, the PQD signals are decomposed into low-pass and high-pass sub-bands using the tunable-Q wavelet transform (TQWT). However, the utilization of default decomposition parameters to address nonstationary PQ signals may lead to information loss and reduced performance of the system. To avoid this limitation, an OTQWT as an enhanced technique to TQWT based on an Adaptive Particle Swarm Optimization (APSO) is suggested. A modified objective function based on the mean square error (MSE) is used to improve the decomposition process. In the third stage, an efficient classifier is suggested based on the ISVM. Lastly, to test and evaluate the performance of the proposed approach, twelve types of PQDs including noise and multiple occurrences are considered. The comparative analysis with other popular methods reflects the better performance of the proposed approach and justifies its use for PQ detection and classification purposes in real-time​ conditions.

Developing a hybrid technique for energy demand forecasting based on optimized improved SVM by the boosted multi-verse optimizer: Investigation on affecting factors

Electricity demand prediction accuracy is crucial for operational energy resource management and strategy. In this study, we provide a multi-form model for electricity demand prediction in China that based on incorporating of an upgraded Support Vector Machine (SVM) and a Boosted Multi-Verse Optimizer (BMVO). The suggested model is proposed to address the shortcomings of existing prediction approaches, which frequently fail to internment the complicated nonlinear interactions between demand for electricity and the variables that influence it. The improved SVM algorithm incorporates a modified genetic algorithm based on kernel function for enhancing the stability of the model. The BMVO technique is employed to optimize the combined model's weights and increase its generalization effectiveness. The suggested approach is tested by real-world Chinese energy demand data. The findings show that it outperforms existing prediction approaches in terms of reliability and precision. Further, the number of samples chosen affects how well the proposed BMVO linked with the Incremental SVM (ISVM) predicts outcomes. Particularly, when 1735 samples are chosen, the lowest level of Mean Absolute Percent Error (MAPE) was noted. The Root Mean Square Error (RMSE) and MAPE values under the proposed BMVO/ISVM model are reduced by 53.72% and 55.22%, respectively, compared to the Artificial Neural Network (ANN) approach reported in literature. Finally, the suggested model is capable of accurately predicting the electricity demand in China and has the potential to be applied to other energy-demand prediction applications.

Enhancing handwritten numeric string recognition through incremental support vector machines

Handwritten digit recognition systems are integral to diverse applications such as postal services, banking, and document processing in our digitally-driven society. This research addresses the challenges posed by evolving datasets and dynamic scenarios in handwritten digit recognition by proposing an approach based on incremental support vector machines (ISVM). ISVM is an extension of traditional support vector machines (SVM) designed to handle scenarios where new data points become available over time. The dataset includes handwritten images (numbers “0” to “6”) and trials introducing new classes (“7”, “8”, and “9”). Evaluation utilizes k-fold cross-validation for robustness. Digital image processing involves converting images into numeric data using the histogram method. The result showed the positive outcomes of using ISVM in handwritten digit recognition, emphasizing its adaptability to incremental learning and its ability to maintain robust performance in the face of evolving datasets, which is crucial for real-world applications.

Deep Error-Correcting Output Codes

Ensemble learning, online learning and deep learning are very effective and versatile in a wide spectrum of problem domains, such as feature extraction, multi-class classification and retrieval. In this paper, combining the ideas of ensemble learning, online learning and deep learning, we propose a novel deep learning method called deep error-correcting output codes (DeepECOCs). DeepECOCs are composed of multiple layers of the ECOC module, which combines several incremental support vector machines (incremental SVMs) as base classifiers. In this novel deep architecture, each ECOC module can be considered as two successive layers of the network, while the incremental SVMs can be viewed as weighted links between two successive layers. In the pre-training procedure, supervisory information, i.e., class labels, can be used during the network initialization. The incremental SVMs lead this procedure to be very efficient, especially for large-scale applications. We have conducted extensive experiments to compare DeepECOCs with traditional ECOC, feature learning and deep learning algorithms. The results demonstrate that DeepECOCs perform, not only better than existing ECOC and feature learning algorithms, but also related to deep learning ones in most cases.

Error compensation method of industrial robots considering non-kinematic and weak rigid base errors

Industrial robots are receiving increasing attention owing to their numerous advantages, such as flexibility, maneuverability, and low cost. In robotic machining systems, improving the absolute positioning accuracy of the robot is essential for improving workpiece quality. This study proposes a robot error compensation method that considers nonkinematic and weak rigid base errors. A mapping model between the theoretical position of the robot and the actual error was established based on an incremental support vector machine (ISVM), and the nonkinematic error compensation of the robot was realized. The robot error before and after axis 1 rotation is compensated for by modeling the rotational error of axis 1 and the deformation error of the robot base and is verified on the milling robot KUKA KR 160. The experimental results show that the maximum absolute position error of the robot end-effector is reduced by 85.27% (from 1.3277 to 0.1956 mm) after compensation. Milling experiments have shown that the position error of the empty running trajectory before compensation is reduced from −1.3231 to −0.1297 mm, and the z-direction error of the workpiece is reduced from −0.8771 to 0.3267 mm, which meets the machining accuracy requirements of the milling robot system in this paper.

Adaptive SVM-based pixel accumulation technique for a SPAD-based lidar system.

This paper proposes an adaptive support vector machine (SVM)-based pixel accumulation technique for the single-photon avalanche diode (SPAD)-based flash lidar system to greatly improve the accuracy of depth images. In this method, an adaptive incremental SVM classifier is proposed to distinguish the target and background to avoid the boundary blur caused by accumulation. With the SVM classifier, only the pixels belonging to the same target are accumulated. Then by accumulating the multiple measurements of the results of neighboring pixels, the influence of noisy photons can be reduced, and an accurate peak position of the laser signal in the histogram generated by the SPAD detector can be derived. Different from most existing classification algorithms in which the SVM model is used to recognize the target in depth, the image is trained with huge amounts of offline data, which neglect the ability to handle the changes of target's features by building a small subset of input data that contains all needed information of the trained SVM model. The new method could quickly update itself with the motion of target or light intensity changes online and definitely enhance the practicability of the proposed depth image-processing method. Experimental results show that using the proposed adaptive pixel accumulation technique, the mean squared error of depth images can be reduced by 61% compared with raw images and 44% compared with the static SVM-based method when the target moves.

An ISVM Algorithm Based on High-Dimensional Distance and Forgetting Characteristics

In the face of the batch, dynamic access data, or the flow of data that continuous changes over time, the traditional support vector machine algorithm cannot dynamically adjust the previous classification model. To overcome this shortcoming, the incremental support vector machine (ISVM) algorithm is proposed. However, many incremental support vector algorithms still have shortcomings such as low efficiency, memory limitation, and poor generalization. This paper puts forward the new ISVM algorithm, HDFC-ISVM ∗ algorithm, based on the high-dimensional distance and forgetting characteristics. This paper firstly proposes the original HDFC-ISVM algorithm that first learns the distribution characteristics of the samples according to the distance between the samples and the normative hyperplane. Then, it introduces the forgetting factor. In the incremental learning process, the classifier gradually accumulates the spatial distribution knowledge of samples, eliminates the samples that have no contributions to the classifier, and selectively forgets some useless samples according to the forgetting factor, which overcomes the shortcomings such as low efficiency and poor accuracy of some algorithms. But, the original HDFC-ISVM algorithm is sensitive to parameters, and different settings of the parameters have a great impact on the final classification accuracy of the algorithm. Therefore, on the basis of the original algorithm, an improved algorithm HDFC-ISVM ∗ based on the adjustments to the initialization strategy and updating rules of the forgetting factor is proposed. The initialization strategy and updating rules of the forgetting factor are adjusted to adapt datasets with different distributions in this improved algorithm. The rationality of the improved strategy about the forgetting factor is discussed theoretically. At the same time, the proposed algorithm has better classification accuracy, classification efficiency, and better generalization ability than other algorithms, which is verified by experiments.

A Kalman filter-based prediction strategy for multiobjective multitasking optimization

Multiobjective multitasking optimization (MO-MTO) can solve multiple optimization tasks simultaneously through knowledge transfer across tasks. However, how to design an efficient knowledge transfer method is the main challenge. Keeping this in mind, this paper proposes an evolutionary multitasking algorithm based on Kalman filter prediction strategy. Specifically, the incremental support vector machine classifier is used to find valuable solutions. Moreover, the Kalman filter prediction strategy is designed to utilize valuable solutions and historical evolutionary information to estimate the predictive solutions. Finally, the scoring scheme is constructed to adaptively select valuable solutions and predictive solutions as transfer knowledge. Experimental results on three MO-MTO test suites demonstrate that the proposed algorithm can achieve competitive performance.

Using Domain Adaptation for Incremental SVM Classification of Drift Data

A common assumption in machine learning is that training data is complete, and the data distribution is fixed. However, in many practical applications, this assumption does not hold. Incremental learning was proposed to compensate for this problem. Common approaches include retraining models and incremental learning to compensate for the shortage of training data. Retraining models is time-consuming and computationally expensive, while incremental learning can save time and computational costs. However, the concept drift may affect the performance. Two crucial issues should be considered to address concept drift in incremental learning: gaining new knowledge without forgetting previously acquired knowledge and forgetting obsolete information without corrupting valid information. This paper proposes an incremental support vector machine learning approach with domain adaptation, considering both crucial issues. Firstly, a small amount of new data is used to fine-tune the previous model to generate a model that is sensitive to the new data but retains the previous data information by transferring parameters. Secondly, an ensemble and model selection mechanism based on Bayesian theory is proposed to keep the valid information. The computational experiments indicate that the performance of the proposed model improved as new data was acquired. In addition, the influence of the degree of data drift on the algorithm is also explored. A gain in performance on four out of five industrial datasets and four synthetic datasets has been demonstrated over the support vector machine and incremental support vector machine algorithms.

IMCOD: Incremental multi-class outlier detection model in data streams

Multi-class outlier detection from multi-class data streams is a crucial yet challenging task for many real-life applications and services. Various detection methods are available e.g., derivation from one-class classification algorithms, prior knowledge of the underlying data distribution, method-specific hyperparameters tuning for different data sets. However, almost all of these methods are designed to work in the batch learning mode. Therefore, it is difficult to apply the existing methods to multi-class data stream applications. To overcome these limitations, this study proposes a new incremental multi-class outlier detection model (iMCOD). Unlike previous approaches, iMCOD incorporates an incremental support vector machine (iSVM) with an incremental local outlier factor (iLOF) in a unified framework. The incoming query sample was classified using iSVM, and the iLOF score was computed using the newly developed class-based nearest neighbor (NN) concept. Employing the class-based median of the NN average deviation metric, we determined whether the query sample is an outlier, and the iSVM was updated accordingly for the next sample. The iMCOD can detect outliers from multi-class data streams without the need for hyperparameter tuning. Extensive experimental and statistical analyses on 15 real-world data sets demonstrated that iMCOD outperformed 13 competitors.

Incremental learning of upper limb action pattern recognition based on mechanomyography

Aiming at the problem that the traditional mechanomyography (MMG) pattern recognition model for upper limb rehabilitation action has poor recognition effect on non-identical distribution test data, this research proposes an incremental learning method for deep learning based on MMG. By collecting 8-channels of MMG of 4 types of hand movements (wrist flexion, wrist extension, wrist ulnar flexion, and wrist radial flexion), after signal preprocessing, feature extraction and dimensionality reduction, 12 groups of non-identical distributed data were obtained. The model was trained by using deep neural network (DNN), and five commonly used machine learning algorithms were used as comparison for incremental training. Finally, the recognition rate of DNN was 88.25%, and the final recognition rates of Passive Aggressive (PA), Incremental Support Vector Machine (ISVM), Perceptron, Bernoulli Naive Bayes (BNB) and Multinomial Naive Bayes (MNB) were 85.94%, 84.82%, 81.01%, 73.07% and 61.20% respectively. Among them, both DNN and PA had better upward trends, and DNN had the highest final recognition rate. By comparing the confusion matrices before and after DNN incremental learning, it could be seen that DNN incremental training can significantly improve the accuracy and precision in confusion matrix. The experimental results demonstrate that the method of incremental learning can not only achieve the recognition of non-identically distributed test data, but also improve the recognition rates and the generalization performance of the model.

An Online Prediction Approach Based on Incremental Support Vector Machine for Dynamic Multiobjective Optimization

Real-world multiobjective optimization problems usually involve conflicting objectives that change over time, which requires the optimization algorithms to quickly track the Pareto-optimal front (POF) when the environment changes. In recent years, evolutionary algorithms based on prediction models have been considered promising. However, most existing approaches only make predictions based on the linear correlation between a finite number of optimal solutions in two or three previous environments. These incomplete information extraction strategies may lead to low prediction accuracy in some instances. In this article, an incremental support vector machine (ISVM)-based dynamic multiobjective evolutionary algorithm, in short called ISVM-DMOEA, is proposed. We treat the solving of dynamic multiobjective optimization problems (DMOPs) as an online learning process, using the continuously obtained optimal solution to update an ISVM without discarding the solution information at earlier time. ISVM is then used to filter random solutions and generate an initial population for the next moment. To overcome the obstacle of insufficient training samples, a synthetic minority oversampling strategy is implemented before the training of ISVM. The advantage of this approach is that the nonlinear correlation between solutions can be explored online by ISVM, and the information contained in all historical optimal solutions can be exploited to a greater extent. The experimental results and comparison with the chosen state-of-the-art algorithms demonstrate that the proposed algorithm can effectively tackle DMOPs.

Acoustic-based whistle detection of drain hole for wind turbine blade

The blade is a crucial part of large-scale wind turbine for converting wind energy into electricity. Increasing attentions have been paid to the blade health monitoring in recent years. This paper focuses on acoustic-based surface damage detection of the blade and presents a novel intelligent detection method of the whistle produced by the drain hold using the pattern recognition. In the algorithm, a developed preprocessing strategy with multiband adaptive spectral subtraction can well reduce the random wind noise from raw acoustic signal. Moreover, a correlated empirical mode decomposition method in combination with morphological filtering is proposed for extracting time–frequency ridge features of whistle event. Finally, an incremental support vector machine based on adaptive reserved set strategy is designed for recognizing the whistle event. Experimental results demonstrate that proposed method is feasible and effective in whistle detection of drain hole.

Hybrid Online Model for Predicting Diabetes Mellitus

Modern healthcare systems have become smart by synergizing the potentials of wireless sensors, the medical Internet of things, and big data science to provide better patient care while decreasing medical expenses. Large healthcare organizations generate and accumulate an incredible volume of data continuously. The already daunting volume of medical information has a massive amount of diagnostic features and logged details of patients for certain diseases such as diabetes. Diabetes mellitus has emerged as along-haul fatal disease across the globe and particularly in developing countries. Exact and early diagnosis of diabetes from big medical data is vital for the deterrence of disease and the selection of proper therapy. Traditional machine learning-based diagnosis systems have been initially established as offline (non-incremental) approaches that are trained with a pre-defined database before they can be applied to handle prediction problems. The major objective of the proposed work is to predict and classify diabetes mellitus by implementing a Hybrid Online Model for Early Detection of diabetes disease (HOMED) using machine learning algorithms. Our proposed online (incremental) diabetes diagnosis system exploits (i) an Adaptive Principal Component Analysis (APCA) technique for missing value imputation, data clustering, and feature selection; and (ii) an enhanced incremental support vector machine (ISVM) for classification. The efficiency of HOMED is estimated on different performance metrics such as accuracy, precision, specificity, sensitivity, positive predictive value, and negative predictive value. Experimental results on Pima Indian diabetes dataset (768 samples: 500 non diabetic and 268 diabetic patients) reveal that HOMED considerably increases the classification accuracy and decreases computational complexity with respect to the offline models. The proposed system can assist healthcare professionals as a decision support system.

Incremental Fisher linear discriminant based on data denoising

In this article we consider Incremental Fisher linear discriminant (IFLD) based on data denoising. The data denoising is completed by Markov sampling such that the generated non-noise sample sequence is an uniformly ergodic Markov chain (u.e.M.c.). We first establish the generalization bounds of IFLD with u.e.M.c. samples, and prove that the IFLD algorithm with u.e.M.c. samples is consistent. We also present two new IFLD classification algorithms based on Markov sampling, IFLD based on Markov sampling (IFLD-MS) and improved IFLD based on Markov sampling (IIFLD-MS). Experimental results of benchmark repository suggest that IFLD-MS and IIFLD-MS have better performance than the classical IFLD, the incremental support vector machine (ISVM) and other IFLD algorithms.

Improving Vulnerability Inspection Efficiency Using Active Learning

Software engineers can find vulnerabilities with less effort if they are directed towards code that might contain more vulnerabilities. HARMLESS is an incremental support vector machine tool that builds a vulnerability prediction model from the source code inspected to date, then suggests what source code files should be inspected next. In this way, HARMLESS can reduce the time and effort required to achieve some desired level of recall for finding vulnerabilities. The tool also provides feedback on when to stop (at that desired level of recall) while at the same time, correcting human errors by double-checking suspicious files. This paper evaluates HARMLESS on Mozilla Firefox vulnerability data. HARMLESS found 80, 90, 95, 99 percent of the vulnerabilities by inspecting 10, 16, 20, 34 percent of the source code files. When targeting 90, 95, 99 percent recall, HARMLESS could stop after inspecting 23, 30, 47 percent of the source code files. Even when human reviewers fail to identify half of the vulnerabilities (50 percent false negative rate), HARMLESS could detect 96 percent of the missing vulnerabilities by double-checking half of the inspected files. Our results serve to highlight the very steep cost of protecting software from vulnerabilities (in our case study that cost is, for example, the human effort of inspecting 28,750 × 20% = 5,750 source code files to identify 95 percent of the vulnerabilities). While this result could benefit the mission-critical projects where human resources are available for inspecting thousands of source code files, the research challenge for future work is how to further reduce that cost. The conclusion of this paper discusses various ways that goal might be achieved.

Fast and robust supervised machine learning approach for classification and prediction of Parkinson’s disease onset

ABSTRACT Parkinson’s disease (PD) is an incurable long-term neurodegenerative disorder that mainly influence the motor system and eventually results in significant morbidity. The use of computational tools for classification and prediction of PD is desirable for detecting the symptoms at the onset. In this paper, we developed a novel machine learning technique based on incremental support vector machine and modified Frank-Wolfe method (SVM-MFW) for classification and prediction of PD onset. The proposed model works on heterogeneous data gathered from the Parkinson’s progression markers initiative (PPMI) data repository. The work when compared to earlier state-of-the-art techniques outplays them in terms of both classification and prediction. It is shown that the model achieves an equivalent accuracy of 98.3% and a low cross-entropy value in less computational time.

An evolutional SVM method based on incremental algorithm and simulated indicator diagrams for fault diagnosis in sucker rod pumping systems

Fault diagnosis of the sucker rod pumping system (SRPS) is a challenging problem in the oil industry. Recently, the computer-aided indicator diagram (ID) recognition techniques are becoming useful measurements to help engineers monitor the wells. However, the slow variation of SRPS state makes it difficult for a well to experience all the fault conditions. Using the IDs collected from other oil wells as training data will lead to a large difference between the distribution of training data and target data because of the interference information including noise and system operation characteristics in IDs. Moreover, the different occurrence probability of each fault leads to the imbalance of training data, therefore the fault tolerance of the diagnosis models will result in lower accuracy in identifying the fault IDs with less training samples. In addition, due to one-off training, the parameters of diagnostic model are always fixed resulting in lack of adaptive ability. To address this issue, an evolutional support vector machine (SVM) method for SRPS diagnosis is proposed. First, to obtain balanced training data with similar distribution to the target data, a novel model describing the working process of SRPS under various fault conditions is established. Subsequently, the static apparent stiffness feature and its extraction algorithm is proposed to fully retain the fault information of SRPS in IDs. Then, the incremental SVM is used to construct the diagnosis model based on generated IDs. At last, the proposed method is verified experimentally through the system parameters and IDs of many wells collected from oilfields, and then some conventional techniques are employed in the comparison studies. The obtained results show that the accuracy of the diagnosis model trained by simulated IDs is 11.2% higher than that trained by collected IDs. Besides, the parameters of proposed diagnosis model can be continuously evolved to improve the diagnosis accuracy and generalization ability. Furthermore, the proposed diagnosis model also has advantages in training efficiency and memory consumption.

Efficiently Updating ECG-Based Biometric Authentication Based on Incremental Learning

Recently, the interest in biometric authentication based on electrocardiograms (ECGs) has increased. Nevertheless, the ECG signal of a person may vary according to factors such as the emotional or physical state, thus hindering authentication. We propose an adaptive ECG-based authentication method that performs incremental learning to identify ECG signals from a subject under a variety of measurement conditions. An incremental support vector machine (SVM) is adopted for authentication implementing incremental learning. We collected ECG signals from 11 subjects during 10 min over six days and used the data from days 1 to 5 for incremental learning, and those from day 6 for testing. The authentication results show that the proposed system consistently reduces the false acceptance rate from 6.49% to 4.39% and increases the true acceptance rate from 61.32% to 87.61% per single ECG wave after incremental learning using data from the five days. In addition, the authentication results tested using data obtained a day after the latest training show the false acceptance rate being within reliable range (3.5–5.33%) and improvement of the true acceptance rate (70.05–87.61%) over five days.