Online Classification Research Articles

Online Passive-Aggressive Multilabel Classification Algorithms.

Most existing multilabel classification methods are batch learning methods, which may suffer from expensive retraining costs when dealing with new incoming data. In order to overcome the drawbacks of batch learning, we develop a family of online multilabel classification algorithms, which can update the model instantly and efficiently, and make a timely online prediction when new data arrive. Our algorithms all take a closed-form update, which is obtained by solving a constrained optimization problem in each round of online learning. Label correlation is explicitly modeled in our optimization problem. The label thresholding function, an important component of our online classifier, can also be learned online. Our algorithms can be easily generalized to the nonlinear prediction cases using Mercer kernels. The worst case loss bounds for our algorithms are provided. The bounds are relative to the cumulative loss suffered by the best fixed predictive model that can be attained in hindsight. Finally, we corroborate the merits of our algorithms in both linear and nonlinear predictions on nine open multilabel benchmark datasets.

Fast online classification of network traffic using new feature-embedded hierarchical structure

The fast online classification (FOC) of network traffic plays a critical role in the network resource management and quality of service support. However, traditional network flow features result in poor performance in FOC (with fewer packets). To tackle the issue, this study proposes two new features: (1) The conditional frequency of packet size (PSize), for which the PSize is quantized into several equal bins and the PSize-level conditional frequency of two consecutive packets is calculated; (2) The statistical feature of rate sequence that is obtained by dividing the inter-arrival time into the PSize sequences. Due to the real-time requirement of online classification, we analyze the time complexity of flow feature calculation and attempt to balance the classification speed and the accuracy in feature selection by reducing the feature dimensionality. In addition, a new feature-embedded hierarchical classification structure is developed for the scenario in which the network video traffic accounts for a relatively large proportion. Fewer packets are used in the early stage of binary classification of non-video vs. video, and then, the subsequent data packets are employed for the fine-grained classification of their respective flows. The effectiveness of the proposed method is evaluated on two real-world network datasets, and our method is compared with the state-of-the-art methods in terms of time performance, resource usage, and classification accuracy. The experimental results confirm the superiority of our approach in fast online classification.

Adversarial Laws of Large Numbers and Optimal Regret in Online Classification

Adversarial Laws of Large Numbers and Optimal Regret in Online Classification

Active Object Learning for intelligent social robots

The applications of social robots have continued to rise in number over the years: there are now mainly used in nursing homes and hospitals, performing different tasks like assistance, physical and cognitive rehabilitation, and supervision. However, robots’ ability to learn from humans remains a problem to be solved. This paper presents the development of an Active Object Learning architecture in which the robot can identify unknown objects and autonomously initiate the learning process, supported by user interactions. We study the feasibility of generating a real-time dataset, including techniques for reducing image repetition and online classification model training. The work considers the analysis of machine learning metrics such as F1-Score, accuracy, confidence, and hit rate as the robot learns new objects. The accuracy of the classifier remains constant as the number of classes increases, indicating that the regularisation methods reduce overfitting and decrease the generalisation gap. The results prove that the robot can perform the classifications correctly and differentiate between known and unknown objects. The paper also includes a case study to demonstrate the feasibility of the proposed system integrated into a real social robot.

Influence of nuisance variables on the PMU-based disturbance classification in power transmission systems

Abstract The online classification of grid disturbances in power transmission systems has been investigated since many years and shows promising results on measured and simulated PMU signals. Nonetheless, a practical deployment of machine learning techniques is still challenging due to robustness problems, which may lead to severe misclassifications in the model application. This paper formulates an advanced evaluation procedure for disturbance classification methods by introducing additional measurement noise, unknown operational points, and unknown disturbance events in the test dataset. Based on preliminary work, Siamese Sigmoid Networks are used as classification approach and are compared against several benchmark models for a simulated power transmission system at 400 kV. Different test scenarios are proposed to evaluate the disturbance classification models assuming a limited and full observability of the grid with PMUs.

Online Passive-Aggressive Active Learning for Trapezoidal Data Streams.

The idea of combining the active query strategy and the passive-aggressive (PA) update strategy in online learning can be credited to the PA active (PAA) algorithm, which has proven to be effective in learning linear classifiers from datasets with a fixed feature space. We propose a novel family of online active learning algorithms, named PAA learning for trapezoidal data streams (PAA [Formula: see text] ) and multiclass PAA [Formula: see text] (MPAA [Formula: see text] ) (and their variants), for binary and multiclass online classification tasks on trapezoidal data streams where the feature space may expand over time. Under the context of an ever-changing feature space, we provide the theoretical analysis of the mistake bounds for both PAA [Formula: see text] and MPAA [Formula: see text] . Our experiments on a wide variety of benchmark datasets have confirm that the combination of the instance-regulated active query strategy and the PA update strategy is much more effective in learning from trapezoidal data streams. We have also compared PAA [Formula: see text] with online learning with streaming features (OL [Formula: see text] )-the state-of-the-art approach in learning linear classifiers from trapezoidal data streams. PAA [Formula: see text] could achieve much better classification accuracy, especially for large-scale real-world data streams.

ADAPTIVE DOUBLE NEO-FUZZY NEURON AND ITS COMBINED LEARNING

The subject of the study in the article is the process of data classification under conditions of fuzziness and a limited volume of training sample. The goal is to enhance the double neo-fuzzy neuron within the framework of solving the data classification task with constraints on the training sample volume, processing time, as well as fuzziness and nonstationarity of input data. The tasks include improving the double neo-fuzzy neuron to enhance the system's approximation properties and developing a combined system learning method to ensure fast performance in an online mode. The approaches used are lazy learning, supervised learning, and self-learning. The following results have been obtained: the double neo-fuzzy neuron has been modified by introducing a compressive activation function at the output, creating conditions for building a neo-fuzzy network capable of adapting to non-stationary input data in an online mode and avoiding the vanishing gradient problem. Conclusion. A combined learning method for the double neo-fuzzy neuron has been proposed, involving parallel utilization of lazy learning, supervised learning, and self-learning with the "Winner Takes All" rule, followed by automatic formation of membership functions, enabling fast online classification in the presence of outliers in the input data.

Fuzzy Decision Rule-Based Online Classification Algorithm in Fuzzy Formal Decision Contexts

Formal concept analysis (FCA) extracts interpretable rules by using implication relationships between concepts, which is an effective method for acquiring knowledge. In this paper, a fuzzy decision rule-based online classification algorithm called OFRCA is designed by combining online learning theory and FCA in a fuzzy formal decision context. First, the original weight vector of all attributes is obtained by fusing the original deterministic decision rules. Second, incremental fuzzy decision rules are obtained according to the added objects. Then for the incremental fuzzy decision rules, OFRCA updates the weight vector based on minimizing the loss function. At the end of the process, all the rules are fused into an attributeweighted classifier with the final weight vector. In this paper, we employ the regret to obtain a learning guarantee for OFRCA. The growth rate of OFRCA's regret converges to 0 as the number of iterations increases in the ideal state, which provides an effective learning guarantee for OFRCA. To verify the effectiveness of the proposed algorithm, this paper conducted numerical experiments and systematically analyzed the experimental results, which demonstrated that OFRCA achieved the highest advanced classification performance among the compared algorithms.

SWINN: Efficient nearest neighbor search in sliding windows using graphs

Nearest neighbor search (NNS) is one of the main concerns in data stream applications since similarity queries can be used in multiple scenarios. Online NNS is usually performed on a sliding window by lazily scanning every element currently stored in the window. This paper proposes Sliding Window-based Incremental Nearest Neighbors (SWINN), a graph-based online search index algorithm for speeding up NNS in potentially never-ending and dynamic data stream tasks. Our proposal broadens the application of online NNS-based solutions, as even moderately large data buffers become impractical to handle when a naive NNS strategy is selected. SWINN enables efficient handling of large data buffers by using an incremental strategy to build and update a search graph supporting any distance metric. Vertices can be added and removed from the search graph. To keep the graph reliable for search queries, lightweight graph maintenance routines are run. According to experimental results, SWINN is significantly faster than performing a naive complete scan of the data buffer while keeping competitive search recall values. We also apply SWINN to online classification and regression tasks and show that our proposal is effective against popular online machine learning algorithms.

Hyperspectral imaging combined with deep learning for discrimination of Lingwu long jujube in terms of the time after bruising

The timely detection and classification of bruised Lingwu long jujube is crucial for reducing economic losses in the agricultural industry. Traditional machine learning methods had been effectively used for bruise time classification with limited samples while the challenges were presented when the amount of data expanded. Deep learning methods can offer a solution. The hyperspectral image of the jujube sample was obtained using near-infrared hyperspectral imaging system (900–1700 nm). The optimal input variables were obtained by comparing the performance of the partial least squares-discriminant analysis, artificial neural network, and one-dimensional convolutional neural network (1D-CNN) models based on the spectral data and the fusion data of spectral and textural features. The optimal variable was the fusion data that could improve the model performance (from 92.68% to 96.10%). The above-mentioned three classifiers were used to establish models based on fusion data for two different dataset sizes consisting of large datasets (LD) (820) and small datasets (SD) (200) to explore the influence of dataset size on model performance. As the sample size increased, the 1D-CNN had good robustness with a classification rate of 94.00% for SD and 96.10% for LD in the prediction sets for all samples. The classification rate of prediction sets for 0 h after bruising increased from 90.00% to 92.68%, and from 80.00% to 90.24% in the class of 24 h after bruising. The findings provided a new approach to building a large spectral library and developing an online classification and detection system, thereby providing new ideas for the development of grading systems for Lingwu long jujube.

Raw EMG classification using extreme value machine

Electromyogram (EMG) signal is considered as an easy-to-capture (i.e. skin-mounted) and promissing biometric for the control of prosthetic hands. Despite the plethora number of researches on EMG-based control of prosthetics, two major challenges have not sufficiently been addressed , i.e. realtime classification, and robustness against noise. Our contribution in this paper is two fold. First, we have proposed a deep neural network (DNN) model with customized architecture to learn features directly from raw EMG data. In this model, a self-improvement module enhances the accuracy and robustness against artifact. Second, we utilize extreme value machine (EVM) for classification of the learnt feature. EVM models the statistical distribution of the latent space and classifies finger movements based on the maximum cumulative distribution probability to the fitted model. Our experimental results, using public domain EMG data, are very promising. They indicate the average accuracy of 98.5% for 750 msec window for 10-class classification. This result is superior or competitive compared with other online classifications reported in the literature on the same dataset. Additionally, our results illustrate that self-improvement feature learner (SIFL) is much more resistant against the noise than the models that employ engineered features. For example, in presence of severe white noise, the accuracy of our methodology degrades 10−15% compared to 30−50% of others.

Online Classification of Network Traffic Based on Granular Computing

At Presently, it is still a great challenge to achieve online classification of traffic flows due to the highly varying network environments, e.g., unpredictable new traffic classes, network noise, and congestion. Traditional classification methods work well in stable network environments, but may not exhibit their performance in dynamic environments. To address online classification issues, a granular computing-based classification model (GCCM) is developed, where the spatial and temporal flow granules are defined to make GCCM robust against variations and less sensitive to noise, and the correlations among flow granules are explored to establish the granular relation matrix (GRM). The inherent burst features between packets indicated by GRM prompt GCCM to achieve fine classification in unstable network environments. GCCM analyzes the burst features of packets without inspecting the payload information, and thus can be used to classify encrypted traffic as well as unencrypted traffic at a fast speed. In addition, the GCCM model, depending on difference measurement <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$D(\cdot)$</tex-math> </inline-formula> , is a threshold-based classification, and therefore can be used to distinguish between time-varying classes. The validity of GCCM for online traffic classification is examined through theoretical results. The experimental evaluation of classification for fine and varied classes under dynamic network environments with noise and congestion also demonstrates its superiority in terms of classification accuracy and real-time performance with the state-of-the-art.

Ultrasonic Aspirator for Tissue Contact Detection: An Online Classification on Time-Series.

The goal of neurosurgical tumor surgery is to remove the tumor completely without damaging healthy brain structures and thereby impairing the patient's neurological functions. This requires careful planning and execution of the operation by experienced neurosurgeons using the latest intraoperative technologies to achieve safe and rapid tumor reduction without harming the patient. To achieve this goal, a standard ultrasonic aspirator designed for tissue removal is equipped with additional intraoperative tissue detection using machine learning methods.Since decision-making in a clinical context must be fast, online contact detection is critical. Data are generated on three types of artificial tissue models in a CNC machine-controlled environment with four different ultrasonic aspirator settings. Contact classification on artificial tissue models is evaluated on four classification algorithms: change point detection (CPD), random forest (RF), recurrent neural network (RNN) and temporal convolutional network (TCN). Data preprocessing steps are applied, and their impacts are investigated. All methods are evaluated on five-fold cross-validation and provide generally good results with a performance of up to 0.977±0.007 in mean F1-score. Preprocessing the data has a positive effect on the classification processes for all methods and consistently improves the metrics. Thus, this work indicates in a first step that contact classification is feasible in an online context for an ultrasonic aspirator. Further research is necessary on different tissue types, as well as hand-held use to more closely resemble the intraoperative clinical conditions.

On-line surface roughness classification for multiple CNC milling conditions based on transfer learning and neural network

On-line surface roughness classification for multiple CNC milling conditions based on transfer learning and neural network

Online detection of compensatory strategies in human movement with supervised classification: a pilot study.

Stroke survivors often compensate for the loss of motor function in their distal joints by altered use of more proximal joints and body segments. Since this can be detrimental to the rehabilitation process in the long-term, it is imperative that such movements are indicated to the patients and their caregiver. This is a difficult task since compensation strategies are varied and multi-faceted. Recent works that have focused on supervised machine learning methods for compensation detection often require a large training dataset of motions with compensation location annotations for each time-step of the recorded motion. In contrast, this study proposed a novel approach that learned a linear classifier from energy-based features to discriminate between healthy and compensatory movements and identify the compensating joints without the need for dense and explicit annotations. Six healthy physiotherapists performed five different tasks using healthy movements and acted compensations. The resulting motion capture data was transformed into joint kinematic and dynamic trajectories. Inspired by works in bio-mechanics, energy-based features were extracted from this dataset. Support vector machine (SVM) and logistic regression (LR) algorithms were then applied for detection of compensatory movements. For compensating joint identification, an additional condition enforcing the independence of the feature calculation for each observable degree of freedom was imposed. Using leave-one-out cross validation, low values of mean brier score (<0.15), mis-classification rate (<0.2) and false discovery rate (<0.2) were obtained for both SVM and LR classifiers. These methods were found to outperform deep learning classifiers that did not use energy-based features. Additionally, online classification performance by our methods were also shown to outperform deep learning baselines. Furthermore, qualitative results obtained from the compensation joint identification experiment indicated that the method could successfully identify compensating joints. Results from this study indicated that including prior bio-mechanical information in the form of energy based features can improve classification performance even when linear classifiers are used, both for offline and online classification. Furthermore, evaluation compensation joint identification algorithm indicated that it could potentially provide a straightforward and interpretable way of identifying compensating joints, as well as the degree of compensation being performed.

An intelligent network monitoring approach for online classification of Darknet traffic

The Internet plays a crucial role in supporting global applications and businesses, but security remains a major challenge. Within the Internet, there exists a parallel network known as the Darknet, where malicious activities and traffic are present and require real-time classification. Many methods aim to classify this Darknet traffic in real-time due to its significant volume within Internet traffic. However, online Darknet traffic classification faces challenges, particularly in determining the optimal packet sampling amount for achieving a high classification rate in high-performance networks. To address this, our paper presents a novel approach that combines Convolutional Neural Network (CNN) and Reinforcement Learning (RL) techniques for intelligent and adaptive packet sampling rates in high-performance network interfaces. This method reduces overhead on monitored entities, especially in high-speed networks with a high bit rate. Our findings demonstrate a TOR traffic prediction accuracy of 99.84% and successful classification tasks in high-throughput networks using our method.

RETRACTED: A Newly-Designed Wearable Robotic Hand Exoskeleton Controlled by EMG Signals and ROS Embedded Systems

One of the most difficult parts of stroke therapy is hand mobility recovery. Indeed, stroke is a serious medical disorder that can seriously impair hand and locomotor movement. To improve hand function in stroke patients, new medical technologies, such as various wearable devices and rehabilitation therapies, are being developed. In this study, a new design of electromyography (EMG)-controlled 3D-printed hand exoskeleton is presented. The exoskeleton was created to help stroke victims with their gripping abilities. Computer-aided design software was used to create the device’s 3D architecture, which was then printed using a polylactic acid filament. For online classifications, the performance of two classifiers—the support vector machine (SVM) and the K-near neighbor (KNN)—was compared. The Robot Operating System (ROS) connects all the various system nodes and generates the decision for the hand exoskeleton. The selected classifiers had high accuracy, reaching up to 98% for online classification performed with healthy subjects. These findings imply that the new wearable exoskeleton, which could be controlled in accordance with the subjects’ motion intentions, could aid in hand rehabilitation for a wider motion range and greater dexterity.

An open-source human-in-the-loop BCI research framework: method and design.

Brain-computer interfaces (BCIs) translate brain activity into digital commands for interaction with the physical world. The technology has great potential in several applied areas, ranging from medical applications to entertainment industry, and creates new conditions for basic research in cognitive neuroscience. The BCIs of today, however, offer only crude online classification of the user's current state of mind, and more sophisticated decoding of mental states depends on time-consuming offline data analysis. The present paper addresses this limitation directly by leveraging a set of improvements to the analytical pipeline to pave the way for the next generation of online BCIs. Specifically, we introduce an open-source research framework that features a modular and customizable hardware-independent design. This framework facilitates human-in-the-loop (HIL) model training and retraining, real-time stimulus control, and enables transfer learning and cloud computing for the online classification of electroencephalography (EEG) data. Stimuli for the subject and diagnostics for the researcher are shown on separate displays using web browser technologies. Messages are sent using the Lab Streaming Layer standard and websockets. Real-time signal processing and classification, as well as training of machine learning models, is facilitated by the open-source Python package Timeflux. The framework runs on Linux, MacOS, and Windows. While online analysis is the main target of the BCI-HIL framework, offline analysis of the EEG data can be performed with Python, MATLAB, and Julia through packages like MNE, EEGLAB, or FieldTrip. The paper describes and discusses desirable properties of a human-in-the-loop BCI research platform. The BCI-HIL framework is released under MIT license with examples at: bci.lu.se/bci-hil (or at: github.com/bci-hil/bci-hil).

Study on a comprehensive indicator and online classification of early warning of low frequency oscillation in power system

With the development of smart grid technology in China, the study of low frequency oscillation early warning has become an important element of power system stability research. Aiming at the problems of single warning indicator with poor accuracy and lack of online warning system in the early warning strategy of low frequency oscillation in power system, a comprehensive low frequency oscillation early warning algorithm based on the evaluation method of kernel matrix hierarchical analysis of fuzzy comprehensive is proposed. Firstly, this method screens out the original data collected by Phasor Measurement Unit (PMU), and extracts the key data volume to form multiple early warning indicators, and generates the trapezoidal fuzzy number judgment matrix. Secondly, by calculating the kernel matrix and consistency test, it is determined whether or not the generated fuzzy matrix is reasonable and calculate the relative weights of the evaluation indicators. The affiliation function is constructed for each early warning indicator. A comprehensive indicator is calculated for multiple indicators, and the reasons of low frequency oscillation in power system are classified by the Random Forest (RF) algorithm. Then, the low-frequency oscillation system is constructed, and the algorithm convergence is proved by the Lyapunov indirect method. The algorithm can automatically determine whether low frequency oscillation occurs or not and classify different types of low frequency oscillations online. Finally, the correctness and validity of this paper’s method is verified by a New England 10-machine 39-node system. Compared with the traditional Analytic Hierarchy Process (AHP) method, the proposed method in this paper greatly improves the accuracy of early warning and meets the need of online warning.

PGNAA Spectral Classification of Metal With Density Estimations

For environmental, sustainable economic and political reasons, recycling processes are becoming increasingly important, aiming at a much higher use of secondary raw materials. Currently, for the copper and aluminium industries, no method for the non-destructive online analysis of heterogeneous materials are available. The Prompt Gamma Neutron Activation Analysis (PGNAA) has the potential to overcome this challenge. A difficulty when using PGNAA for online classification arises from the small amount of noisy data, due to short-term measurements. In this case, classical evaluation methods using detailed peak by peak analysis fail. Therefore, we propose to view spectral data as probability distributions. Then, we can classify material using maximum log-likelihood with respect to kernel density estimation and use discrete sampling to optimize hyperparameters. For measurements of pure aluminium alloys we achieve near perfect classification of aluminium alloys under 0.25 second.