Incremental Learning Algorithm Research Articles

Video object matching across multiple non-overlapping camera views based on multi-feature fusion and incremental learning

Matching objects across multiple cameras with non-overlapping views is a necessary but difficult task in the wide area video surveillance. Owing to the lack of spatio-temporal information, only the visual information can be used in some scenarios, especially when the cameras are widely separated. This paper proposes a novel framework based on multi-feature fusion and incremental learning to match the objects across disjoint views in the absence of space–time cues. We first develop a competitive major feature histogram fusion representation (CMFH11CMFH is the abbreviation of Competitive Major Feature Histogram fusion representation.) to formulate the appearance model for characterizing the potentially matching objects. The appearances of the objects can change over time and hence the models should be continuously updated. We then adopt an improved incremental general multicategory support vector machine algorithm (IGMSVM22IGMSVM is the abbreviation of Incremental General Multicategory Support Vector Machine learning algorithm.) to update the appearance models online and match the objects based on a classification method. Only a small amount of samples are needed for building an accurate classification model in our method. Several tests are performed on CAVIAR, ISCAPS and VIPeR databases where the objects change significantly due to variations in the viewpoint, illumination and poses. Experimental results demonstrate the advantages of the proposed methodology in terms of computational efficiency, computation storage, and matching accuracy over that of other state-of-the-art classification-based matching approaches. The system developed in this research can be used in real-time video surveillance applications.

Cognitive fault diagnosis in Tennessee Eastman Process using learning in the model space

This paper focuses on the Tennessee Eastman (TE) process and for the first time investigates it in a cognitive way. The cognitive fault diagnosis does not assume prior knowledge of the fault numbers and signatures. This approach firstly employs deterministic reservoir models to fit the multiple-input and multiple-output signals in the TE process, which map the signal space to the (reservoir) model space. Then we investigate incremental learning algorithms in this reservoir model space based on the “function distance” between these models. The main contribution of this paper is to provide a cognitive solution to this popular benchmark problem. Our approach is not only applicable to fault detection, but also to fault isolation without knowing the prior information about the fault signature. Experimental comparisons with other state-of-the-art approaches confirmed the benefits of our approach. Our algorithm is efficient and can run in real-time for practical applications.

Detecting spamming activities in a campus network using incremental learning

Most spam filters deployed on the receiver side are good at curbing email spam for end users, but help little to crack down the spamming sources. This work is intended to nip the spamming hosts in the bud. We collected the logs of SMTP sessions initiated from the hosts in the campus for half a year, and analyzed the activities of the hosts with the rates of successful deliveries and various types of failure messages in the sessions as the features. We use an incremental passive-aggressive learning algorithm to efficiently adapt the classifier to the latest spamming activities for detecting the spamming hosts. The detection accuracy can reach 93.5% after the classifier is adjusted in just few rounds. This design will be useful for the network administrators to reliably detect and crack down the internal spamming hosts.

Incremental kernel minimum squared error (KMSE)

Kernel minimum squared error (KMSE) is a simple and efficient learning algorithm extensively used by the machine learning community. The solution of KMSE is typically not sparse, which may be harmful for real time applications. Current proposed algorithms show hindrances when dealing with sample flow learning systems. A significant improvement utilizes incremental learning, namely IKMSE, to achieve better sparsity however it is not enough. We propose two additional adaptive incremental learning algorithms, viz. CRKMSE and PRKMSE. CRKMSE adopts a complete reduction strategy while PRKMSE uses the partial reduction. Generally, PRKMSE is superior to CRKMSE in terms of training time. And PRKMSE owns a better real time than CRKMSE in the testing phase due to fewer significant nodes. Finally, a lot of experiments are reported to confirm our viewpoints in this paper.

A tree-based learning approach for document structure analysis and its application to web search

AbstractIn this paper, we study the problem of structural analysis of Web documents aiming at extracting the sectional hierarchy of a document. In general, a document can be represented as a hierarchy of sections and subsections with corresponding headings and subheadings. We developed two machine learning models: heading extraction model and hierarchy extraction model. Heading extraction was formulated as a classification problem whereas a tree-based learning approach was employed in hierarchy extraction. For this purpose, we developed an incremental learning algorithm based on support vector machines and perceptrons. The models were evaluated in detail with respect to the performance of the heading and hierarchy extraction tasks. For comparison, a baseline rule-based approach was used that relies on heuristics and HTML document object model tree processing. The machine learning approach, which is a fully automatic approach, outperformed the rule-based approach. We also analyzed the effect of document structuring on automatic summarization in the context of Web search. The results of the task-based evaluation on TREC queries showed that structured summaries are superior to unstructured summaries both in terms of accuracy and user ratings, and enable the users to determine the relevancy of search results more accurately than search engine snippets.

Incremental Naïve Bayesian Learning Algorithm based on Classification Contribution Degree

In order to improve the ability of gradual learning on the training set gotten in batches of Naive Bayesian classifier, an incremental Naive Bayesian learning algorithm is improved with the research on the existing incremental Naive Bayesian learning algorithms. Aiming at the problems with the existing incremental amending sample selection strategy, the paper introduced the concept of sample Classification Contribution Degree in the process of incremental learning, based on the comprehensive consideration about classification discrimination, noisy and redundancy of the new training data. The definition and theoretical analysis of sample Classification Contribution Degree is given in this paper. Then the paper proposed the incremental Naive Bayesian classification method based on the Classification Contribution Degree. The experimental results show that the algorithm simplified the incremental learning process, improved the classification accuracy of incremental learning.

Robust Visual Tracking via Appearance Modeling and Sparse Representation

When appearance variation of object, partial occlusion or illumination change in object images occurs, most existing tracking approaches fail to track the target effectively. To deal with the problem, this paper proposed a robust visual tracking method based on appearance modeling and sparse representation. The proposed method exploits two-dimensional principal component analysis (2DPCA) with sparse representation theory for constructing appearance model. Then tracking is achieved by Bayesian inference framework, in which a particle filter is applied to evaluate the target state sequentially over time. In addition, to make the observation model more robust, the incremental learning algorithm is used to update the template set. Both qualitative and quantitative evaluations on four publicly available benchmark video sequences demonstrate that the proposed visual tracking algorithm performs better than several state-of-the-art algorithms.

Application of Evolutionary Algorithm for Optimal Directional Overcurrent Relay Coordination

In this paper, two Evolutionary Algorithms (EAs) i.e., an improved Genetic Algorithms (GAs) and Population Based Incremental Learning (PBIL) algorithm are applied for optimal coordination of directional overcurrent relays in an interconnected power system network. The problem of coordinating directional overcurrent relays is formulated as an optimization problem that is solved via the improved GAs and PBIL. The simulation results obtained using the improved GAs are compared with those obtained using PBIL. The results show that the improved GA proposed in this paper performs better than PBIL.

ROBIL: Robot Path Planning Based on PBIL Algorithm

Genetic algorithm (GAs) have attracted considerable interest for their usefulness in solving complex robot path planning problems. Specifically, researchers have combined conventional GAs with problem-specific operators and initialization techniques to find the shortest paths in a variety of robotic environments. Unfortunately, these approaches have exhibited inherently unstable performance, and they have tended to make other aspects of the problem-solving process (e.g., adjusting parameter sensitivities and creating high-quality initial populations) unmanageable. As an alternative to conventional GAs, we propose a new population-based incremental learning (PBIL) algorithm for robot path planning, a probabilistic model of nodes, and an edge bank for generating promising paths. Experimental results demonstrate the computational superiority of the proposed method over conventional GA approaches.

A Mahalanobis Hyperellipsoidal Learning Machine Class Incremental Learning Algorithm

A Mahalanobis hyperellipsoidal learning machine class incremental learning algorithm is proposed. To each class sample, the hyperellipsoidal that encloses as many as possible and pushes the outlier samples away is trained in the feature space. In the process of incremental learning, only one subclassifier is trained with the new class samples. The old models of the classifier are not influenced and can be reused. In the process of classification, considering the information of sample’s distribution in the feature space, the Mahalanobis distances from the sample mapping to the center of each hyperellipsoidal are used to decide the classified sample class. The experimental results show that the proposed method has higher classification precision and classification speed.

A New Incremental Learning Algorithm with Probabilistic Weights Using Extended Data Expression

New incremental learning algorithm using extended data expression, based on probabilistic compounding, is presented in this paper. Incremental learning algorithm generates an ensemble of weak classifiers and compounds these classifiers to a strong classifier, using a weighted majority voting, to improve classification performance. We introduce new probabilistic weighted majority voting founded on extended data expression. In this case class distribution of the output is used to compound classifiers. UChoo, a decision tree classifier for extended data expression, is used as a base classifier, as it allows obtaining extended output expression that defines class distribution of the output. Extended data expression and UChoo classifier are powerful techniques in classification and rule refinement problem. In this paper extended data expression is applied to obtain probabilistic results with probabilistic majority voting. To show performance advantages, new algorithm is compared with Learn++, an incremental ensemble-based algorithm.

An adaptive training based on classification system for patterns in facial expressions using SURF descriptor templates

Most facial expression recognition (FER) systems used facial expression data created during a short period of time and this data is used for learning/training of FER systems. There are many facial expression patterns (i.e. a particular expression can be represented in many different patterns) which cannot be generated and used as learning/training data in a short time. Therefore, in order to maintain its high accuracy and robustness for a long time of a facial expression recognition system, the classifier should be evolved adaptively over time and space. We proposed a facial expression recognition system that has the aptitude of incrementally learning and thus can learn all possible patterns of expressions that may be generated in feature. After extraction of region of interest (face), the system extracts Speeded-Up Robust Features (SURF). A novel SURF descriptor template based nearest neighbor classifier is proposed for classification. This classifier is used as base/weak classifier for incremental learning algorithm Learn++. A vast range of experimentation is performed on five different databases that demonstrate the incremental learning capability of the proposed system. The experiments using the incrementally learning classification demonstrate promising results.

Conceptual Imitation Learning Based on Perceptual and Functional Characteristics of Action

This paper presents a conceptual model for imitation learning to abstract spatio-temporal demonstrations based on their perceptual and functional characteristics. To this end, the concepts are represented by prototypes irregularly scattered in the perceptual space but sharing the same functionality. Functional similarity between demonstrations is understood by reinforcements of the teacher or recognizing the effects of actions. Abstraction, concept acquisition, and self-organization of prototypes are performed through incremental and gradual learning algorithms. In these algorithms, hidden Markov models are used to prototype perceptually similar demonstrations. In addition, a mechanism is introduced to integrate perceptions of different modalities for multimodal concept recognition. Performance of the proposed model is evaluated in two different tasks. The first one is imitation learning of some hand gestures through interaction with the teachers. In this task, the perceptions from different modalities, including vision, motor, and audition, are used in a variety of experiments. The second task is to learn a set of actions by recognizing their emotional effects. Results of the experiments on a humanoid robot show the efficacy of our model for conceptual imitation learning.

An incremental extreme learning machine for online sequential learning problems

A fast and outstanding incremental learning algorithm is required to meet the demand of online applications where data comes one by one or chunk by chunk to avoid retraining and save precious time. Although many interesting research results have been achieved, there are still a lot of difficulties in real applications because of their unsatisfying generalization performance or intensive computation cost. This paper presents an Incremental Extreme Learning Machine (IELM) which is developed based on Extreme Learning Machine (ELM), a unified framework of LS-SVM and PSVM presented by Hang et al. (2011) in [15]. Under different application demand and different computational cost and efficiency, three different alternative solutions of IELM are achieved. Detailed comparisons of the IELM algorithm with other incremental algorithms are achieved by simulation on benchmark problems and real critical dimension (CD) prediction problem in lithography of actual semiconductor production line. The results show that kernel based IELM solution performs best while least square IELM solution is the fastest of the three alterative solutions when the number of training data is huge. All the results show that the presented IELM algorithms have better performance than other incremental algorithms such as online sequential ELM (OS-ELM) presented by Liang et al. (2006) [8] and fixed size LSSVM presented by Espinoza et al. (2006) [11].

Label Consistent K-SVD: Learning a Discriminative Dictionary for Recognition

A label consistent K-SVD (LC-KSVD) algorithm to learn a discriminative dictionary for sparse coding is presented. In addition to using class labels of training data, we also associate label information with each dictionary item (columns of the dictionary matrix) to enforce discriminability in sparse codes during the dictionary learning process. More specifically, we introduce a new label consistency constraint called "discriminative sparse-code error" and combine it with the reconstruction error and the classification error to form a unified objective function. The optimal solution is efficiently obtained using the K-SVD algorithm. Our algorithm learns a single overcomplete dictionary and an optimal linear classifier jointly. The incremental dictionary learning algorithm is presented for the situation of limited memory resources. It yields dictionaries so that feature points with the same class labels have similar sparse codes. Experimental results demonstrate that our algorithm outperforms many recently proposed sparse-coding techniques for face, action, scene, and object category recognition under the same learning conditions.

ILClass: Error-driven antecedent learning for evolving Takagi-Sugeno classification systems

The purpose of this research work is to go beyond the traditional classification systems in which the set of recognizable categories is predefined at the conception phase and keeps unchanged during its operation. Motivated by the increasing needs of flexible classifiers that can be continuously adapted to cope with dynamic environments, we propose a new evolving classification system and an incremental learning algorithm called ILClass. The classifier is learned in incremental and lifelong manner and able to learn new classes from few samples. Our approach is based on first-order Takagi-Sugeno (TS) system. The main contribution of this paper consists in proposing a global incremental learning paradigm in which antecedent and consequent are learned in synergy, contrary to the existing approaches where they are learned separately. Output feedback is used in controlled manner to bias antecedent adaptation toward difficult data samples in order to improve system accuracy. Our system is evaluated using different well-known benchmarks, with a special focus on its capacity of learning new classes.

Constructing of Multimedia Resources for Second Language Teaching Based on Intelligent Information Processing of Movie Resources

Based on the topic models database and time density of captionâ??s dialogue, designed a adaptive, iterative incremental learning algorithm, which could meanwhile carry out topic detection, words clustering and calculating wordsâ?? used degree. Lastly, based on wordsâ?? used degree, by calculating degree of difficulty of captionâ??s dialogue, graded videosâ?? corresponding to captions, constructed a large multimedia database for oral Chinese teaching.

Predicting Drug-Target Interactions of Nuclear Receptors Based on Molecular Descriptors Information

Predicting the interaction between drugs and target proteins is one of the most important tasks for bioinformatics. Selecting suitable features for drug-target interaction prediction is an effective way for achieving this important goal. In this article, the molecular descriptors information of drugs combined with several important biological features of nuclear receptor (NR) proteins were first utilized for feature selection. Then, the incremental learning algorithm with support vector machines was used to obtain the optimal feature subset for drug-target interaction prediction. The results demonstrate that the feature selection method can lead to promising improvement on prediction accuracy of this target family. Keywords: Drug-target interaction, Molecular descriptors, Feature selection.

Robust Bio-Signal Based Control of an Intelligent Wheelchair

In this paper, an adaptive human-machine interaction (HMI) method that is based on surface electromyography (sEMG) signals is proposed for the hands-free control of an intelligent wheelchair. sEMG signals generated by the facial movements are obtained by a convenient dry electrodes sensing device. After the signals features are extracted from the autoregressive model, control data samples are updated and trained by an incremental online learning algorithm in real-time. Experimental results show that the proposed method can significantly improve the classification accuracy and training speed. Moreover, this method can effectively reduce the influence of muscle fatigue during a long time operation of sEMG-based HMI.

Online Learning of Single- and Multivalued Functions with an Infinite Mixture of Linear Experts

We present a supervised learning algorithm for estimation of generic input-output relations in a real-time, online fashion. The proposed method is based on a generalized expectation-maximization approach to fit an infinite mixture of linear experts (IMLE) to an online stream of data samples. This probabilistic model, while not fully Bayesian, can efficiently choose the number of experts that are allocated to the mixture, this way effectively controlling the complexity of the resulting model. The result is an incremental, online, and localized learning algorithm that performs nonlinear, multivariate regression on multivariate outputs by approximating the target function by a linear relation within each expert input domain and that can allocate new experts as needed. A distinctive feature of the proposed method is the ability to learn multivalued functions: one-to-many mappings that naturally arise in some robotic and computer vision learning domains, using an approach based on a Bayesian generative model for the predictions provided by each of the mixture experts. As a consequence, it is able to directly provide forward and inverse relations from the same learned mixture model. We conduct an extensive set of experiments to evaluate the proposed algorithm performance, and the results show that it can outperform state-of-the-art online function approximation algorithms in single-valued regression, while demonstrating good estimation capabilities in a multivalued function approximation context.