Incremental Learning Process Research Articles

Perpetual Learning Framework based on Type-2 Fuzzy Logic System for a Complex Manufacturing Process

Abstract: This paper introduces a perpetual type-2 Neuro-Fuzzy modelling structure for continuous learning and its application to the complex thermo-mechanical metal process of steel Friction Stir Welding (FSW). The ‘perpetual’ property refers to the capability of the proposed system to continuously learn from new process data, in an incremental learning fashion. This is particularly important in industrial/manufacturing processes, as it eliminates the need to retrain the model in the presence of new data, or in the case of any process drift. The proposed structure evolves through incremental, hybrid (supervised/unsupervised) learning, and accommodates new sample data in a continuous fashion. The human-like information capture paradigm of granular computing is used along with an interval type-2 neural-fuzzy system to develop a modelling structure that is tolerant to the uncertainty in the manufacturing data (common challenge in industrial/manufacturing data). The proposed method relies on the creation of new fuzzy rules which are updated and optimised during the incremental learning process. An iterative pruning strategy in the model is then employed to remove any redundant rules, as a result of the incremental learning process. The rule growing/pruning strategy is used to guarantee that the proposed structure can be used in a perpetual learning mode. It is demonstrated that the proposed structure can effectively learn complex dynamics of input-output data in an adaptive way and maintain good predictive performance in the metal processing case study of steel FSW using real manufacturing data.

An Improved Incremental Learning Approach for KPI Prognosis of Dynamic Fuel Cell System.

The key performance indicator (KPI) has an important practical value with respect to the product quality and economic benefits for modern industry. To cope with the KPI prognosis issue under nonlinear conditions, this paper presents an improved incremental learning approach based on available process measurements. The proposed approach takes advantage of the algorithm overlapping of locally weighted projection regression (LWPR) and partial least squares (PLS), implementing the PLS-based prognosis in each locally linear model produced by the incremental learning process of LWPR. The global prognosis results including KPI prediction and process monitoring are obtained from the corresponding normalized weighted means of all the local models. The statistical indicators for prognosis are enhanced as well by the design of novel KPI-related and KPI-unrelated statistics with suitable control limits for non-Gaussian data. For application-oriented purpose, the process measurements from real datasets of a proton exchange membrane fuel cell system are employed to demonstrate the effectiveness of KPI prognosis. The proposed approach is finally extended to a long-term voltage prediction for potential reference of further fuel cell applications.

A Cognitive Neural Architecture Able to Learn and Communicate through Natural Language.

Communicative interactions involve a kind of procedural knowledge that is used by the human brain for processing verbal and nonverbal inputs and for language production. Although considerable work has been done on modeling human language abilities, it has been difficult to bring them together to a comprehensive tabula rasa system compatible with current knowledge of how verbal information is processed in the brain. This work presents a cognitive system, entirely based on a large-scale neural architecture, which was developed to shed light on the procedural knowledge involved in language elaboration. The main component of this system is the central executive, which is a supervising system that coordinates the other components of the working memory. In our model, the central executive is a neural network that takes as input the neural activation states of the short-term memory and yields as output mental actions, which control the flow of information among the working memory components through neural gating mechanisms. The proposed system is capable of learning to communicate through natural language starting from tabula rasa, without any a priori knowledge of the structure of phrases, meaning of words, role of the different classes of words, only by interacting with a human through a text-based interface, using an open-ended incremental learning process. It is able to learn nouns, verbs, adjectives, pronouns and other word classes, and to use them in expressive language. The model was validated on a corpus of 1587 input sentences, based on literature on early language assessment, at the level of about 4-years old child, and produced 521 output sentences, expressing a broad range of language processing functionalities.

Describing and learning of related parts based on latent structural model in big data

In this paper, we propose a novel latent structural model for big data image recognition. It addresses the problem that large amount of labeled training samples are needed in traditional structural models. This method first builds an initial structural model by using only one labeled image. After pooling unlabeled samples into the initial model, an incremental learning process is used to find more candidate parts and to update the model. The appearance features of the parts are described by multiple kernel learning method that assembles more information of the parts, such as color, edge, and texture. Therefore, the proposed model considers not only independent components but also their inherent spatial and appearance relationships. Finally, the updated model is applied to recognition tasks. Experiments show that this method is effective in handling big data problems and has achieved better performance than several state-of-the-art methods.

Incremental generalized low rank approximation of matrices for visual learning and recognition

In this paper, we consider the problem of incrementally computing low-rank approximations of matrices. We propose an incremental low rank approximation algorithm of a sequence of matrices instead of one single matrix. Based on the generalized Low Rank Approximation of matrices approach, the proposed method uses an iterative approach to update the generated low rank structures. We tested the proposed incremental learning method on five face databases to evaluate its efficiency in terms of recognition rate and compared its performance with the Incremental Singular Value Decomposition method. We also showed that the overall reconstruction error is kept bounded during the incremental learning process.

A Lightweight Data Preprocessing Strategy with Fast Contradiction Analysis for Incremental Classifier Learning

A prime objective in constructing data streaming mining models is to achieve good accuracy, fast learning, and robustness to noise. Although many techniques have been proposed in the past, efforts to improve the accuracy of classification models have been somewhat disparate. These techniques include, but are not limited to, feature selection, dimensionality reduction, and the removal of noise from training data. One limitation common to all of these techniques is the assumption that the full training dataset must be applied. Although this has been effective for traditional batch training, it may not be practical for incremental classifier learning, also known as data stream mining, where only a single pass of the data stream is seen at a time. Because data streams can amount to infinity and the so-called big data phenomenon, the data preprocessing time must be kept to a minimum. This paper introduces a new data preprocessing strategy suitable for the progressive purging of noisy data from the training dataset without the need to process the whole dataset at one time. This strategy is shown via a computer simulation to provide the significant benefit of allowing for the dynamic removal of bad records from the incremental classifier learning process.

A Cognitive Neural Model of Executive Functions in Natural Language Processing

Abstract Although extensive research has been devoted to cognitive models of human language, the role of executive functions in language processing has little been explored. In this work we present a neural-network-based cognitive architecture which models the development of the procedural knowledge that underpin language processing. The large scale organization of the architecture is based on a multi-component working memory model, with a central executive that controls the flow of information among the slave systems through neural gating mechanisms. The system was validated, starting from a tabula rasa condition, on a on a corpus of five datasets, each devoted to a thematic group, based on literature on early language assessment, at the level of a preschool child. The results show that the system is capable of learning different word classes, and to use them in expressive language, through an open-ended incremental learning process, expressing a broad range of language processing functionalities.

A rapid learning algorithm for vehicle classification

AdaBoost is a popular method for vehicle detection, but the training process is quite time-consuming. In this paper, a rapid learning algorithm is proposed to tackle this weakness of AdaBoost for vehicle classification. Firstly, an algorithm for computing the Haar-like feature pool on a 32×32 grayscale image patch by using all simple and rotated Haar-like prototypes is introduced to represent a vehicle’s appearance. Then, a fast training approach for the weak classifier is presented by combining a sample’s feature value with its class label. Finally, a rapid incremental learning algorithm of AdaBoost is designed to significantly improve the performance of AdaBoost. Experimental results demonstrate that the proposed approaches not only speed up the training and incremental learning processes of AdaBoost, but also yield better or competitive vehicle classification accuracies compared with several state-of-the-art methods, showing their potential for real-time applications.

Evolution of cooperation driven by incremental learning

It has been shown that the details of microscopic rules in structured populations can have a crucial impact on the ultimate outcome in evolutionary games. So alternative formulations of strategies and their revision processes exploring how strategies are actually adopted and spread within the interaction network need to be studied. In the present work, we formulate the strategy update rule as an incremental learning process, wherein knowledge is refreshed according to one’s own experience learned from the past (self-learning) and that gained from social interaction (social-learning). More precisely, we propose a continuous version of strategy update rules, by introducing the willingness to cooperate W, to better capture the flexibility of decision making behavior. Importantly, the newly gained knowledge including self-learning and social learning is weighted by the parameter ω, establishing a strategy update rule involving innovative element. Moreover, we quantify the macroscopic features of the emerging patterns to inspect the underlying mechanisms of the evolutionary process using six cluster characteristics. In order to further support our results, we examine the time evolution course for these characteristics. Our results might provide insights for understanding cooperative behaviors and have several important implications for understanding how individuals adjust their strategies under real-life conditions.

Research on the Chinese Word Segmentation System Based on Incremental Learning

In order to cope with the increasing size of the training corpus and adapt to the requirements of incremental learning, this paper introduces a feature selection algorithm of maximum entropy model into the research of Chinese word segmentation technology, designs and implements a Chinese word segmentation system based on incremental learning. The experimental results show that the system gradually improves the segmentation accuracy in the incremental learning process which without wasting time to restudy.

From spontaneous motor activity to coordinated behaviour: a developmental model.

In mammals, the developmental path that links the primary behaviours observed during foetal stages to the full fledged behaviours observed in adults is still beyond our understanding. Often theories of motor control try to deal with the process of incremental learning in an abstract and modular way without establishing any correspondence with the mammalian developmental stages. In this paper, we propose a computational model that links three distinct behaviours which appear at three different stages of development. In order of appearance, these behaviours are: spontaneous motor activity (SMA), reflexes, and coordinated behaviours, such as locomotion. The goal of our model is to address in silico four hypotheses that are currently hard to verify in vivo: First, the hypothesis that spinal reflex circuits can be self-organized from the sensor and motor activity induced by SMA. Second, the hypothesis that supraspinal systems can modulate reflex circuits to achieve coordinated behaviour. Third, the hypothesis that, since SMA is observed in an organism throughout its entire lifetime, it provides a mechanism suitable to maintain the reflex circuits aligned with the musculoskeletal system, and thus adapt to changes in body morphology. And fourth, the hypothesis that by changing the modulation of the reflex circuits over time, one can switch between different coordinated behaviours. Our model is tested in a simulated musculoskeletal leg actuated by six muscles arranged in a number of different ways. Hopping is used as a case study of coordinated behaviour. Our results show that reflex circuits can be self-organized from SMA, and that, once these circuits are in place, they can be modulated to achieve coordinated behaviour. In addition, our results show that our model can naturally adapt to different morphological changes and perform behavioural transitions.

Self-Tuning Closed-Loop Fuzzy Logic Control Algorithm for Adaptive Prestressed Structures

Constantly increasing traffic loads, unpredictable influences on and unplanned conditions of bridges represent new challenges in the field of engineering. Innovative strategies and construction techniques are required to meet these and future challenges. Adaptive prestressing offers great potential to increase a structure’s load-carrying capacity and to optimize its load-induced stresses. In this manner, cost efficiency and sustainability of constructions can be enhanced. Therefore, a control algorithm with high reliability and redundancy is required to meet the new challenges. In this paper, an adaptive fuzzy logic-based closed-loop control system for realization of adaptive prestressed structures is presented. Based on the new concept, two different specialized control algorithms are developed. In order to decrease deflections, the prestressing force of an aluminium truss is controlled. The algorithm includes a fuzzy system with model-based knowledge base. The learning process is established on direct synchronization of the calculated deflections of the model with the measured reactions of the structure. Another fuzzy system for adaptive prestressing of a reinforced concrete T-beam uses a model-free knowledge base. Optimization of load-induced stresses is accomplished utilizing expert knowledge. An incremental learning process is performed in accordance with the algorithm’s basic rules. Experiments on the two prototypes are conducted to investigate the functionality of the developed algorithms. In addition to efficiency, high reliability and redundancy are shown by the control system. In this article, the potential of fuzzy-controlled adaptive prestressing is illustrated.

A semi-supervised incremental learning method based on adaptive probabilistic hypergraph for video semantic detection

Semantic categorization for the complex videos is an ambiguous task. The semi-supervised learning method based on hypergraph model can achieve multi-semantics labels, but a hypergraph model is sensitive to the radius parameter when it is constructed and the number of vertices belonging to a hyperedge is fixed. In this paper, a semi-supervised incremental learning method based on adaptive probabilistic hypergraph for video semantic detection is presented. In the probabilistic hypergraph modeling, a formula is presented as a measurement to adaptively decide whether a vertex is belonged to a hyperedge or not. The model has high robustness and can overcome the defect of fixed number of vertices belonging to the same hyperedge in the traditional probabilistic hypergraph model. In the semi-supervised incremental learning process, a threshold is defined, which is used to judge whether unlabeled sample can be added into the modeling, in order that the model learning result for unlabeled samples has high certainty. The experimental results show that our method can improve the model generalization ability and utilize the unlabeled samples effectively. In the aspects of recall rate and precision rate for semantic video concept detection from complex videos, our proposed method outperforms the compared methods.

Incremental Naïve Bayesian Learning Algorithm based on Classification Contribution Degree

In order to improve the ability of gradual learning on the training set gotten in batches of Naive Bayesian classifier, an incremental Naive Bayesian learning algorithm is improved with the research on the existing incremental Naive Bayesian learning algorithms. Aiming at the problems with the existing incremental amending sample selection strategy, the paper introduced the concept of sample Classification Contribution Degree in the process of incremental learning, based on the comprehensive consideration about classification discrimination, noisy and redundancy of the new training data. The definition and theoretical analysis of sample Classification Contribution Degree is given in this paper. Then the paper proposed the incremental Naive Bayesian classification method based on the Classification Contribution Degree. The experimental results show that the algorithm simplified the incremental learning process, improved the classification accuracy of incremental learning.

Optimal Boundary SVM Incremental Learning Algorithm

Support vectors (SVs) cant be selected completely in support vector machine (SVM) incremental, resulting incremental learning process cant be sustained. In order to solve this problem, the article proposes optimal boundary SVM incremental learning algorithm. Based on in-depth analysis of the trend of the classification surface and make use of the KKT conditions, selecting the border of the vectors include the support vectors to participate SVM incremental learning. The experiment shows that the algorithm can be completely covered the support vectors and have the identical result with the classic support vector machine, it also saves lots of time. Therefore it can provide the conditions for future large sample classification and incremental learning sustainability.

Online incremental learning for tool condition classification using modified Fuzzy ARTMAP network

Condition monitoring of tool wear is paramount for guaranteeing the quality of workpiece and improving the lifetime of the cutter. To improve the training speed and the flexibility of the incremental learning, a modified Fuzzy ARTMAP classifier is developed in which the resonance layer is linked with the category node directly by many to one mapping. Therefore, the weight value and model structure can be updated simultaneously during the online incremental learning process. To testify the effectiveness of the presented method, experiments of tool condition classification in the process of end milling of Titanium alloy are carried out and two incremental learning cases are simulated. The analysis of online learning process in both cases shows that the structure and parameters of the model can be adjusted automatically without requiring access to the previous training data. At the same time, the accuracy analysis demonstrates that the presented method has strong ability to learn the new knowledge without forgetting the previous knowledge.

Racing to the top? Catch-up strategies of Chinese and Indian independent car manufacturers

The study examines the attempts of five independent automobile latecomer firms (BYD, Chery, Geely, Tata Motors, Mahindra & Mahindra) to catch-up with their rivals from advanced economies. While catch-up speed is influenced by the country-specific context, the firms’ capability acquisition is generally based on leveraging directly accessible resources of advanced automotive firms (e.g., through reverse engineering, supplier involvement, in-licensing, or small scale M&As), and on progressively developing independent design and manufacturing skills. In this stage, international sales are confined to other developing countries only. After the initial fast catch-up trajectory, the firms’ incremental learning and capability building processes come up against a more challenging ‘catch up barrier’. The acquisitions of Jaguar Land Rover and Volvo by Tata Motors and Geely can be seen as attempts to ‘leapfrog’ over this otherwise lengthy and uncertain ‘last mile’ of the catch-up process.

Performance evaluation of incremental decision tree learning under noisy data streams

Big data has become a significant problem in software applications nowadays. Extracting classification model from such data requires an incremental learning process. The model should update when new data arrive, without re-scanning historical data. A single-pass algorithm suits continuously arrival data environment. However, one practical and important aspect that has gone relatively unstudied is noisy data streams. Such data are inevitable in real-world applications. This paper presents a new classification model with a single decision tree, so called incrementally Optimised Very Fast Decision Tree iOVFDT that embeds multi-objectives incremental optimisation and functional tree leaf. In the performance evaluation, noisy values were added into synthetic data. This evaluation investigated the performance under noisy data scenario. The result showed that iOVFDT outperforms the existing algorithms.

Rethinking HCI Education for Design: Problem-Based Learning and Virtual Worlds at an HCI Design Studio

The practice of teaching and learning human–computer interaction (HCI) design has to develop its own approaches that balance method and technology use with reflective and situated practice. This article presents our HCI design studio course that makes a combined use of constructivist pedagogies and virtual worlds aiming to aid students critically review and reflect on the use of related methods and technologies in design and to cultivate more general skills like self-directed learning, intrinsic motivation, and critical thinking. Our HCI design studio course can be thought of as an iterative and incremental teaching and learning process that blends HCI methods, design practice, and technology between a real and a virtual design studio. The course introduces problem-based learning to the pedagogies of project-based learning and studio-based learning currently employed in most HCI design studios. The positive student responses and our reflection and experiences focusing on a number of challenges for further implementation are also outlined.

Generic Project Definitions for Improvement of Health Care Delivery

The purpose of this article is to create actionable knowledge, making the definition of process improvement projects in health care delivery more effective. This study is a retrospective analysis of process improvement projects in hospitals, facilitating a case-based reasoning approach to project definition. Data sources were project documentation and hospital-performance statistics of 271 Lean Six Sigma health care projects from 2002 to 2009 of general, teaching, and academic hospitals in the Netherlands and Belgium. Objectives and operational definitions of improvement projects in the sample, analyzed and structured in a uniform format and terminology. Extraction of reusable elements of earlier project definitions, presented in the form of 9 templates called generic project definitions. These templates function as exemplars for future process improvement projects, making the selection, definition, and operationalization of similar projects more efficient. Each template includes an explicated rationale, an operationalization in the form of metrics, and a prototypical example. Thus, a process of incremental and sustained learning based on case-based reasoning is facilitated. The quality of project definitions is a crucial success factor in pursuits to improve health care delivery. We offer 9 tried and tested improvement themes related to patient safety, patient satisfaction, and business-economic performance of hospitals.