Sequential Learning Method Research Articles

Beyond semantic attributes: Auxiliary feature discovery for image classification

Semantic attributes have been proved to be an effective representation approach for image classification in recent years. However, most of the existing semantic attributes are explicitly pre-defined and the attribute number is also very limited in practice. Therefore, we often encounter difficulties by using a fixed incomplete semantic attribute set for different classification tasks. One possible solution is to expand the semantic attribute representation with some non-semantic mid-level features. However, how to make the expanded features more effective and discriminative is still seldom exploited. In this paper, we propose a Sequential augmented Attributes learning (SAL) method to implement semantic attribute augmentation. In our SAL method, the non-semantic mid-level features are learned one by one under a sequential error-correcting scheme so that we can obtain more discriminating power with very compact expanded features. Extensive experiments are conducted on two public datasets and the results show that our approach achieves encouraging performance.

Early detection and classification of microaneurysms in retinal fundus images using sequential learning methods

Diabetic retinopathy is the most common diabetic eye disease and causes blindness if not treated on time. Microaneurysms are one of the first clinical signs of diabetic retinopathy and appear as small red dots on fundus images. The incidence of blindness can be reduced by detecting microaneurysms at an earlier stage. In this paper, a new method is proposed for detection of microaneurysms from the colour fundus retinal images to assist the eye care specialist to examine large populations of patient. The microaneurysms are detected from the colour fundus image by applying the pre-processing techniques in order to remove the optic disc and similar blood vessels using morphological operations. The pre-processed image was then used for feature extraction, and these features were used for the purpose of classification. The classifiers used are support vector machine, Meta-cognitive Neural Network (McNN) and Self-adaptive Resource Allocation Network (SRAN), and their performances are compared and presented.

Weighted Online Sequential Extreme Learning Machine for Class Imbalance Learning

Most of the existing sequential learning methods for class imbalance learn data in chunks. In this paper, we propose a weighted online sequential extreme learning machine (WOS-ELM) algorithm for class imbalance learning (CIL). WOS-ELM is a general online learning method that alleviates the class imbalance problem in both chunk-by-chunk and one-by-one learning. One of the new features of WOS-ELM is that an appropriate weight setting for CIL is selected in a computationally efficient manner. In one-by-one learning of WOS-ELM, a new sample can update the classification model without waiting for a chunk to be completed. Extensive empirical evaluations on 15 imbalanced datasets show that WOS-ELM obtains comparable or better classification performance than competing methods. The computational time of WOS-ELM is also found to be lower than that of the competing CIL methods.

A Comprehensive Evaluation of Weight Growth and Weight Elimination Methods Using the Tangent Plane Algorithm

The tangent plane algorithm is a fast sequential learning method for multilayered feedforward neural networks that accepts almost zero initial conditions for the connection weights with the expectation that only the minimum number of weights will be activated. However, the inclusion of a tendency to move away from the origin in weight space can lead to large weights that are harmful to generalization. This paper evaluates two techniques used to limit the size of the weights, weight growing and weight elimination, in the tangent plane algorithm. Comparative tests were carried out using the Extreme Learning Machine which is a fast global minimiser giving good generalization. Experimental results show that the generalization performance of the tangent plane algorithm with weight elimination is at least as good as the ELM algorithm making it a suitable alternative for problems that involve time varying data such as EEG and ECG signals.

Online annotation of faces in personal videos by sequential learning

This paper addresses semi-automatic annotation of faces in personal videos. Different from previous offline annotation systems, this paper studies online annotation of faces. During an annotation session, few annotations are requested from the user only for some part of the video online. These annotations are used to train a system that will perform annotation automatically for the rest of the video. The automatic annotation results are presented to the user during the same session and the user is allowed to correct any automatic annotation mistakes. Thus, only fast and accurate face recognition methods are considered. Instead of batch learning, which has been used in the existing annotation systems, this paper proposes sequential learning methods to be used as face classifiers. Different classification methods are tested with various feature extraction methods using the same database so that a fair comparison is made among them. The results are evaluated in terms of recognition accuracies and execution time requirements.

Progressive interactive training: A sequential neural network ensemble learning method

This paper introduces a progressive interactive training scheme (PITS) for neural network (NN) ensembles. The scheme trains NNs in an ensemble one by one in a sequential fashion where the outputs of all previously trained NNs are stored and updated in a common location, called information center (IC). The communication among NNs is maintained indirectly through IC , reducing interaction among NNs. In this study, PITS is formulated as a derivative of simultaneous interactive training, negative correlation learning. The effectiveness of PITS is evaluated on a suite of 20 benchmark classification problems. The experimental results show that the proposed training scheme can improve the performance of ensembles. Furthermore, the PITS is incorporated with two very popular ensemble training methods, bagging and boosting. It is found that the performance of bagging and boosting algorithms can be improved by incorporating PITS with their training processes.

A novel compensation-based recurrent fuzzy neural network and its learning algorithm

Based on detailed study on several kinds of fuzzy neural networks, we propose a novel compensation-based recurrent fuzzy neural network (CRFNN) by adding recurrent element and compensatory element to the conventional fuzzy neural network. Then, we propose a sequential learning method for the structure identification of the CRFNN in order to confirm the fuzzy rules and their correlative parameters effectively. Furthermore, we improve the BP algorithm based on the characteristics of the proposed CRFNN to train the network. By modeling the typical nonlinear systems, we draw the conclusion that the proposed CRFNN has excellent dynamic response and strong learning ability.

Non-visual game design and training in gameplay skill acquisition – A puzzle game case study

This paper reports the results of a study on the design and evaluation of the game and techniques which allow puzzles to be played in the absence of visual feedback. We have demonstrated that a camera-mouse can be used successfully for blind navigation and target location acquisition within a game field. To gradually teach the players the sequential learning method was applied. Blind exploration of the gamespace was augmented with sticky labels and overview sound cues, verbal and non-verbal, which can significantly reduce the cognitive load and facilitate mental matching and integration. The full-sticky labels technique does not require fine motor skills and allows a user to gain control over the game with a minimum level of skills. With the vertical sticky labels technique training was focused on the development of accurate head movements only on a horizontal plane. With practice, the players can use the non-sticky labels technique. After 240 trials (3–4 h), the cumulative experience of the blindfolded players was increased 22.5–27 times compared to the initial 10 trials.

Bagging Linear Sparse Bayesian Learning Models for Variable Selection in Cancer Diagnosis

This paper investigates variable selection (VS) and classification for biomedical datasets with a small sample size and a very high input dimension. The sequential sparse Bayesian learning methods with linear bases are used as the basic VS algorithm. Selected variables are fed to the kernel-based probabilistic classifiers: Bayesian least squares support vector machines (BayLS-SVMs) and relevance vector machines (RVMs). We employ the bagging techniques for both VS and model building in order to improve the reliability of the selected variables and the predictive performance. This modeling strategy is applied to real-life medical classification problems, including two binary cancer diagnosis problems based on microarray data and a brain tumor multiclass classification problem using spectra acquired via magnetic resonance spectroscopy. The work is experimentally compared to other VS methods. It is shown that the use of bagging can improve the reliability and stability of both VS and model prediction.

A sequential subspace learning method and its application to dynamic texture analysis

Incremental update of subspace has new and interesting research applications in vision such as active recognition, object tracking and dynamic texture analysis. In this paper, a sequential subspace learning method is proposed for dynamic texture analysis. The learning algorithm can update adaptively dynamic texture subspace based on sequential observation data, and has higher computation efficiency and numerical stableness. Also our learning method considers the change of the texture sample mean when each new observation datum arrives, whereas existing subspace learning methods ignore the fact that the sample mean varies over time. Experimental results show the learning method for dynamic texture subspace is efficient and effective.

Computationally efficient sequential learning algorithms for direct link resource-allocating networks

Computationally efficient sequential learning algorithms are developed for direct-link resource-allocating networks (DRANs). These are achieved by decomposing existing recursive training algorithms on a layer by layer and neuron by neuron basis. This allows network weights to be updated in an efficient parallel manner and facilitates the implementation of minimal update extensions that yield a significant reduction in computation load per iteration compared to existing sequential learning methods employed in resource-allocation network (RAN) and minimal RAN (MRAN) approaches. The new algorithms, which also incorporate a pruning strategy to control network growth, are evaluated on three different system identification benchmark problems and shown to outperform existing methods both in terms of training error convergence and computational efficiency.

Sequential and Reinforcement Learning for Neurocontrol

This paper presents a novel sequential learning neural network implementation of action dependent adaptive critics. Sequential learning neural networks provide a systematic way of adding neurons in response to new data features as well as removing neurons which cease to contribute to the overall performance of the network. The convergence rate of the sequential learning method is enhanced by applying a modified Recursive Prediction Error algorithm to adjust network parameters. The new methodology, which provides a fully autonomous controller, is benchmarked against the conventional MLP neurocontroller on a highly nonlinear inverted pendulum system and shown to achieve superior perfonnance.

Kernel Relaxation과 동적 모멘트를 조합한 Support Vector Machine의 학습 성능 향상

본 논문에서는 커널완화법과 동적모멘트를 이용한 support vector machines의 학습성능 개선을 제안하였다. 제안된 학습 방법은 기존의 정적모멘트와는 달리 수렴 정도에 따라 현재의 학습에 과거의 학습 속성을 반영하는 동적모멘트 방법이다. 기존의 정적 상수로 정의된 모멘트가 전체 학습에 동등하게 영향을 주는 반면 제안된 동적모멘트를 이용한 학습방법은 학습 수행에 따라 동적으로 모멘트를 변경함으로써 수렴속도와 학습 성능을 효과적으로 제어할 수 있다. 제안된 학습법을 support vector machine의 새로운 순차 학습 방법인 커널완화법에 적용하였다. 신경망 분류기 표준 평가 데이터인 SONAR 데이터를 이용하여 실험한 곁과 동적모멘트를 이용한 방법이 수렴속도와 학습 성능면에서 기존의 커널완화법과 정적모멘트를 이용한 학습법에 비해 향상된 성능을 보이는 것을 확인하였다. This paper proposes learning performance improvement of support vector machine using the kernel relaxation and the dynamic momentum. The dynamic momentum is reflected to different momentum according to current state. While static momentum is equally influenced on the whole, the proposed dynamic momentum algorithm can control to the convergence rate and performance according to the change of the dynamic momentum by training. The proposed algorithm has been applied to the kernel relaxation as the new sequential learning method of support vector machine presented recently. The proposed algorithm has been applied to the SONAR data which is used to the standard classification problems for evaluating neural network. The simulation results of proposed algorithm have better the convergence rate and performance than those using kernel relaxation and static momentum, respectively.

A sequential learning method for boundary detection

In various fields digitized pictures are sometimes analysed, where it is desired to find the boundary of a figure on a background, when the background level is not strictly uniform but may be subject to very slow spatial variations of a certain importance with respect to the figure-background contrast. Moreover, the demarcation between the regions “figure” and “background” is at times characterized by a more or less sharp discontinuity of the gradient, rather than a level discontinuity. As is known, in such cases the boundary detection is frequently rather critical, if the image is blurred by a superimposed noise. In this work we propose a sequential learning method for boundary detection, which is particularly efficient in such critical cases. This method is based on using the information supplied by the pixels while they are been classified “on the boundary”, to obtain a Bayesian updating of the information on the two classes “figure” and “background”. This updating allows the decision rule to “learn” slow variations of both the background level and the gradient near the boundary in the “figure” region. This method works sequentially, since at each step it determines a new boundary pixel, referring to the likelihood of all the neighbours of the previous boundary pixel, by minimizing the “ a posteriori” probability of error.

A real-time simulator of the rainfall-runoff process

The real-time surveilance of evolving hydrological systems is particularly attractive in situations where models are used in a control capacity. If the parameters of a process are time variant, it is meaningless to approach the identification problem in any way other than to track their variation in real-time. A study is presented of an adaptive rainfall-runoff model specifically designed for use in real-time forecasting situations. The model is essentially a linear input-output model that utilises an estimate of the effective rainfall input to predict the outflow from a catchment some time ahead. The model parameters are re-estimated at each sampling instant by utilising a sequential learning method to obtain the impulse response of the system. The convolution integral is then employed to convert rainfall to runoff. The forecast errors are corrected by using a digital filter, and a combination forecast based upon this correction is issued for several time periods ahead. The model is applied to several actual storm events and indications are given about its practical application and performance.