Online Learning Mechanism Research Articles

Statistical Mechanics of On-Line Learning Using Correlated Examples

We consider a model composed of nonlinear perceptrons and analytically investigate its generalization performance using correlated examples in the framework of on-line learning by a statistical mechanical method. In Hebbian and AdaTron learning, the larger the number of examples used in an update, the slower the learning. In contrast, Perceptron learning does not exhibit such behaviors, and the learning becomes fast in some time region.

Online Learning Mechanisms for Bayesian Models of Word Segmentation

In recent years, Bayesian models have become increasingly popular as a way of understanding human cognition. Ideal learner Bayesian models assume that cognition can be usefully understood as optimal behavior under uncertainty, a hypothesis that has been supported by a number of modeling studies across various domains (e.g., Griffiths and Tenenbaum, Cognitive Psychology, 51, 354–384, 2005; Xu and Tenenbaum, Psychological Review, 114, 245–272, 2007). The models in these studies aim to explain why humans behave as they do given the task and data they encounter, but typically avoid some questions addressed by more traditional psychological models, such as how the observed behavior is produced given constraints on memory and processing. Here, we use the task of word segmentation as a case study for investigating these questions within a Bayesian framework. We consider some limitations of the infant learner, and develop several online learning algorithms that take these limitations into account. Each algorithm can be viewed as a different method of approximating the same ideal learner. When tested on corpora of English child-directed speech, we find that the constrained learner’s behavior depends non-trivially on how the learner’s limitations are implemented. Interestingly, sometimes biases that are helpful to an ideal learner hinder a constrained learner, and in a few cases, constrained learners perform equivalently or better than the ideal learner. This suggests that the transition from a computational-level solution for acquisition to an algorithmic-level one is not straightforward.

Segmenting CT prostate images using population and patient‐specific statistics for radiotherapy

In the segmentation of sequential treatment-time CT prostate images acquired in image-guided radiotherapy, accurately capturing the intrapatient variation of the patient under therapy is more important than capturing interpatient variation. However, using the traditional deformable-model-based segmentation methods, it is difficult to capture intrapatient variation when the number of samples from the same patient is limited. This article presents a new deformable model, designed specifically for segmenting sequential CT images of the prostate, which leverages both population and patient-specific statistics to accurately capture the intrapatient variation of the patient under therapy. The novelty of the proposed method is twofold: First, a weighted combination of gradient and probability distribution function (PDF) features is used to build the appearance model to guide model deformation. The strengths of each feature type are emphasized by dynamically adjusting the weight between the profile-based gradient features and the local-region-based PDF features during the optimization process. An additional novel aspect of the gradient-based features is that, to alleviate the effect of feature inconsistency in the regions of gas and bone adjacent to the prostate, the optimal profile length at each landmark is calculated by statistically investigating the intensity profile in the training set. The resulting gradient-PDF combined feature produces more accurate and robust segmentations than general gradient features. Second, an online learning mechanism is used to build shape and appearance statistics for accurately capturing intrapatient variation. The performance of the proposed method was evaluated on 306 images of the 24 patients. Compared to traditional gradient features, the proposed gradient-PDF combination features brought 5.2% increment in the success ratio of segmentation (from 94.1% to 99.3%). To evaluate the effectiveness of online learning mechanism, the authors carried out a comparison between partial online update strategy and full online update strategy. Using the full online update strategy, the mean DSC was improved from 86.6% to 89.3% with 2.8% gain. On the basis of full online update strategy, the manual modification before online update strategy was introduced and tested, the best performance was obtained; here, the mean DSC and the mean ASD achieved 92.4% and 1.47 mm, respectively. The proposed prostate segmentation method provided accurate and robust segmentation results for CT images even under the situation where the samples of patient under radiotherapy were limited. A conclusion that the proposed method is suitable for clinical application can be drawn.

Statistical Mechanics of Nonlinear On-line Learning for Ensemble Teachers

We analyze the generalization performance of a student in a model composed of nonlinear perceptrons: a true teacher, ensemble teachers, and the student. We calculate the generalization error of the student analytically or numerically using statistical mechanics in the framework of on-line learning. We treat two well-known learning rules: Hebbian learning and perceptron learning. As a result, it is proven that the nonlinear model shows qualitatively different behaviors from the linear model. Moreover, it is clarified that Hebbian learning and perceptron learning show qualitatively different behaviors from each other. In Hebbian learning, we can analytically obtain the solutions. In this case, the generalization error monotonically decreases. The steady value of the generalization error is independent of the learning rate. The larger the number of teachers is and the more variety the ensemble teachers have, the smaller the generalization error is. In perceptron learning, we have to numerically obtain the solutions. In this case, the dynamical behaviors of the generalization error are nonmonotonic. The smaller the learning rate is, the larger the number of teachers is; and the more variety the ensemble teachers have, the smaller the minimum value of the generalization error is.

Adaptive, Fast Walking in a Biped Robot under Neuronal Control and Learning

Human walking is a dynamic, partly self-stabilizing process relying on the interaction of the biomechanical design with its neuronal control. The coordination of this process is a very difficult problem, and it has been suggested that it involves a hierarchy of levels, where the lower ones, e.g., interactions between muscles and the spinal cord, are largely autonomous, and where higher level control (e.g., cortical) arises only pointwise, as needed. This requires an architecture of several nested, sensori–motor loops where the walking process provides feedback signals to the walker's sensory systems, which can be used to coordinate its movements. To complicate the situation, at a maximal walking speed of more than four leg-lengths per second, the cycle period available to coordinate all these loops is rather short. In this study we present a planar biped robot, which uses the design principle of nested loops to combine the self-stabilizing properties of its biomechanical design with several levels of neuronal control. Specifically, we show how to adapt control by including online learning mechanisms based on simulated synaptic plasticity. This robot can walk with a high speed (>3.0 leg length/s), self-adapting to minor disturbances, and reacting in a robust way to abruptly induced gait changes. At the same time, it can learn walking on different terrains, requiring only few learning experiences. This study shows that the tight coupling of physical with neuronal control, guided by sensory feedback from the walking pattern itself, combined with synaptic learning may be a way forward to better understand and solve coordination problems in other complex motor tasks.

Statistical Mechanics of On-line Learning When a Moving Teacher Goes around an Unlearnable True Teacher

In the framework of on-line learning, a learning machine might move around a teacher due to the differences in structures or output functions between the teacher and the learning machine. In this paper we analyze the generalization performance of a new student supervised by a moving machine. A model composed of a fixed true teacher, a moving teacher, and a student is treated theoretically using statistical mechanics, where the true teacher is a nonmonotonic perceptron and the others are simple perceptrons. Calculating the generalization errors numerically, we show that the generalization errors of a student can temporarily become smaller than that of a moving teacher and can reach the lowest value, even if the student only uses examples from the moving teacher. However, the generalization error of the student eventually becomes the same value with that of the moving teacher. This behavior is qualitatively different from that of a linear model.

Statistical Mechanics of Online Learning for Ensemble Teachers

We analyze the generalization performance of a student in a model composed of linear perceptrons: a true teacher, ensemble teachers, and the student. Calculating the generalization error of the student analytically using statistical mechanics in the framework of on-line learning, it is proven that when learning rate $\eta <1$, the larger the number $K$ and the variety of the ensemble teachers are, the smaller the generalization error is. On the other hand, when $\eta >1$, the properties are completely reversed. If the variety of the ensemble teachers is rich enough, the direction cosine between the true teacher and the student becomes unity in the limit of $\eta \to 0$ and $K \to \infty$.

Analysis of ensemble learning using simple perceptrons based on on-line learning theory

We discuss ensemble learning using K simple nonlinear perceptrons and a sign output function within the framework of statistical mechanics of on-line learning. One objective of statistical learning...

An incremental network for on-line unsupervised classification and topology learning

This paper presents an on-line unsupervised learning mechanism for unlabeled data that are polluted by noise. Using a similarity threshold-based and a local error-based insertion criterion, the system is able to grow incrementally and to accommodate input patterns of on-line non-stationary data distribution. A definition of a utility parameter, the error-radius, allows this system to learn the number of nodes needed to solve a task. The use of a new technique for removing nodes in low probability density regions can separate clusters with low-density overlaps and dynamically eliminate noise in the input data. The design of two-layer neural network enables this system to represent the topological structure of unsupervised on-line data, report the reasonable number of clusters, and give typical prototype patterns of every cluster without prior conditions such as a suitable number of nodes or a good initial codebook.

Neural network modeling and control of cement mills using a variable structure systems theory based on-line learning mechanism

It is well known that the major cause of instability in industrial cement ball mills is the so-called plugging phenomenon. A novel neural network adaptive control scheme for cement milling circuits that is able to fully prevent the mill from plugging is presented. Estimates of the one-step-ahead errors in control signals are calculated through a neural predictive model and used for controller tuning. A robust on-line learning algorithm, based on sliding mode control (SMC) theory is applied to both: to the controller and to the model as well. The proposed approach allows handling of mismatches, uncertainties and parameter changes in the model of the mill. The simulation results from indicate that both the neural model and the controller inherit the major advantages of SMC, i.e. robustness. Furthermore, learning is achieved in a rapid manner.

An Use of Reinforcement Learning in a Multi-Agent Environment.

In order to give more adaptability to a multiagent system, it would be desirable for each agent to have some kind of online learning ability for how to help and guide the other agents, i.e., how to cooperate with each other for the purpose of achieving a task which is given to the whole system. When we browse the field of computational methods of online learning, reinforcement learning seems a good candidate for this mechanism. However, due to the theoretical limitation that it assumes that an environment is Markovian, traditional reinforcement learning algorithms cannot be applied directly to this behavior acquisition problem in a multiagent environment. In this paper, an online learning mechanism is designed by using two reinforcement learning units. The first one is for predicting the move of the other agents at the next time-step, and the second one is for building the appropriate action rule set for an agent itself. Several computer simulations of the cooperative carrying problem are conducted to investigate the effectiveness of the proposed approach.

An Intelligent Sensor Validation and Fault Diagnostic Technique for Diesel Engines

Abstract An intelligent sensor validation and on-line fault diagnosis technique for a 6 cylinder turbocharged diesel engine is proposed and studied. A single auto-associative 3-layer Artificial Neural Network (ANN), is trained to examine the accuracy of the measured data and allocate a confidence level to each signal. The same ANN is used to recover the missing or faulty data with a close approximation. For on-line fault detection a feed-forward ANN is trained to classify and consequently recognize faulty and healthy behavior of the engine for a wide range of operating conditions. The proposed technique is also equipped with an on-line learning mechanism, which is activated when the confidence level in predicted fault is poor. It is hoped that a feasible, practical, and reliable sensor reading, as well as highly accurate fault diagnosis system, would be achieved.

Minimax partial distortion competitive learning for optimal codebook design

The design of the optimal codebook for a given codebook size and input source is a challenging puzzle that remains to be solved. The key problem in optimal codebook design is how to construct a set of codevectors efficiently to minimize the average distortion. A minimax criterion of minimizing the maximum partial distortion is introduced in this paper. Based on the partial distortion theorem, it is shown that minimizing the maximum partial distortion and minimizing the average distortion will asymptotically have the same optimal solution corresponding to equal and minimal partial distortion. Motivated by the result, we incorporate the alternative minimax criterion into the on-line learning mechanism, and develop a new algorithm called minimax partial distortion competitive learning (MMPDCL) for optimal codebook design. A computation acceleration scheme for the MMPDCL algorithm is implemented using the partial distance search technique, thus significantly increasing its computational efficiency. Extensive experiments have demonstrated that compared with some well-known codebook design algorithms, the MMPDCL algorithm consistently produces the best codebooks with the smallest average distortions. As the codebook size increases, the performance gain becomes more significant using the MMPDCL algorithm. The robustness and computational efficiency of this new algorithm further highlight its advantages.

Statistical Mechanics of Online Learning of Drifting Concepts: A Variational Approach

We review the application of statistical mechanics methods to the study of online learning of a drifting concept in the limit of large systems. The model where a feed-forward network learns from examples generated by a time dependent teacher of the same architecture is analyzed. The best possible generalization ability is determined exactly, through the use of a variational method. The constructive variational method also suggests a learning algorithm. It depends, however, on some unavailable quantities, such as the present performance of the student. The construction of estimators for these quantities permits the implementation of a very effective, highly adaptive algorithm. Several other algorithms are also studied for comparison with the optimal bound and the adaptive algorithm, for different types of time evolution of the rule.

Self-organizing rule-based control of multivariable nonlinear servomechanisms

By employing a feedforward and feedback PD control structure, this paper presents a simple approach to the problem of controlling multivariable nonlinear servomechanisms. The feedforward control action is deduced by a rule-based system employing a simplified fuzzy reasoning algorithm. Instead of relying on human experts the required rule-base is constructed automatically via a self-organizing counterpropagation network in cooperation with an on-line learning mechanism providing the required teacher signals. The convergence property of the learning mechanism is analyzed in some detail. Particular attention is paid to the problem of generalization, that is, the problem of how the learned knowledge can be used to handle novel situations without need for relearning. In the paper, it is suggested that local generalization may be achieved by nonlinear interpolation of fuzzy reasoning algorithm whereas linear generalization can be obtained by the appropriate utilization of the linear factor. A particular system under consideration is a multivariatle nonlinear passive line-of-sight (LOS) stabilization system. Simulation results on the LOS system have shown that the proposed control structure yields better performances than PD control alone, the rule-base can be constructed relatively fast in terms of requiring only a few learning cycles, and the suggested schemes for achieving generalization are useful and effective.