Recursive Prediction Error Algorithm Research Articles

Linear Models Applied to Monthly Seasonal Streamflow Series Prediction

Linear models are widely used to perform time series forecasting. The Autoregressive models stand out, due to their simplicity in the parameters adjustment based on close-form solution. The Autoregressive and Moving Average models (ARMA) and Infinite Impulse Response filters (IIR) are also good alternatives, since they are recurrent structures. However, their adjustment is more complex, since the problem has no analytical solution. This investigation performs linear models to predict monthly seasonal streamflow series, from to Brazilian hydroelectric plants. The goal is to reach the best achievable performance addressing linear approaches. We propose the application of recurrent models, estimating their parameters via an immune algorithm. To compare the optimization performance, the Least Mean Square (LMS) and Recursive Prediction Error (RPE) algorithms are utilized. Also, the AR model and the Holt-Winters method were performed. The results showed that the insertion of feedback loops increases the quality of the responses. The ARMA models optimized by the immune algorithms achieved the best overall performance.

Active vibration control for the time-varying systems with a new adaptive algorithm

This paper proposes a new adaptive algorithm for the active vibration control of time-varying systems in the presence of broadband or narrowband disturbances. The new algorithm combines the conventional filtered-x least mean square algorithm with the recursive prediction error (RPE) algorithm after the gradient modification of the RPE algorithm. The modified RPE algorithm is used to estimate the model of the control path online. The well-known filtered-x least mean square (FxLMS) algorithm is effective for the uncertain or time-varying systems, and adopts an auxiliary white noise approach to estimate the model of the control path online. However, the auxiliary excitation will degrade the control performance to some extent. In the new algorithm, the auxiliary excitation is eliminated at the expense of a larger computational burden. The influence of the estimated finite impulse response series on the convergence is also discussed. A propulsion shafting model with the time-varying dynamics is established by frequency response function synthesis. Numerical simulation for the established model is presented to demonstrate the superior performance of the proposed algorithm as compared with the FxLMS algorithm.

Research on grid-connected photovoltaic inverter based on quasi-PR controller adjusting by dynamic diagonal recurrent neural network

The single-phase grid-connected photovoltaic inverter system is studied in this paper. In view of the non-linear and time-varying characteristics of this system, the three-closed-loop control strategy consisting of DC voltage outer loop, grid-connected current inner loop and capacitive current inner loop based on quasi-PR control is proposed. Since the quasi-PR controller of fixed parameters is unable to adapt to changes of parameters in power network, a quasi-PR control method of dynamic self-tuning based on a dynamic Diagonal Recurrent Neural Network (DRNN) is presented. DRNN is based on the Recursive Prediction Error (RPE) algorithm with second-order gradient, which has a faster convergence rate than the BP algorithm. The simulation and experiment results prove that the grid-connected photovoltaic inverter with the above control algorithm has a good quality of the output current and fast performance in dynamic response.

Power Transformers Fault Diagnosis Based on DRNN

In recent years, improved three-ratio is an effective method for transformer fault diagnosis based on Dissolved Gas Analysis (DGA). In this paper, diagonal recurrent neural network (DRNN) is used to resolve the online fault diagnosis problems for oil-filled power transformer based on DGA. To overcome disadvantages of BP algorithm, a new recursive prediction error algorithm (RPE) is used in this paper.In addition, to demonstrate the effectiveness and veracity of the proposed method, some cases are used in the simulation. The simulation results are satisfactory.

Convergence analysis of a recursive prediction error method.

A convergence analysis is performed for a recursive prediction error algorithm discretised using the midpoint integration method. Several conditions are formulated such that the stability of an associated differential equation can be tied to the local and global convergence properties of the algorithm. This shows that convergence to the true parameters is possible. The theoretical analysis of this paper is complemented by numerical example.

Intelligent computer vision system for Sprague-Dawley rat sperm classification using gray-level microscopic images

This paper presents a development of intelligent computer vision system for Sprague-Dawley rat sperm classification. The system is developed to help pathologists and improve the current conventional detection methods and classification of Sprague-Dawley rat sperm. The conventional method tends to result in errors and is also time consuming. Thus, the new computer vision system is developed to overcome these problems and, consequently, produce more accurate results. The system is capable of classifying rat sperm into two groups, namely, normal and abnormal, based on the morphological characteristics of the sperm head. Furthermore, the system has the ability to classify the shapes of abnormal sperms which are banana and hookless shapes. The new system consists of four stages. In the first stage, sperms are segmented using the adaptive fuzzy moving K-means (AFMKM) clustering algorithm. In the second stage, a new automatic cropping algorithm was used to isolate the sperm head and to exclude other details. In the third stage, the feature extraction process was performed, wherein five features namely curvature angle, two flattened angles, curvature depth and number of curvatures are proposed to classify the sperm head of the rat. In the fourth (final) stage, the sperm head is classified using hybrid multilayered perceptron (HMLP) neural network trained with modified recursive prediction error (MRPE) algorithm. The developed system achieved an accuracy rate of 98.7%. Key words: Sprague-Dawley rat sperm, adaptive fuzzy moving K-means clustering algorithm, hybrid multilayered perceptron network, modified recursive prediction error algorithm.

Sprague Dawley rat sperm classification using hybrid multilayered Perceptron network

As of now, the analysis of sperm such as counting and detection processes are still operated manually which tends to produce errors. False detection in sperm analysis must be minimized as possible. Therefore, the current study focuses on developing a Sprague Dawley rat sperm classification system to assist the detection process by pathologist. The system has the ability to classify the Sprague Dawley rat sperm into two classes namely normal and abnormal based on the morphological characteristic of sperm's head. The proposed system employs the Hybrid Multilayered Perceptron (HMLP) neural network trained with the Modified Recursive Prediction Error (MRPE) algorithm as the intelligent classifier tool. This study will also investigate three significant morphological characteristics of rat sperm's head namely opened degree, width of the curve and template matching percentage as the input data for the HMLP network. Furthermore, this study further classifies the abnormal sperm into hookless and banana shape. Promising result with 100% and 94.62% of accuracy has been achieved for two classes and three classes classification of rat sperm respectively.

Distributed and Recursive Parameter Estimation in Parametrized Linear State-Space Models

We consider a network of sensors deployed to sense a spatio-temporal field and infer parameters of interest about the field. We are interested in the case where each sensor's observation sequence is modeled as a state-space process that is perturbed by random noise, and the models across sensors are parametrized by the same parameter vector. The sensors collaborate to estimate this parameter from their measurements, and to this end we propose a distributed and recursive estimation algorithm, which we refer to as the incremental recursive prediction error algorithm. This algorithm has the distributed property of incremental gradient algorithms and the on-line property of recursive prediction error algorithms.

Modified Recursive Least Squares algorithm to train the Hybrid Multilayered Perceptron (HMLP) network

In this paper, a new learning algorithm, called the Modified Recursive Least Square (MRLS), is introduced for the Hybrid Multilayered Perceptron (HMLP) network. Adopting the Recursive Least Square (RLS) algorithm as its basis, the MRLS algorithm differs from RLS in the way that the weight of the linear connections for the HMLP network is estimated. The convergence rate of the MRLS algorithm is further improved by varying the forgetting factor, optimizing the way the momentum and learning rate are assigned. To investigate its applicability, the MRLS algorithm is demonstrated on the HMLP network using six benchmark data sets obtained from the UCI repository. The classification performance of the HMLP network trained with the MRLS algorithm is compared with those of the HMLP network trained with the Modified Recursive Prediction Error (MRPE) algorithm and the MLP trained with the standard RLS algorithm as well as with other commonly adopted machine learning classifiers. The comparison results indicated that the proposed MRLS trained HMLP network provides significant improvement over RLS trained MLP network, MRPE trained HMLP network, and other machine learning classifiers in terms of accuracy, convergence rate and mean square error (MSE).

On the identification of model structure in hydrological and environmental systems

The paper presents a new recursive estimation algorithm designed expressly for the purpose of model structure identification (not for state estimation or primarily for parameter estimation) and discusses two applications thereof, one to a motivating, hypothetical example and one to data from whole‐pond manipulations designed to explore sediment‐nutrient‐phytoplankton dynamics. The algorithm is the current culmination of a long‐term technical development from state estimation using a Kalman filter, through state parameter estimation using an extended Kalman filter, through a recursive prediction error (RPE) algorithm for parameter estimation cast in the state space and recently modified for estimating time‐varying model parameters, to an RPE algorithm for estimating time‐varying parameters but cast in a parameter space formulation. It is concluded that the algorithm performs well, in the sense of being robust and indeed in revealing specifically where (but less so exactly how) a prior candidate model's structure may be in error.

A locally recurrent fuzzy neural network with application to the wind speed prediction using spatial correlation

In this paper, a locally feedback dynamic fuzzy neural network (LF-DFNN) for modeling of temporal processes is suggested. The model is composed of dynamic TSK-type fuzzy rules where the consequent sub-models are implemented by recurrent neural networks with internal feedback paths and dynamic neuron synapses. The LF-DFNN exhibits some interesting features, such as enhanced representation power, local modeling characteristics, model parsimony, and stable learning. Training of the LF-DFNN models is achieved using an optimal on-line learning scheme, the decoupled recursive prediction error algorithm (DRPE). The method has reduced computational demands and is derived through decomposition of the weight vector to several mutually exclusive weight groups. The partial derivatives required for the implementation of the training algorithm are calculated using the adjoint model approach, adapted to the fuzzy network's architecture exercised here. The paper deals with the wind speed prediction in wind farms, using spatial information from remote measurement stations. The LF-DFNN networks are used as advanced forecast models, providing multi-step ahead wind speed estimates from 15 min to 3 h ahead. Extensive simulation results demonstrate that our models exhibit superior performance compared to other network types suggested in the literature. Furthermore, it is shown that DRPE outperforms three gradient descent algorithms, in training of the recurrent forecast models.

Suitable features selection for the HMLP and MLP networks to identify the shape of aggregate

Aggregates are one of the main ingredients for producing concrete. In order to produce ‘high performance’ concrete, the properties of aggregates matter most. In particular, shape and surface texture of aggregates immensely influence the strength and structure of the resulting concrete. Often, low quality shape aggregates reduce the durability and strength of the concrete with much more requirement of sand, cement and water. Because the strength of concrete depends on the ratio of well-shaped aggregates to poor shape ones contained in the concrete mixture, the classification process for aggregates into well shaped and poor shaped is an essential step toward high quality concrete. Traditional technique requires aggregate to pass a stringent series of mechanical, chemical and physical test in order to demonstrate that they will perform satisfactorily, and meet or exceed specifications. Such stringent tests, often perform manually, tends to be slow, highly subjective and laborious. Therefore, an intelligent system that can separate the aggregates automatically could help to overcome these problems. This paper discusses an implementation of an intelligent classification system for the aggregate particles using neural network. The features obtained with image analysis, which are area, perimeter, Hu’s moments, Zernike’s moments of the aggregate’s area and perimeter were used as input data for the neural network. The hybrid multilayered perceptron network trained using modified recursive prediction error algorithm is employed to perform the classification. Performance analysis of the hybrid multilayered perceptron network is compared with the standard multilayered perceptron network trained using four different training algorithms, i.e. recursive prediction error, Bayesian Regulazation, Levenberg–Marquardt and scale conjugate gradient. The experimental results show that the performance for hybrid multilayered perceptron and multilayered perceptron networks are significantly better using the moments extracted based on object’s perimeter compared to object’s area. Model comparison confirms that the selected features combined with hybrid multilayered perceptron network attains the best performance in aggregates recognition.

Understanding complex environmental systems: a dual approach

AbstractAn approach to interpreting field data exploiting the duality of data‐ and theory‐based models, and their associated methods of system identification, is presented. This approach seeks to overcome the respective limitations of the two branches of the duality: that theory‐based models are not unambiguously identifiable from the observations, while a well‐identified data‐based model may not be capable of a satisfactory theoretical interpretation. The purpose of the approach is thereby to gain a deeper understanding of complex, poorly defined environmental systems. Recursive methods of time‐series analysis are used to identify the data‐based models (as transfer functions) and companion recursive methods, specifically the recursive prediction error (RPE) algorithm, are employed for structure identification and parameter estimation of the theory‐based models (in ordinary differential equation forms). The results of these identification exercises for the two classes of models can be compared in terms of the macro‐parameters of the studied system's time constant and steady‐state gain. Two case studies are presented to illustrate the overall performance of the dual‐thrust approach, on an activated sludge system and an aquaculture pond. It is found that: (1) as opposed to the exclusive use of either approach alone, more is to be gained through the joint application of the two classes of models, which historically have tended to reflect quite separate, unconnected approaches to interpreting environmental systems data; (2) to some extent, identifying the structure and estimating the parameters of one type of model can be readily improved by recourse to the corresponding results for the other; and (3) reconciliation of the results from identifying the two classes of model in the parameter space—of the time constant and steady‐state gain—has significant advantages over the more familiar process of evaluating a model's performance in the terms of its (observed) state‐space features. Copyright © 2006 John Wiley & Sons, Ltd.

On adaptive optimal input design: A bioreactor case study

AbstractThe problem of optimal input design (OID) for a fed‐batch bioreactor case study is solved recursively. Here an adaptive receding horizon optimal control problem, involving the so‐called E‐criterion, is solved “on‐line,” using the current estimate of the parameter vector θ at each sample instant {tk, k = 0, …, N − h}, where N marks the end of the experiment and h is the control horizon for which the input design problem is solved. The optimal feed rate F(tk) thus obtained is applied and the observation y(tk+1) that becomes available is subsequently used in a recursive prediction error algorithm to find an improved estimate of the actual parameter estimate θˆ(tk). The case study involves an identification experiment with a Rapid Oxygen Demand TOXicity device (RODTOX) for estimation of the biokinetic parameters μmax and KS in a Monod type of growth model. It is assumed that the dissolved oxygen probe is the only instrument available, which is an important limitation. Satisfactory results are presented and compared to a “naïve” input design in which the system is driven by an independent binary random sequence. This comparison shows that the OID approach yields improved confidence intervals on the parameter estimates. © 2006 American Institute of Chemical Engineers AIChE J, 2006

Long-Term Wind Speed and Power Forecasting Using Local Recurrent Neural Network Models

This paper deals with the problem of long-term wind speed and power forecasting based on meteorological information. Hourly forecasts up to 72-h ahead are produced for a wind park on the Greek island of Crete. As inputs our models use the numerical forecasts of wind speed and direction provided by atmospheric modeling system SKIRON for four nearby positions up to 30 km away from the wind turbine cluster. Three types of local recurrent neural networks are employed as forecasting models, namely, the infinite impulse response multilayer perceptron (IIR-MLP), the local activation feedback multilayer network (LAF-MLN), and the diagonal recurrent neural network (RNN). These networks contain internal feedback paths, with the neuron connections implemented by means of IIR synaptic filters. Two novel and optimal on-line learning schemes are suggested for the update of the recurrent network's weights based on the recursive prediction error algorithm. The methods assure continuous stability of the network during the learning phase and exhibit improved performance compared to the conventional dynamic back propagation. Extensive experimentation is carried out where the three recurrent networks are additionally compared to two static models, a finite-impulse response NN (FIR-NN) and a conventional static-MLP network. Simulation results demonstrate that the recurrent models, trained by the suggested methods, outperform the static ones while they exhibit significant improvement over the persistent method.

Towards a synthesis of data-based and theory-based models of environmental systems

A two-pronged approach to interpreting field data through the use of models is presented. This approach builds upon both data- and theory-based models and their associated methods of system identification. It seeks to overcome their respective limitations: that theory-based models are not unambiguously identifiable from the observations, while a well identified data-based model may not be capable of a satisfactory theoretical interpretation. The purpose of the approach is thereby to gain a deeper understanding of complex environmental systems. Recursive methods of time-series analysis are used to identify the data-based models and the modified recursive prediction error algorithm is employed for parameter estimation of the theory-based models. The results of these identification exercises for the two classes of models can be compared in terms of the macro-parameters of the studied system's time constant and steady-state gain. Two case studies are presented to illustrate the overall performance of the two-pronged approach. It is found that: (1) more is to be gained through the joint application of the two classes of models than the exclusive use of either; (2) to some extent, identifying the structure and estimating the parameters of one type of model can be improved by recourse to the corresponding results for the other; and (3) reconciliation of the results from identifying the two classes of model in the parameter space has significant advantages over the more familiar process of evaluating a model's performance in the terms of its (observed) state space features.

Recursive prediction error identification and scaling of non-linear state space models using a restricted black box parameterization

A recursive prediction error algorithm for identification of systems described by non-linear ordinary differential equation (ODE) models is presented. The model is an ODE model, parameterized with coefficients of a multi-variable polynomial that describes one component of the right-hand side function of the ODE. This avoids over-parameterization problems. The selected model can also handle systems with more complicated right-hand side structure, by identification of a local input–output equivalent system in the coordinate system of the selected state variables. A novel technique based on scaling of the sampling period is proposed. The technique can improve the conditioning of the identification problem, thereby enhancing the chances of convergence to the correct minimum of the criterion. The algorithm is applied to live data from a system consisting of two cascaded tanks, with promising results. A MATLAB software package, which implements the proposed algorithm and a set of support scripts, can be freely downloaded from http://www.it.uu.se/research/reports/.

Locally recurrent neural networks for long-term wind speed and power prediction

The paper deals with a real-world application, the long-term wind speed and power forecasting in a wind farm using locally recurrent multilayer networks as forecast models. To cope with the complexity of the process and to improve the performance of the models, a class of optimal on-line learning algorithms is employed for training the locally recurrent networks based on the recursive prediction error (RPE) algorithm. A global RPE algorithm is devised and three local learning algorithms are suggested by partitioning the GRPE into a set of sub-problems at the neuron level to reduce computational complexity and storage requirements. Experimental results on the wind prediction problem demonstrate that the proposed algorithms exhibit enhanced performance, in terms of convergence speed and the accuracy of the attained solutions, compared to conventional gradient-based methods. Furthermore, it is shown that the suggested recurrent forecast models outperform the atmospheric and time-series models.

Fault detection in a real-time monitoring network for water quality in the lagoon of Venice (Italy)

In the context of monitoring water quality in natural ecosystems in real time, on-line data quality control is a very important issue for effective system surveillance and for optimizing maintenance of the monitoring network. This paper presents some applications of recursive state-parameter estimation algorithms to real-time detection of signal drift in high-frequency observations. Two continuous-discrete recursive estimation schemes, namely the Extended Kalman Filter and the Recursive Prediction Error algorithm, were applied to assuring the quality of the dissolved oxygen (DO) time series, as obtained from the Lagoon of Venice (Italy) during August 2002, through the real-time monitoring network of the Magistrato alle Acque (the Venice Water Authority). Results demonstrate the effectiveness of the methodology in early detection of a probable drift in the DO signal. Comparison of these results with those obtained from the application of a related recursive scheme (a Dynamic Linear Regression procedure) suggests the strong benefits of approaching the problem of on-line data quality control with several (not merely a single) independent such estimation methods.

Sequential and Reinforcement Learning for Neurocontrol

This paper presents a novel sequential learning neural network implementation of action dependent adaptive critics. Sequential learning neural networks provide a systematic way of adding neurons in response to new data features as well as removing neurons which cease to contribute to the overall performance of the network. The convergence rate of the sequential learning method is enhanced by applying a modified Recursive Prediction Error algorithm to adjust network parameters. The new methodology, which provides a fully autonomous controller, is benchmarked against the conventional MLP neurocontroller on a highly nonlinear inverted pendulum system and shown to achieve superior perfonnance.