Abstract

During the last 15 to 20 years, evolving (neuro-) fuzzy systems (E(N) FS) have enjoyed more and more attraction in the context of data stream mining and modeling processes. This is because they can be updated on the fly in a single-pass sample-wise manner and are able to perform autonomous changes of the models on structural level in order to react onto process drifts. A wide variety of evolving (neuro-) fuzzy systems approaches have been proposed in order to handle data stream mining and modeling processes by dynamically updating the rule structure and antecedents. The current denominator in the update of the consequent (output weight) parameters is the usage of the recursive (fuzzily weighted) least squares estimator (R(FW) LS), as being applied in almost all E(N) FS approaches. In this paper, we propose and examine alternative variants for consequent parameter updates, namely multi-innovation RFWLS, recursive correntropy and especially recursive weighted total least squares (RWTLS). Multi-innovation RFWLS guarantees more stability in the update whenever structural changes (i.e. changes in the antecedents) in the E(N) FS are performed. This is because rule membership degrees are actualized on (a portion of) past samples and properly integrated in each update step. Recursive correntropy addresses the problematic of outliers by down-weighing the influence of higher errors in the parameter updates. Recursive weighted total least squares also takes into account a possible noise level in the input variables (and not solely in the target variable as done in RFWLS). The approaches are compared with standard RFWLS i.) on three data stream regression problems from practical applications, which are affected by noise levels and where one embeds a known drift, and ii.) on a time-series based forecasting problem. The results based on accumulated prediction error trends over time indicate that RFWLS can be largely outperformed by the proposed alternative variants, and this with even lower sensitivity on various data noise levels. So, the proposed variants could be worth of being further considered as promising and serious alternatives.

Highlights

  • Recursive weighted total least squares (RWTLS) (Section 6): an extension of the LS objective function formulation by integrating the sample reconstruction error to properly account for both, noise in the inputs and output(s) (whereas recursive fuzzily weighted least squares (RFWLS), which is based on the standard LS formulation, only respects noise in the output(s))

  • From the first part of the table, it can be realized that the conventional RFWLS method can be outperformed for all three data sets in terms of final achieved errors (MAE%): for engine test benches and YearMSD by all three alternative variants, for rolling mills only by multi-innovation LS

  • Our interpretation is that the forgetting has a much less intense effect on the update of the smallest eigenvector than on the update of the parameter vectors directly – an interesting observation where Recursive Weighted Total Least Squares (RWTLS) leaves some room for improvement in the future

Read more

Summary

Motivation and State-of-the-Art

In today’s industrial processes, social media and health-care systems [16], as well as in predictive maintenance and factories of the future (FoF) production lines [40], there is an increasing demand on autonomous and self-adaptive data-driven

Lughofer
Content of this paper
Problem statement
Multi-Innovation RFWLS
Recursive correntropy
Problem formulation and batch mode solution
Recursive solution
Recursive estimation of the noise covariance matrix
Method
Streaming data sets
Evolving learning engine
Evaluation strategy
Results on stream regression sets
Methods
Results on stream-based time-series forecasting
Noise sensitivity analysis
Parameter convergence analysis
Conclusion
Full Text
Published version (Free)

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call