An online learning algorithm with dimension selection using minimal hyper basis function networks

Abstract

In this study, the authors propose an HMRAN (Hyper MRAN) for which dimension selection is possible, by extending a minimal resource allocating network (MRAN) for online learning with minimized resources, with the application of a growth strategy and pruning strategy for hidden units in a Gaussian radial basis function (GRBF) network. When the input is a multidimensional pattern, it is often the case that dimensions with an explicit relationship to the output are limited to a single section and it is difficult to develop a correspondence with a MRAN having an isotropic Gaussian function as the basis function. Dimension selection can be flexibly performed in basis function units with the use of a hyper basis function (Hyper BF) instead of a Gaussian function. Furthermore, computational complexity is reduced by using a localized extended Kalman filter (LEKF) and the introduction of a merging strategy for the basis function and improvements to the pruning strategy resulted in a reduction of the resources needed. A number of benchmark tests were performed and it was confirmed that the proposed technique can be implemented with good accuracy and minimal resources compared to the conventional technique. © 2006 Wiley Periodicals, Inc. Syst Comp Jpn, 37(11): 11–21, 2006; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.20507