Generalized Relief Error Networks and Patterns with Lower Errors

Abstract

For the standard Back-Propagation training algorithm, the objective function is defined as the squared error between the actual and desired output. In [3], we introduced a modular system for training BP-networks by means of the so-called Generalized Relief Error Networks (GREN-networks) approximating the unknown objective function. In comparison with the standard Back-Propagation training algorithm, the proposed system does not require the knowledge of the desired output activity for the proper weight adjustment. During training, the necessary error terms are determined by the pre-trained GREN-network. Furthermore, GREN-networks can be applied for searching patterns similar to those presented to and recalled by the trained BP-network, but with a lower error at the output of the GREN-network. This searching process is performed iteratively by the applied GREN-network. Here, we will focus on analyzing the performance of the above model, especially with regard to its knowledge extraction ability. Experimen...