Incremental constructive ridgelet neural network

Abstract

In this paper, a new kind of neural network is proposed by combining ridgelet with feedforward neural network (FNN). The network adopts ridgelet as the activation function in the hidden layer, and an incremental constructive method is employed to determine the structure of the network. Since ridgelets are efficient in describing linear, curvilinear, and hyperplane like structures in high dimensions, accordingly the network can approximate quite a wide range of multivariate functions in a more stable and efficient way, especially those with certain kinds of spatial inhomogeneities. Moreover, the incremental extreme learning machine makes adding the hidden nodes one by one possible, and it only needs to adjust the output weights linking the hidden layer and the output layer when more hidden neurons are added. By defining the cost function as the difference between the previously approximated function and the currently approximating one, a genetic algorithm is used to determine the optimal directions of ridgelet neurons. The construction and learning of the network are presented in detail. The superiority of the proposed model is demonstrated by simulation experiments in function learning and image compression.