A sum-of-product neural network (SOPNN)

Abstract

Listen

Translate article

This paper presents a sum-of-product neural network (SOPNN) structure. The SOPNN can learn to implement static mapping that multilayer neural networks and radial basis function networks normally perform. The output of the neural network has the sum-of-product form ∑ Np i=1∏ N v j=1 f ij (x j) , where x j 's are inputs, N v is the number of inputs, f ij( ) is a function generated through network training, and N p is the number of product terms. The function f ij ( x j ) can be expressed as ∑ k w ijk B jk ( x j ), where B jk( ) is a single-variable basis function and W ijk 's are weight values. Linear memory arrays can be used to store the weights. If B jk( ) is a Gaussian function, the new neural network degenerates to a Gaussian function network. This paper focuses on the use of overlapped rectangular pulses as the basis functions. With such basis functions, W ijk B jk ( x j ) will equal either zero or W ijk , and the computation of f ij ( x j ) becomes a simple addition of some retrieved W ijk 's. The structure can be viewed as a basis function network with a flexible form for the basis functions. Learning can start with a small set of submodules and have new submodules added when it becomes necessary. The new neural network structure demonstrates excellent learning convergence characteristics and requires small memory space. It has merits over multilayer neural networks, radial basis function networks and CMAC in function approximation and mapping in high-dimensional input space. The technique has been tested for function approximation, prediction of a time series, learning control, and classification.