Partially Connected Locally Recurrent Probabilistic Neural Networks

Abstract

In this chapter, we review existing locally recurrent neural networks and introduce a novel artificial neural network architecture that merges the locally recurrent probabilistic neural networks (LRPNN) with swarm intelligence algorithms and concepts. In particular, we develop an enhanced LRPNN model, referred to as Partially Connected LRPNN (PC-LRPNN). In contrast to LRPNN, where the recurrent layer consists of a set of fully connected neurons, the proposed new architecture assumes a swarm of neurons in the recurrent layer. Each neuron of the swarm presumes a neighbourhood of neurons with which it communicates through interconnections. The locality that determines the neighbourhoods is defined based on existing neighbourhood and communication schemes proposed in the swarm intelligence literature. Obviously, the PC-LRPNN offers a more general scheme, in which the fully connected LRPNN can be considered as a particular case, where all links in the recurrent layer are implemented. The neighbourhood topology of the new, swarm-based recurrent layer can be either static or dynamic. Dynamic neighbourhoods have been studied extensively in the field of swarm intelligence, since swarms with dynamic communication schemes among individuals have been shown to achieve remarkably better results than swarms with static communication schemes in the field of optimization. Also, the plasticity of the neighbourhoods can be useful in cases where better fit to unknown data is required. In the present chapter we will limit our exposition to the static neighbourhoods, which are defined once during training, and remain unchanged during the operation of the PC-LRPNN. However, the concepts that we introduce here can be extended further to the dynamic counterparts. The aforementioned local neighbourhoods and communications schemes facilitate the optimization of the recurrent layer linkage, which leads to much faster operation of the neural network, when compared to the fully linked structure. Furthermore, it significantly reduces the computational load for the overall training of the recurrent layer, which is performed at each case using the Particle Swarm Optimization (PSO) algorithm. Equipping the PC-LRPNN with PSO, results in an efficient hybrid scheme that takes advantage of the virtues of the probabilistic neural networks (PNN), recurrent neural networks (RNN), swarm intelligence concept, and that can tackle successfully real-life classification problems that assume temporal or spatial correlations among subsequent events. O pe n A cc es s D at ab as e w w w .ite ch on lin e. co m