Single-layer folded RNN for time series prediction and classification under a non-Von Neumann architecture

Abstract

A delay dynamical system can fold a feedforward neural network into one nonlinear neuron and multiple delay loops under the non-Von Neumann structure, greatly decreasing the hardware requirements. In this paper, we transform the folded-in-time DNN (Fit-DNN) into a folded-in-time RNN (Fit-RNN) and derive the backpropagation algorithm for it. The performance of the folded reservoir computing (DRC), Fit-DNN, and Fit-RNN is compared on time-series prediction and classification tasks, respectively. The impact of virtual node separation on the performance of Fit-RNN is analyzed. The limits of Fit-RNN's capabilities in conducting effective time-series predictions were determined based on the NARMA-X series tasks. We calculate the 5-order information processing capacity (IPC) of DRC and Fit-RNN on NARMA10. The results indicate that Fit-RNN has almost the necessary information transformation ability for the task. In the recognition tasks conducted separately on spoken digits and speakers, we investigate the impact of the number and separation of virtual nodes on the recognition capability of the three folded networks. The results demonstrate that Fit-RNN shows more promise in handling long sequences and large-scale recognition tasks. The recognition accuracy is further enhanced by increasing the number of virtual nodes and node separation. Furthermore, the introduction of two common types of noise in the speaker recognition task environment further highlights the potential of Fit-RNN in practical applications. Furthermore, two common types of noise are introduced into the speaker recognition task, further highlighting the potential of Fit-RNN in practical applications.