Deep and Wide Neural Networks on Multiple sets of Temporal data with Correlation

Abstract

Both Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) have played important roles in deep learning in recent years. The combination of them is an important direction to improve the performance of the prediction model. In the process of studying temporal data, it is found that temporal data are closely related to each other rather than isolated. Therefore it is necessary to take into account the correlation between different temporal data while processing the temporal data along the time steps. However general RNN model cannot achieve the requirement. In this paper, RNN and CNN are combined to extract the correlation characteristics of different RNNs by using CNNs while the RNNs run along the time steps. Thus, we propose the Deep and Wide Neual Networks (DWNNs) which can not only process temporal data in depth direction (time step), but also extract the correlation feature in breadth direction (multiple sets of data). We use temporal data from the stock market for our experiment, which has an interesting correlation structure. The empirical results show that the performance of DWNN is 30% higher than that of ordinary RNN.