A novel hybrid deep fuzzy model based on gradient descent algorithm with application to time series forecasting

Abstract

Deep fuzzy systems are widely used in time series forecasting tasks due to their excellent nonlinear transformation capabilities and interpretability. However, traditional optimization methods and fixed network structures cannot make the deep fuzzy system obtain the expected prediction accuracy and generalization ability. Therefore, a novel hybrid deep fuzzy model (HDFM) is proposed in this paper. Firstly, two types fuzzy modules, namely the type-1 TSK fuzzy module (T1TSKFM) and the interval type-2 TSK fuzzy module (IT2TSKFM), are respectively presented and designed in detail. And, the gradient expressions of the fuzzy parameters, including the antecedent and consequent parameters, are also derived in detail. Secondly, in order to optimize the fuzzy parameters and to further accelerate the module convergence, a novel parameter optimization strategy is presented, combining the gradient descent with the Regularization, the DropRule and the AdaBound algorithms. Thirdly, a new stacked hybrid deep fuzzy architecture is proposed, which can be automatically trained and constructed using the designed T1TSKFM and the IT2TSKFM. Then, the detailed data-driven learning and updating strategy are given in step by step way. In addition, both the layered structure interpretability and the fuzzy rule interpretability are respectively analyzed. This can guarantee that the proposed model not only has the better forecasting accuracy, but also has the higher interpretability. Finally, in order to verify the effectiveness of the proposed method, several comparative experiments are given. Experimental results show that the forecasting performance of the proposed model outperforms the other comparisons, such as the DIRM-DFM, the IT2DIRM-DFM, the DCFS, and the ANFIS method. At the same time, the proposed model has better interpretability and more flexible construction property.