Combining predictive base models using deep ensemble learning

Abstract

Since information science and communication technologies had improved significantly, data volumes had expanded. As a result of that situation, advanced pre-processing and analysis of collected data became a crucial topic for extracting meaningful patterns hidden in the data. Therefore, traditional machine learning algorithms generally fail to gather satisfactory results when analyzing complex data. The main reason of this situation is the difficulty of capturing multiple characteristics of the high dimensional data. Within this scope, ensemble learning enables the integration of diversified single models to produce weak predictive results. The final combination is generally achieved by various voting schemes. On the other hand, if a large amount of single models are utilized, voting mechanism cannot be able to combine these results. At this point, Deep Learning (DL) provides the combination of the ensemble results in a considerable time. Apart from previous studies, we determine various predictive models in order to forecast the outcome of two different case studies. Consequently, data cleaning and feature selection are conducted in advance and three predictive models are defined to be combined. DL based integration is applied substituted for voting mechanism. The weak predictive results are fused based on Recurrent Neural Network (RNN) and Long Short Term Memory (LSTM) using different parameters and datasets and best predictors are extracted. After that, different experimental combinations are evaluated for gathering better prediction results. For comparison, grouped individual results (clusters) with proper parameters are compared with DL based ensemble results.