A Comparative Analysis of Sentiment Classification Based on Deep and Traditional Ensemble Machine Learning Models

Classical Machine Learning (ML) models, like Random Forests, struggle with weather variability. Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) models, though designed for time-dependent predictions, also face challenges with unseen data and extended rainfall forecasts. To address these, the Fuzzy-Integrated Deep Ensemble for Rainfall Forecasting with advanced Feature Selection (FIDERFFS) is developed, combining LSTM and GRU with Fuzzy Inference Systems to create Fuzzified LSTM and GRU hybrids, integrating them with a Transformer model, and leveraging Fuzzy Rough Feature Selection to improve accuracy, generalisation, and interpretability. Tested on meteorological datasets (Aberystwyth and Bath, UK), the Enhanced Ensemble Model outperforms benchmarks and the Foundational Ensemble Model by 12.26% (MAE), 3.21% (RMSE), and 8.77% (RMSLE), in Aberystwyth’s complex terrain. Results show how topography and maritime proximity influence forecasting and position FIDERFFS as a superior alternative to Classical ML while complementing Numerical Weather Prediction, by reducing computational demands and time lags. ABBREVIATIONS: AI: Artificial Intelligence; AIML: Artificial Intelligence and Machine Learning; ANN: Artificial Neural Network; BiLSTM: Bidirectional Long Short-Term Memory; BM: Baseline Models; ECMWF: European Centre for Medium-Range Weather Forecasts; EDA: Exploratory Data Analysis; FIDERFFS: Fuzzy-Integrated Deep Ensemble for Rainfall Forecasting with advanced Feature Selection; FIS: Fuzzy Inference Systems; FuzzGRU/GRU-FIS: Fuzzified Gated Recurrent Unit; FuzzBiLSTM/BiLSTM-FIS: Fuzzified Bidirectional LSTM; FuzzStLSTM/StLSTM-FIS: Fuzzified Stacked LSTM; FuzzMtLSTM: Fuzzified Multilayer LSTM; FuzzLSTMGRU-Trans: Enhanced Ensemble Model (also referred to as FIDERFFS, used interchangeably); FuzzLSTMGRU: Foundational Ensemble Model; GBM: Gradient Boosting Machines; GPH: Geopotential Height; GRU: Gated Recurrent Unit; HWD: Historical Weather Data; IEEE: Institute of Electrical and Electronics Engineers.; LSTM: Long Short-Term Memory; FuzzLSTM/LSTM-FIS: Fuzzified Long Short-Term Memory; LSTM (Standard): Standard Long Short-Term Memory; MtLSTM: Multilayer Long Short-Term Memory; RNN: Recurrent Neural Network; NWP: Numerical Weather Prediction; SVM: Support Vector Machine; Trans: Transformer Model; WF: Weather Forecasting; XGBoost: Extreme Gradient Boosting

In the last 50 years, with the growth of cities and increase in the number of vehicles and mobility, traffic has become troublesome. As a result, traffic flow prediction started to attract attention as an important research area. However, despite the extensive literature, traffic flow prediction still remains as an open research problem, specifically for long-term traffic flow prediction. Compared to the models developed for short-term traffic flow prediction, the number of models developed for long-term traffic flow prediction is very few. Based on this shortcoming, in this study, we focus on long-term traffic flow prediction and propose a novel deep ensemble model (DEM). In order to build this ensemble model, first, we developed a convolutional neural network (CNN), a long short-term memory (LSTM) network and a gated recurrent unit (GRU) network as deep learning models, which formed the base learners. In the next step, we combine the output of these models according to their individual forecasting success. We use another deep learning model to determine the success of the individual models. Our proposed model is a flexible ensemble prediction model that can be updated based on traffic data. To evaluate the performance of the proposed model, we use a publicly available dataset. Experimental results show that the developed DEM model has a mean square error of 0.06 and a mean absolute error of 0.15 for single-step prediction; it shows that achieves a mean square error of 0.25 and a mean absolute error of 0.32 for multi-step prediction. We compared our proposed model with many models in different categories; individual deep learning models (i.e., LSTM, CNN, GRU), selected traditional machine learning models (i.e., linear regression, decision tree regression, k-nearest-neighbors regression) and other ensemble models such as random-forest regression. These results also support the claim that ensemble learning models perform better than individual models.

The work presented in this paper aims to improve the accuracy of forecasting models in univariate time series, for this it is experimented with different hybrid models of two and four layers based on recurrent neural networks such as Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU). It is experimented with two time series corresponding to downward thermal infrared and all sky insolation incident on a horizontal surface obtained from NASA’s repository. In the first time series, the results achieved by the two-layer hybrid models (LSTM + GRU and GRU + LSTM) outperformed the results achieved by the non-hybrid models (LSTM + LSTM and GRU + GRU); while only two of six four-layer hybrid models (GRU + LSTM + GRU + LSTM and LSTM + LSTM + GRU + GRU) outperformed non-hybrid models (LSTM + LSTM + LSTM + LSTM and GRU + GRU + GRU + GRU). In the second time series, only one model (LSTM + GRU) of two hybrid models outperformed the two non-hybrid models (LSTM + LSTM and GRU + GRU); while the four-layer hybrid models, none could exceed the results of the non-hybrid models.

The task of predicting internet traffic is challenging, particularly in multi-step forecasting due to the volatile and random nature of data. In addition, real-world traffic may contain outlier data points, so developing a prediction model that integrates anomaly detection and mitigation is necessary. This paper compares several deep sequence models, such as Recurrent Neural Network (RNN), Long Short-Term Memory (LSTM), LSTM Encoder-Decoder (LSTM En De), LSTM Encoder-Decoder with attention layer (LSTM En De Atn), and Gated Recurrent Unit (GRU), with our proposed methodology for single-step prediction. Our proposed LSTM En De model, integrated with outlier detection, outperforms traditional deep sequence models in single-step prediction, reducing the deviation between actual and predicted traffic by over 11%. We also apply our methodology to multi-step forecast analysis, using multiple output strategies for forecast horizons of 3, 6, 9, and 12 steps ahead. Experimental results demonstrate the effectiveness of our proposed methodology in improving the accuracy of singlestep prediction and multi-step forecasting tasks, especially when dealing with outlier data points that adversely affect model accuracy. In summary, this paper investigates the challenges of real-world internet traffic prediction, proposes a novel prediction model integrated with anomaly detection and mitigation, and compares different deep sequence models for single-step and multi-step forecasting tasks.

Chronic kidney disease (CKD) involves numerous variables, but only a few significantly impact the classification task. The statistically equivalent signature (SES) method, inspired by constraint-based learning of Bayesian networks, is employed to identify essential features in CKD. Unlike conventional feature selection methods, which typically focus on a single set of features with the highest predictive potential, the SES method can identify multiple predictive feature subsets with similar performance. However, most feature selection (FS) classifiers perform suboptimally with strongly correlated data. The FS approach faces challenges in identifying crucial features and selecting the most effective classifier, particularly in high-dimensional data. This study proposes using the Least Absolute Shrinkage and Selection Operator (LASSO) in conjunction with the SES method for feature selection in CKD identification. Following this, an ensemble deep-learning model combining long short-term memory (LSTM) and gated recurrent unit (GRU) networks is proposed for CKD classification. The features selected by the hybrid feature selection method are fed into the ensemble deep-learning model. The model’s performance is evaluated using accuracy, precision, recall, and F1 score metrics. The experimental results are compared with individual classifiers, including decision tree (DT), Random Forest (RF), logistic regression (LR), and support vector machine (SVM). The findings indicate a 2% improvement in classification accuracy when using the proposed hybrid feature selection method combined with the LSTM and GRU ensemble deep-learning model. Further analysis reveals that certain features, such as HEMO, POT, bacteria, and coronary artery disease, contribute minimally to the classification task. Future research could explore additional feature selection methods, including dynamic feature selection that adapts to evolving datasets and incorporates clinical knowledge to enhance CKD classification accuracy further.

The accurate forecast of wastewater treatment plant (WWTP) key features can comprehend and predict the plant behavior to support process design and controls, improve system reliability, reduce operational costs, and endorse optimization of overall performances. Deep learning technologies as proven data-driven soft-sensors should be developed for WWTP applications to tackle the process of non-linearity and the dynamic nature of environmental data. This study adopts deep learning-based models as soft-sensors to forecast WWTP key features, such as influent flow, influent temperature, influent biochemical oxygen demand (BOD), effluent chloride, effluent BOD, and power consumption. We constructed six deep learning models derived from long short-term memory (LSTM) and gated recurrent unit (GRU), namely traditional LSTM and GRU, the exponentially smoothed LSTM, and the adaptive version of LSTM and smoothed LSTM. The employment of a smoothed LSTM technique is expected to reduce the outlier effect and to improve forecasting accuracy. Meanwhile, the usage of adaptive deep models will enhance the capabilities of the LSTM to quickly and accurately follow the trend of future data. We compared the performance of these models with Bi-directional LSTM (BiLSTM) and the seasonal decomposition using local regression. The historical records from a coastal municipal WWTP in Saudi Arabia are used to verify the investigated models' effectiveness. The proposed models provide promising forecasting results but require no assumptions on the data distributions. In terms of efficiency, GRU based models converge faster than LSTM based models. In terms of accuracy, the LSTM soft-sensor shows overall the optimal result for all key features followed by the exponentially-smoothed GRU and LSTM. By contrast, the adaptive models achieved the lowest forecasting performance compared to the other models. These findings will benefit practitioners to achieve data-driven WWTP management.

A new deep learning ensemble credit risk evaluation model with an improved synthetic minority oversampling technique

The cryptocurrency market has been developed at an unprecedented speed over the past few years. Cryptocurrency works similar to standard currency, however, virtual payments are made for goods and services without the intervention of any central authority. Although cryptocurrency ensures legitimate and unique transactions by utilizing cryptographic methods, this industry is still in its inception and serious concerns have been raised about its use. Analysis of the sentiments about cryptocurrency is highly desirable to provide a holistic view of peoples’ perceptions. In this regard, this study performs both sentiment analysis and emotion detection using the tweets related to the cryptocurrency which are widely used for predicting the market prices of cryptocurrency. For increasing the efficacy of the analysis, a deep learning ensemble model LSTM-GRU is proposed that combines two recurrent neural networks applications including long short term memory (LSTM) and gated recurrent unit (GRU). LSTM and GRU are stacked where the GRU is trained on the features extracted by LSTM. Utilizing term frequency-inverse document frequency, word2vec, and bag of words (BoW) features, several machine learning and deep learning approaches and a proposed ensemble model are investigated. Furthermore, TextBlob and Text2Emotion are studied for emotion analysis with the selected models. Comparatively, a larger number of people feel happy with the use of cryptocurrency, followed by fear and surprise emotions. Results suggest that the performance of machine learning models is comparatively better when BoW features are used. The proposed LSTM-GRU ensemble shows an accuracy of 0.99 for sentiment analysis, and 0.92 for emotion prediction and outperforms both machine learning and state-of-the-art models.

Deep ensemble learning and error correction method for remaining useful life prediction of rolling bearings

IntroductionInfectious diseases present significant challenges to global healthcare systems due to their rapid spread and associated profound health implications. Early detection of unusual increases in case numbers is crucial for achieving efficient resource allocation and effective response planning.MethodTherefore, this research proposes and develops a time series predictive framework based on long short-term memory (LSTM), bidirectional LSTM (BiLSTM), and gated recurrent unit (GRU) neural network models to forecast the number of COVID-19 cases in Saudi Arabia and detect any unusual increase in cases. Google Trends and time series data for search terms, including “fever,” “COVID,” and “cough,” serve as input, enabling models to detect the temporal patterns associated with a surge in cases. The framework is specifically designed to model temporal dependencies in sequential data, allowing the identification of early signs of anomalies in COVID-19 case trends. Therefore, we propose training the models on preprocessed time series data while adjusting for time lags to improve predictive accuracy. Evaluations of performance are conducted using mean square error (MSE) and F1-score metrics.Results and discussionThe experimental results demonstrate that BiLSTM returns the highest F1-score of 0.83 for the term “COVID”, while LSTM and GRU reach 0.73 and 0.77, respectively. Moreover, BiLSTM outperforms LSTM and GRU at all early time lags for the search terms “fever” and “cough”. The results reveal the F1-scores for the term “fever” to be 0.77, 0.62, and 0.5 for BiLSTM, GRU, and LSTM, respectively. Whereas, the F1-scores for the search term “cough” are 0.62, 0.62, and 0.5 for BiLSTM, GRU, and LSTM, respectively. Although BiLSTM incurs higher computational costs, LSTM and GRU offer efficient alternatives to deliver rapid execution. These results highlight the effectiveness of deep learning models in instances of early anomaly detection, supporting timely healthcare interventions and advancing the development of real-time monitoring systems.

Internet traffic in the real world is susceptible to various external and internal factors which may abruptly change the normal traffic flow. Those unexpected changes are considered outliers in traffic. However, deep sequence models have been used to predict complex IP traffic, but their comparative performance for anomalous traffic has not been studied extensively. In this paper, we investigated and evaluated the performance of different deep sequence models for anomalous traffic prediction. Several deep sequences models were implemented to predict real traffic without and with outliers and show the significance of outlier detection in real-world traffic prediction. First, two different outlier detection techniques, such as the Three-Sigma rule and Isolation Forest, were applied to identify the anomaly. Second, we adjusted those abnormal data points using the Backward Filling technique before training the model. Finally, the performance of different models was compared for abnormal and adjusted traffic. LSTM_Encoder_Decoder (LSTM_En_De) is the best prediction model in our experiment, reducing the deviation between actual and predicted traffic by more than 11% after adjusting the outliers. All other models, including Recurrent Neural Network (RNN), Long Short-Term Memory (LSTM), LSTM_En_De with Attention layer (LSTM_En_De_Atn), Gated Recurrent Unit (GRU), show better prediction after replacing the outliers and decreasing prediction error by more than 29%, 24%, 19%, and 10% respectively. Our experimental results indicate that the outliers in the data can significantly impact the quality of the prediction. Thus, outlier detection and mitigation assist the deep sequence model in learning the general trend and making better predictions.

Prediction of wind turbine blade icing fault based on selective deep ensemble model

Comparative analysis of Gated Recurrent Units (GRU), long Short-Term memory (LSTM) cells, autoregressive Integrated moving average (ARIMA), seasonal autoregressive Integrated moving average (SARIMA) for forecasting COVID-19 trends

In this paper, we propose the empirical analysis of Hierarchical Attention Network (HAN) as a feature extractor that works conjointly with eXtreme Gradient Boosting (XGBoost) as the classifier to recognize insufficiently supported arguments using a publicly available dataset. Besides HAN + XGBoost, we performed experiments with several other deep learning models, such as Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), bidirectional LSTM, and bidirectional GRU. All results with the best hyper-parameters are presented. In this paper, we present the following three key findings: (1) Shallow models work significantly better than the deep models when using only a small dataset. (2) Attention mechanism can improve the deep model’s result. In average, it improves Area Under the Receiver Operating Characteristic Curve (ROC-AUC) score of Recurrent Neural Network (RNN) with a margin of 18.94%. The hierarchical attention network gave a higher ROC-AUC score by 2.25% in comparison to the non-hierarchical one. (3) The use of XGBoost as the replacement for the last fully connected layer improved the F1 macro score by 5.26%. Overall our best setting achieves 1.88% improvement compared to the state-of-the-art result.

In winter, the ice and snow on the asphalt pavement reduce the friction coefficient of the pavement, which may lead to serious traffic accidents and large-scale congestion. Taking preventive measures to ensure traffic safety by accurately predicting road surface temperature is an economical and environmentally friendly solution. However, road surface temperature (RST) prediction is a challenging task due to the complicated uncertainty and periodicity. To improve the accuracy of RST prediction, this paper aims to propose an advanced ensemble deep learning model using a gated recurrent unit (GRU) network and long short-term memory (LSTM) network. The ensemble model predicts RST by extracting the periodicity of RST and incorporating the lag and accumulation effects of meteorological factors. To verify the applicability of the ensemble model, RST data and climatic data were collected from a road weather station in Jiangsu, China. Extensive experiments are conducted including predictions for 1, 3, and 6 h ahead. The results demonstrated that the performance of the proposed ensemble deep learning model is validated for 1-, 3-, and 6-h nowcasts of RST, with mean absolute error (MAE) of 0.345, 0.833, and 1.743, respectively, and the prediction accuracy was higher than that of the baseline models [convolutional neural networks (CNN)-LSTM networks, support vector regression (SVR), and backpropagation neural network (BP) networks].