Machine Learning Systems Tuned by Bayesian Optimization to Forecast Electricity Demand and Production

The heart is an important organ that plays a crucial role in maintaining life. Unfortunately, heart disease is one of the major causes of mortality globally. Early and accurate detection can significantly improve the situation by enabling preventive measures and personalized healthcare recommendations. Artificial intelligence is emerging as a powerful tool for healthcare applications, particularly in predicting heart diseases. Researchers are actively working on this, but challenges remain in achieving accurate heart disease prediction. Therefore, experimenting with various models to identify the most effective one for heart disease prediction is crucial. In this view, this paper addresses this need by conducting an extensive investigation of various models. The proposed research considered 11 feature selection techniques and 21 classifiers for the experiment. The feature selection techniques considered for the research are Information Gain, Chi-Square Test, Fisher Discriminant Analysis (FDA), Variance Threshold, Mean Absolute Difference (MAD), Dispersion Ratio, Relief, LASSO, Random Forest Importance, Linear Discriminant Analysis (LDA), and Principal Component Analysis (PCA). The classifiers considered for the research are Logistic Regression, Decision Tree, Random Forest, K-Nearest Neighbors (KNN), Support Vector Machine (SVM), Gaussian Naïve Bayes (GNB), XGBoost, AdaBoost, Stochastic Gradient Descent (SGD), Gradient Boosting Classifier, Extra Tree Classifier, CatBoost, LightGBM, Multilayer Perceptron (MLP), Recurrent Neural Network (RNN), Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), Bidirectional LSTM (BiLSTM), Bidirectional GRU (BiGRU), Convolutional Neural Network (CNN), and Hybrid Model (CNN, RNN, LSTM, GRU, BiLSTM, BiGRU). Among all the extensive experiments, XGBoost outperformed all others, achieving an accuracy of 0.97, precision of 0.97, sensitivity of 0.98, specificity of 0.98, F1 score of 0.98, and AUC of 0.98.

In today's digital world, app stores have become an essential part of software distribution, providing customers with a wide range of applications and opportunities for software developers to showcase their work. This study elaborates on the importance of end-user feedback for software evolution. However, in the literature, more emphasis has been given to high-rating & popular software apps while ignoring comparatively low-rating apps. Therefore, the proposed approach focuses on end-user reviews collected from 64 low-rated apps representing 14 categories in the Amazon App Store. We critically analyze feedback from low-rating apps and developed a grounded theory to identify various concepts important for software evolution and improving its quality including user interface (UI) and user experience (UX), functionality and features, compatibility and device-specific, performance and stability, customer support and responsiveness and security and privacy issues. Then, using a grounded theory and content analysis approach, a novel research dataset is curated to evaluate the performance of baseline machine learning (ML), and state-of-the-art deep learning (DL) algorithms in automatically classifying end-user feedback into frequently occurring issues. Various natural language processing and feature engineering techniques are utilized for improving and optimizing the performance of ML and DL classifiers. Also, an experimental study comparing various ML and DL algorithms, including multinomial naive Bayes (MNB), logistic regression (LR), random forest (RF), multi-layer perception (MLP), k-nearest neighbors (KNN), AdaBoost, Voting, convolutional neural network (CNN), long short-term memory (LSTM), bidirectional long short term memory (BiLSTM), gated recurrent unit (GRU), bidirectional gated recurrent unit (BiGRU), and recurrent neural network (RNN) classifiers, achieved satisfactory results in classifying end-user feedback to commonly occurring issues. Whereas, MLP, RF, BiGRU, GRU, CNN, LSTM, and Classifiers achieved average accuracies of 94%, 94%, 92%, 91%, 90%, 89%, and 89%, respectively. We employed the SHAP approach to identify the critical features associated with each issue type to enhance the explainability of the classifiers. This research sheds light on areas needing improvement in low-rated apps and opens up new avenues for developers to improve software quality based on user feedback.

Sentiment Analysis of Self Driving Car Dataset: A comparative study of Deep Learning approaches

The state of Amapá within the Amazon biome has a high complexity of ecosystems formed by forests, savannas, seasonally flooded vegetation, mangroves, and different land uses. The present research aimed to map the vegetation from the phenological behavior of the Sentinel-1 time series, which has the advantage of not having atmospheric interference and cloud cover. Furthermore, the study compared three different sets of images (vertical–vertical co-polarization (VV) only, vertical–horizontal cross-polarization (VH) only, and both VV and VH) and different classifiers based on deep learning (long short-term memory (LSTM), Bidirectional LSTM (Bi-LSTM), Gated Recurrent Units (GRU), Bidirectional GRU (Bi-GRU)) and machine learning (Random Forest, Extreme Gradient Boosting (XGBoost), k-Nearest Neighbors, Support Vector Machines (SVMs), and Multilayer Perceptron). The time series englobed four years (2017–2020) with a 12-day revisit, totaling 122 images for each VV and VH polarization. The methodology presented the following steps: image pre-processing, temporal filtering using the Savitsky–Golay smoothing method, collection of samples considering 17 classes, classification using different methods and polarization datasets, and accuracy analysis. The combinations of the VV and VH pooled dataset with the Bidirectional Recurrent Neuron Networks methods led to the greatest F1 scores, Bi-GRU (93.53) and Bi-LSTM (93.29), followed by the other deep learning methods, GRU (93.30) and LSTM (93.15). Among machine learning, the two methods with the highest F1-score values were SVM (92.18) and XGBoost (91.98). Therefore, phenological variations based on long Synthetic Aperture Radar (SAR) time series allow the detailed representation of land cover/land use and water dynamics.

Emotion recognition (ER) is crucial for understanding human behaviours, social interactions, and psychological well-being. Electroencephalography (EEG) has emerged as a promising tool for capturing the neural correlates of emotions. This work is a systematic review of articles in ER using EEG signals. A total of 120 articles from 1041 articles were selected based on PRISMA guidelines using defined inclusion and exclusion criteria, published between 2018 and 2024. This article aims to provide an in-depth understanding of the current landscape of ER from EEG signals utilizing deep learning (DL). This review offers valuable guidance for researchers and practitioners seeking more refined and reliable emotion classification systems. To explore the effectiveness of DL models in EEG-based ER, several potential DL models, such as convolutional neural network, long short-term memory (LSTM), gated recurrent unit (GRU), hybrid bidirectional LSTM (BiLSTM), bidirectional GRU, and advanced DL models such as convolutional recurrent neural network and EEG-Conformer models are applied to two popular datasets, SEED and GAMEEMO, respectively, to depict the full process of ER. Additionally, the performance of DL models is also compared with the performance of basic machine learning (ML) models such as SVM, k-nearest neighbors, logistic regression, and boosting algorithms such as AdaBoost, XGBoost and LightGBM. Through extensive experiments and performance evaluations, the performance of different models when applied to the datasets mentioned above is compared. The accuracy, precision, recall, and F1-scores are analysed to determine the most effective model for EEG-based ER. The findings of this study demonstrate that the performance of hybrid DL models is more efficacious than that of ML models. The best-performing model (BiLSTM) classified the emotions, with an accuracy of 90.54% when applied to the GAMEEMO dataset. This research contributes to the growing body of literature on ER and provides insights into the feasibility of using EEG signals to understand emotional states, and presents a structured roadmap for future exploration. The findings can aid in the development of more accurate and reliable ER systems, which can have wide-ranging applications in psychology, social sciences, and human-computer interactions.

Sentiment analysis is a Natural Language Processing (NLP) task concerned with opinions, attitudes, emotions, and feelings. It applies NLP techniques for identifying and detecting personal information from opinionated text. Sentiment analysis deduces the author's perspective regarding a topic and classifies the attitude polarity as positive, negative, or neutral. In the meantime, deep architectures applied to NLP reported a noticeable breakthrough in performance compared to traditional approaches. The outstanding performance of deep architectures is related to their capability to disclose, differentiate and discriminate features captured from large datasets. Recurrent neural networks (RNNs) and their variants Long-Short Term Memory (LSTM), Gated Recurrent Unit (GRU), Bi-directional Long-Short Term Memory (Bi-LSTM), and Bi-directional Gated Recurrent Unit (Bi-GRU) architectures are robust at processing sequential data. They are commonly used for NLP applications as they—unlike RNNs—can combat vanishing and exploding gradients. Also, Convolution Neural Networks (CNNs) were efficiently applied for implicitly detecting features in NLP tasks. In the proposed work, different deep learning architectures composed of LSTM, GRU, Bi-LSTM, and Bi-GRU are used and compared for Arabic sentiment analysis performance improvement. The models are implemented and tested based on the character representation of opinion entries. Moreover, deep hybrid models that combine multiple layers of CNN with LSTM, GRU, Bi-LSTM, and Bi-GRU are also tested. Two datasets are used for the models implementation; the first is a hybrid combined dataset, and the second is the Book Review Arabic Dataset (BRAD). The proposed application proves that character representation can capture morphological and semantic features, and hence it can be employed for text representation in different Arabic language understanding and processing tasks.

Accurate and timely forecasting is critical for grid-connected solar power safety and stability, achieved through machine learning (ML) for both common and real-time applications. To mitigate the impact of nonstationarity and volatility in solar power generation, we employed empirical mode decomposition (EMD), ensemble empirical mode decomposition (EEMD), variational mode decomposition (VMD), and complete EEMD with adaptive noise (CEEMDAN) to decompose time series into frequency components, reducing fluctuations and noise. A combination of four decomposition methods (EMD, EEMD, VMD, and CEEMDAN) and two ML models, bidirectional gated recurrent unit (BiGRU) and bidirectional long short-term memory (BiLSTM) were utilized to construct six hybrid forecasting models (EMD-BiLSTM, EMD-BiGRU, EEMD-BiLSTM, EEMD-BiGRU, VMD-BiGRU, and CEEMDAN-BiGRU), which were validated on a dataset from a 20 MW solar power station in Hebei, compared to the seven standalone ML models, backpropagation neural networks (BPNN), support vector machines (SVM), long short-term memory (LSTM), gated recurrent unit (GRU), convolutional neural networks (CNN), BiLSTM, and BiGRU, these six hybrid models demonstrated enhanced forecast accuracy. Of these, EEMD-BiGRU, VMD-BiGRU, and CEEMDAN-BiGRU significantly reduced prediction errors, with percentage reductions in root mean square error (RMSE), mean absolute error (MAE), and mean absolute percentage error (MAPE) ranging from 44.18 ~ 49.43%, 43.67 ~ 48.59%, and 44.64% ~53.53%, respectively. The EEMD-BiGRU model outperformed all hybrid models, achieving an RMSE of 0.7662, MAE of 0.3990, MAPE of 7.982%, and R2 of 0.9865. The findings of this study can provide insights and support for applying a hybrid model based on decomposition methods for the short-term forecasting of solar power generation.

Abstract. This study aims to enhance lung cancer patient screening by developing and evaluating bidirectional Long Short-Term Memory (LSTM) and bidirectional Gated Recurrent Unit (GRU) models using the Lung Cancer dataset from Kaggle. The dataset includes 16 features related to patient symptoms and lung cancer status, providing a broad spectrum of symptoms to improve model accuracy. The research advances Artificial Intelligence (AI)-driven healthcare by integrating these sophisticated machine learning techniques into diagnostic processes. The methodology involves four main steps: preprocessing the dataset for model compatibility, defining the model architecture with bidirectional LSTM and GRU layers and evaluating its performance. The results show an overall accuracy of 52.17%, with accuracy, recall, and F1 scores for both cancerous and non-cancerous categories around 50%. Despite the hybrid model's average performance, it establishes a basis for future enhancements. Optimizing model parameters and exploring additional other techniques to improve prediction accuracy and clinical applicability will be done in the future.

In recent years, deep learning (DL) techniques have been applied in the structural and functional analysis of proteins in bioinformatics, especially in 8-state (Q8) protein secondary structure prediction (PSSP). In this paper, we have explored the performance of various DL architectures for Q8 PSSP, by developing six DL architectures, using convolutional neural networks (CNNs), recurrent neural networks (RNNs), and combinations of them. These architectures are: CNN-SW (CNNs with sliding window); CNN-WP (CNNs with whole protein as input); LSTM+ (Long Short-Term Memory (LSTM) & Bidirectional LSTM (BLSTM)); GRU+ (Gated Recurrent Unit (GRU) & bidirectional GRU (BGRU)); CNN-BGRU (CNNs & BGRUs); and CNN-BLSTM (CNNs & BLSTMs). They include batch normalization, dropout, and fully-connected layers. We have used CB6133 and CB513 datasets for training and testing, respectively. The experiments showed that combining CNN with BLSTM or BGRU overcame overfitting, and achieved better Q8 accuracy, precision, recall and F-score. The experiments on CB513 showed that CNN-SW, CNN-BGRU, and CNN-BLSTM achieved Q8 accuracy comparable with some state-of-the-art models.

Predicting rainfall is a challenging and critical task due to its significant impact on society. Timely and accurate predictions are essential for minimizing human and financial losses. The dependence of approximately 60% of agricultural land in India on monsoon rainfall implies the crucial nature of accurate rainfall prediction. Precise rainfall forecasts can facilitate early preparedness for disasters associated with heavy rains, enabling the public and government to take necessary precautions. In the North-Western Himalayas, where meteorological data are limited, the need for improved accuracy in traditional modeling methods for rainfall forecasting is pressing. To address this, our study proposes the application of advanced machine learning (ML) algorithms, including random forest (RF), support vector regression (SVR), artificial neural network (ANN), and k-nearest neighbour (KNN) along with various deep learning (DL) algorithms such as long short-term memory (LSTM), bi-directional LSTM, deep LSTM, gated recurrent unit (GRU), and simple recurrent neural network (RNN). These advanced techniques hold the potential to significantly improve the accuracy of rainfall prediction, offering hope for more reliable forecasts. Additionally, time series techniques, including autoregressive integrated moving average (ARIMA) and trigonometric, Box-Cox transform, arma errors, trend, and seasonal components (TBATS), are proposed for predicting rainfall across the altitudinal gradients of India’s North-Western Himalayas. This approach can potentially revolutionise how we approach rainfall forecasting, ushering in a new era of accuracy and reliability. The effectiveness and accuracy of the proposed algorithms were assessed using meteorological data obtained from six weather stations at different elevations spanning from 1980 to 2021. The results indicate that DL methods exhibit the highest accuracy in predicting rainfall, as measured by the root mean squared error (RMSE) and mean absolute error (MAE), followed by ML algorithms and time series techniques. Among the DL algorithms, the accuracy order was bi-directional LSTM, LSTM, RNN, deep LSTM, and GRU. For the ML algorithms, the accuracy order was ANN, KNN, SVR, and RF. These findings suggest that altitude significantly affects the accuracy of the models, highlighting the need for additional weather stations in this mountainous region to enhance the precision of rainfall prediction.

Traffic flow prediction plays an important role in intelligent transportation systems. To accurately capture the complex non-linear temporal characteristics of traffic flow, this paper adopts a Bi-directional Gated Recurrent Unit (Bi-GRU) model in traffic flow prediction. Compared to Gated Recurrent Unit (GRU), which can memorize information from the previous sequence, this model can memorize the traffic flow information in both previous and subsequent sequence. To demonstrate the model’s performance, a set of real case data at 1-hour intervals from 5 working days was used, wherein the dataset was separated into training and validation. To improve data quality, an augmented dickey-fuller unit root test and differential processing were performed before model training. Four benchmark models were used, including the Autoregressive Integrated Moving Average (ARIMA), Long Short-Term Memory (LSTM), Bidirectional Long Short-Term Memory (Bi-LSTM), and GRU. The prediction results show the superior performance of Bi-GRU. The Root Mean Square Error (RMSE), Mean Absolute Percentage Error (MAPE), and Mean Absolute Error (MAE) of the Bi-GRU model are 30.38, 9.88%, and 23.35, respectively. The prediction accuracy of LSTM, Bi-LSTM, GRU, and Bi-GRU, which belong to deep learning methods, is significantly higher than that of the traditional ARIMA model. The MAPE difference of Bi-GRU and GRU is 0.48% which is a small prediction error value. The results show that the prediction accuracy of the peak period is higher than that of the low peak. The Bi-GRU model has a certain lag on traffic flow prediction.

Stratification of Depressed and Non-Depressed Texts from Social Media using LSTM and its Variants

In this paper, we have implemented Stacked Recurrent Neural Network for a Robust Speech Recognition System inside a car. To cancel out the high traffic noise, Long Short Term Memory (LSTM) and Gated Recurrent Unit (GRU), Bidirectional LSTM (BLSTM) and Bidirectional GRU (BGRU) have been used for mapping the noisy data with the clean speech signal. Later to increase the complexity of the model, Stacked LSTM, GRU, BLSTM and BGRU were implemented. All the models were applied to the data in Cepstral Domain and analyzed with respect to different Number of Layers and different Signal to Noise Ratio (SNR).

This paper presents supervised deep learning techniques for weekly electrical load demand forecasting at the Suranaree University of Technology, Nakhon Ratchasima, Thailand. Deep learning is a modern artificial intelligent technique developed from neural networks-based machine learning that mimics the nervous system of living organisms. Six supervised deep learning techniques, i.e., Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), Long Short-Term Memory (LSTM), Bidirectional Long Short-Term Memory (BiLSTM), Gated Recurrent Unit (GRU), and Bidirectional Gated Recurrent Unit (BiGRU) techniques, were used in this study. The supervised deep learning model was trained by daily load demand data from January 1, 2018, to May 18, 2022, before forecasting weekly load demand. The forecasting results show that LSTM, BiLSTM, and GRU are all techniques with minor errors. The BiLSTM technique, on the other hand, produced the most accurate forecasts, with an MAE of 13.07 kW, MAPE of 1.66%, and RMSE of 19.826 kW, respectively.

<abstract> <p>Energy operations and schedules are significantly impacted by load and energy forecasting systems. An effective system is a requirement for a sustainable and equitable environment. Additionally, a trustworthy forecasting management system enhances the resilience of power systems by cutting power and load-forecast flaws. However, due to the numerous inherent nonlinear properties of huge and diverse data, the classical statistical methodology cannot appropriately learn this non-linearity in data. Energy systems can appropriately evaluate data and regulate energy consumption because of advanced techniques. In comparison to machine learning, deep learning techniques have lately been used to predict energy consumption as well as to learn long-term dependencies. In this work, a fusion of novel multi-directional gated recurrent unit (MD-GRU) with convolutional neural network (CNN) using global average pooling (GAP) as hybridization is being proposed for load and energy forecasting. The spatial and temporal aspects, along with the high dimensionality of the data, are addressed by employing the capabilities of MD-GRU and CNN integration. The obtained results are compared to baseline algorithms including CNN, Long Short-Term Memory (LSTM), Bidirectional Long Short-Term Memory (Bi-LSTM), Gated Recurrent Unit (GRU), and Bidirectional Gated Recurrent Unit (Bi-GRU). The experimental findings indicate that the proposed approach surpasses conventional approaches in terms of accuracy, Mean Absolute Percentage Error (MAPE), and Root Mean Square Error (RSME).</p> </abstract>