Long-Term Peak Load Forecasting with CNN Via Two-Stage Random–Bayesian Hyperparameter Optimization

The evolution of new technologies has made short-term power load forecasting an essential part of the streamlining process in the management of power grid systems. Machine learning algorithms have been applied widely in this area but with little success towards achieving better accuracy rates. These gaps point out the necessity for better forecasting methods . This study is about the power grid system from Ho Chi Minh city in Vietnam. Ho Chi Minh operates as a metropolitan area on the rise with economic activity and seasonal factors greatly influencing electricity consumption. Due to its intricate fluctuations in consumption pattern, the city is known for having a high level of energy. This makes the city suitable for an in-depth investigation regarding a case study on short-term load forecasting approaches. In this study, the goal is to evaluate the effectiveness of three hyperparameter optimization methods: Random Search, Grid Search, and Bayes Search. All these methods optimize the performance of Convolutional Neural Network (CNN) models for short-term electricity load forecasting in Ho Chi Minh City. The results obtained through this work can also be used as a basis for introducing the methods to other locations in Vietnam. The assessment of the techniques is performed using fundamental error measures such as Mean Absolute Error (MAE) and Mean Absolute Percentage Error (MAPE). Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE), Mean Squared Error (MSE), and Root Mean Squared Error (RMSE). Bayes Search completed with an MAE of 77.93, MAPE of 2.94%, MSE of 10,376.7, and RMSE of 101.9. These results indicate a noticeable enhancement in prediction accuracy when compared with the outcomes from Grid Search and Random Search. Grid Search provided an MAE of 106.23, MAPE of 3.95%, MSE of 17,033.7, and RMSE of 130.5. Random Search produces results of an MAE of 96.8, MAPE of 3.57%, MSE of 14,951.0, RMSE of 122.3. These results are evidence that Bayes Search is better for short-term electricity load forecasting in Ho Chi Minh City. The study also proposes an evaluation framework, which is meant for load forecasting in Vietnam. It is designed for Ho Chi Minh City predicting purposes, thus, integrating innovative concepts with actual forecasting functions. The framework is also applicable to other areas in Vietnam, both rural and urban, having different power consumption patterns. The reduction in forecasting inaccuracies through the use of Bayes Search is found to be promising as observed in the research. This automation supports better decision-making in energy management. It helps reduce costs in dynamic and complex power grid environments. These findings have practical value. They support efforts to build more flexible and efficient energy grids in Vietnam.

An efficient management and better scheduling by the power companies are of great significance for accurate electrical load forecasting. There exists a high level of uncertainties in the load time series, which is challenging to make the accurate short-term load forecast (STLF), medium-term load forecast (MTLF), and long-term load forecast (LTLF). To extract the local trends and to capture the same patterns of short, and medium forecasting time series, we proposed long short-term memory (LSTM), Multilayer perceptron, and convolutional neural network (CNN) to learn the relationship in the time series. These models are proposed to improve the forecasting accuracy. The models were tested based on the real-world case by conducting detailed experiments to validate their stability and practicality. The performance was measured in terms of squared error, Root Mean Square Error (RMSE), Mean Absolute Percentage Error (MAPE), and Mean Absolute Error (MAE). To predict the next 24 hours ahead load forecasting, the lowest prediction error was obtained using LSTM with R2 (0.5160), MLP with MAPE (4.97), MAE (104.33) and RMSE (133.92). To predict the next 72 hours ahead of load forecasting, the lowest prediction error was obtained using LSTM with R2 (0.7153), MPL with MAPE (7.04), MAE (125.92), RMSE (188.33). Likewise, to predict the next one week ahead load forecasting, the lowest error was obtained using CNN with R2 (0.7616), MLP with MAPE (6.162), MAE (103.156), RMSE (150.81). Moreover, to predict the next one-month load forecasting, the lowest prediction error was obtained using CNN with R2 (0.820), MLP with MAPE (5.18), LSTM with MAE (75.12) and RMSE (109.197). The results reveal that proposed methods achieved better and stable performance for predicting the short, and medium-term load forecasting. The findings of the STLF indicate that the proposed model can be better implemented for local system planning and dispatch, while it will be more efficient for MTLF in better scheduling and maintenance operations.

Machine learning methods are being applied to a wide range of problems in biology and bioinformatics. These methods often rely on configuring high level parameters, or hyperparameters, such as regularization hyperparameters in sparse learning models like graph-guided multitask Lasso methods. Different choices for these hyperparameters will lead to different results, which makes finding good hyperparameter combinations an important task when using these hyperparameter dependent methods. There are several different ways to tune hyperparameters including manual tuning, grid search, random search, and Bayesian optimization. In this paper, we apply three hyperparameter tuning strategies to eQTL analysis including grid and random search in addition to Bayesian optimization. Experiments show that the Bayesian optimization strategy outperforms the other strategies in modeling eQTL associations. Applying this strategy to assess eQTL associations using the 1000 Genomes structural variation genotypes and RNAseq data in gEUVADIS, we identify a set of new SVs associated with gene expression changes in a human population.

Valve internal leakage in thermal power stations exhibits a strong concealed nature. If it cannot be discovered and predicted of development trend in time, it will affect the safe and economical operation of plant equipment. This paper proposed an intelligent identification method for valve internal leakage that integrated an Improved Kepler Optimization Algorithm (IKOA) with Support Vector Regression (SVR). The Kepler Optimization Algorithm (KOA) was improved using the Sobol sequence and an adaptive Gaussian mutation strategy to achieve self-optimization of the key parameters in the SVR model. A multi-step sliding cross-validation method was employed to train the model, ultimately yielding the IKOA-SVR intelligent identification model for valve internal leakage quantification. Taking the main steam drain pipe valve as an example, a simulation case validation was carried out. The calculation example used Mean Squared Error (MSE), Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE) and determination coefficient (R2) as performance evaluation metrics, and compared and analyzed the training and testing dataset using IKOA-SVR, KOA-SVR, Particle Swarm Optimization (PSO)-SVR, Random Search (RS)-SVR, Grid Search (GS)-SVR, and Bayesian Optimization (BO)-SVR methods, respectively. For the testing dataset, the MSE of IKOA-SVR is 0.65, RMSE is 0.81, MAE is 0.49, and MAPE is 0.0043, with the smallest values among the six methods. The R2 of IKOA-SVR is 0.9998, with the largest value among the six methods. It indicated that IKOA-SVR can effectively solve problems such as getting stuck in local optima and overfitting during the optimization process. An Out-Of-Distribution (OOD) test was conducted for two scenarios: noise injection and Region-Holdout. The identification performance of all six methods decreased, with IKOA-SVR showing the smallest performance decline. The results show that IKOA-SVR has the strongest generalization ability and robustness, the best effect in improving fitting ability, the smallest identification error, the highest identification accuracy, and results closer to the actual value. The method presented in this paper provides an effective approach to solve the problem of intelligent identification of valve internal leakage in thermal power station.

Identification of surface subsidence risk in deep foundation pits using a Mamba fusion model

This study systematically evaluates the effect of hyperparameter optimization on the performance of predictive models by comparing three main techniques: Grid Search, Random Search, and Bayesian Optimization. Four commonly used machine learning algorithms: Logistic Regression (LR), Support Vector Machine (SVM), K-Nearest Neighbors (KNN), and Gradient Boosting were tested on benchmark datasets from the UC Machine Learning Repository. The results of the study show that hyperparameter optimization significantly improves prediction accuracy compared to baseline models, with the optimal method varying across algorithms. Specifically, Random Search achieved the highest accuracy of 0.883333 for Logistic Regression and 0.833333 for Gradient Boosting, while Bayesian Optimization demonstrated superior performance on SVM with an accuracy of 0.883333. Grid Search proved most effective for KNN, achieving an accuracy of 0.866667. A comprehensive analysis using additional metrics such as precision and recall reinforces these findings, showing that accuracy improvements do not come at the expense of other performance metrics. Bayesian Optimization stands out for computational efficiency, especially for complex models, though Grid Search remains relevant for limited hyperparameter spaces. The main contribution of this research is to provide practical guidance for machine learning practitioners on selecting optimization techniques that align with algorithm characteristics. This study also analyses the trade-off between search thoroughness and computational resources required, and provides recommendations for balancing accuracy and efficiency in model development. Although the research results are promising, several limitations need to be acknowledged, including reliance on a single dataset and the need for further validation across multiple domains. Future research could explore hybrid optimization approaches that combine the strengths of various methods, develop adaptive strategies for ensemble models, or conduct more comprehensive evaluations across diverse datasets. By linking theoretical findings to practical applications, this research provides a valuable framework for efficiently optimizing machine learning models.

Software Effort estimation (SEE) is a vital task for project management as it is essential for resource allocation and project planning. Numerous algorithms have been investigated for forecasting software effort, yet achieving precise predictions remains a significant hurdle in the software industry. To achieve optimal accuracy, machine learning algorithms are employed. Remarkably, Random Forest (RF) algorithm produced better accuracy when compared with various algorithms. In this paper, the prediction is extended by increasing the number of trees and Improved Random Forest (IRF) is implemented by including three decision techniques such as residual analysis, partial dependence plots and feature engineering to improve prediction accuracy. To make improved random forest to be adaptive, it is further extended in this paper by integrating three techniques such as: Bayesian Optimization with Deep Kernel Learning (BO-DKL) to adaptively set hyperparameters, Time-Series Residual Analysis to detect autocorrelation patterns among model error, and Explainable AI techniques Shapley Additive Explanations (SHAP) and Local Interpretable Model-Agnostic Explanations (LIME) to improve feature interpretability. This Improved Adaptive Random Forest (IARF) mutually contributes to a comprehensive evaluation and improvement of accuracy in prediction. Metrics used for evaluation are Mean Absolute Error (MAE), Root Mean Square Error (RMSE), R-Squared, Mean Absolute Percentage Error (MAPE), Mean Absolute Scaled Error (MASE) and Prediction Interval Coverage Probability (PICP). Overall, the improved adaptive RF model had an average improvement ratio of 18.5% on MAE, 20.3% on RMSE, 3.8% on R2, 5.4% on MAPE, 7% reduction in MASE and a 3-5% improvement in PICP across all data sets compared to the Random Forest model, with much improved prediction accuracy. These findings validate that the combination of adaptive learning methods and explainability-based adjustments considerably improves accuracy of software effort estimation models and facilitates more trustworthy decision-making in software development projects.

Machine learning (ML) algorithms are techniques that allow computers to learn from the data without being explicitly programmed. ML techniques consist of hyperparameters that typically influence prediction accuracy, hence requiring tuning. In this study, we systematically evaluate the performance of the genetic algorithm (GA) technique in tuning ML hyperparameters compared to three other common tuning techniques i.e. grid search (GS), random search (RS), and bayesian optimization (BO). While previous studies explored the potential of metaheuristics techniques such as GA in tuning ML models, a systematic comparison with other commonly mentioned techniques is currently lacking. Results indicate that GA slightly outperformed other methods in terms of optimality due to its ability to pick any continuous value within the range. However, apart from GS which took the longest, it was observed that GA is quite a time inefficient compared to RS and BO which were able to find a solution close to the GA within a shorter time (GA – 149 minutes, RS – 88 minutes, BO – 105 minutes, GS – 756 minutes).

Brain tumor is one of the most aggressive diseases nowadays, resulting in a very short life span if it is diagnosed at an advanced stage. The treatment planning phase is thus essential for enhancing the quality of life for patients. The use of Magnetic Resonance Imaging (MRI) in the diagnosis of brain tumors is extremely widespread, but the manual interpretation of large amounts of images requires considerable effort and is prone to human errors. Hence, an automated method is necessary to identify the most common brain tumors. Convolutional Neural Network (CNN) architectures are successful in image classification due to their high layer count, which enables them to conceive the features effectively on their own. The tuning of CNN hyperparameters is critical in every dataset since it has a significant impact on the efficiency of the training model. Given the high dimensionality and complexity of the data, manual hyperparameter tuning would take an inordinate amount of time, with the possibility of failing to identify the optimal hyperparameters. In this paper, we proposed a Bayesian Optimization-based efficient hyperparameter optimization technique for CNN. This method was evaluated by classifying 3064 T-1-weighted CE-MRI images into three types of brain tumors (Glioma, Meningioma, and Pituitary). Based on Transfer Learning, the performance of five well-recognized deep pre-trained models is compared with that of the optimized CNN. After using Bayesian Optimization, our CNN was able to attain 98.70% validation accuracy at best without data augmentation or cropping lesion techniques, while VGG16, VGG19, ResNet50, InceptionV3, and DenseNet201 achieved 97.08%, 96.43%, 89.29%, 92.86%, and 94.81% validation accuracy, respectively. Moreover, the proposed model outperforms state-of-the-art methods on the CE-MRI dataset, demonstrating the feasibility of automating hyperparameter optimization.

Airport pavement roughness is a critical factor influencing aircraft safety, operational efficiency, and maintenance costs. Accurate prediction of pavement roughness is essential for effective pavement management and timely maintenance interventions. However, existing prediction methods often suffer from limited accuracy and inadequate adaptability to varying climatic conditions. To address these challenges, a random forest (RF) model optimized via hyperparameter optimization algorithms was developed to accurately predict airport pavement roughness based on data collected from 6 airports across 5 distinct climatic zones. Feature variables were selected, and their dimensionality was reduced through Pearson correlation coefficients. The grid search (GS), differential evolution (DE), and Bayesian optimization (BO) algorithms were then used to optimize the three different roughness prediction models: support vector machine (SVM), eXtreme gradient boosting (XGBoost), and RF. The performances of the three different models were further compared. Additionally, model interpretation is conducted by the Shapley additive explanations (SHAP) method. The results indicate that the RF model significantly outperforms the SVM and XGBoost models, showing notable improvements in overall prediction accuracy, such as R2, root mean squared error, mean absolute error, and mean absolute percentage error. Compared with the GS and DE algorithms, the BO algorithm is the most effective method for enhancing the precision of the RF roughness prediction model. The BO-RF model shows significant improvements in both accuracy and efficiency, particularly with a sharp reduction in runtime and a substantial increase in precision. SHAP analysis reveals the three key feature variables that significantly influence airport pavement predict roughness model: total sunshine hours, average relative humidity, and impact stiffness modulus. These findings and the optimized BO-RF model provide a robust and efficient approach that can be applied globally to improve airport pavement roughness prediction, supporting better maintenance decision-making under diverse climatic conditions.

In this study, we aim to provide an efficient load prediction system projected for different local feeders to predict the Medium- and Long-term Load Forecasting. This model improves future requirements for expansions, equipment retailing or staff recruiting to the electric utility company. We aimed to improve ahead forecasting by using hybrid approach and optimizing the parameters of our models. We used Long Short-Term Memory (LSTM), Convolutional Neural Network (CNN), Multilayer perceptron (MLP) and hybrid methods. We used Root Mean Square Error (RMSE), Mean Absolute Percentage Error (MAPE), Mean Absolute Error (MAE) and squared error for comparison. To predict the 3 months ahead load forecasting, the lowermost prediction error was acquired using LSTM MAPE (2.70). For 6 months ahead forecasting prediction, MLP gives highest predictions with MAPE (2.36). Moreover, to predict the 9 months ahead load forecasting, the highest prediction has been attained using LSTM in terms of MAPE (2.37). Likewise, ahead 1 years MAPE (2.25) was yielded using LSTM and ahead six years MAPE (2.49) was provided using MLP. The proposed methods attain stable and better performance for prediction of load forecasting. The finding indicates that this model can be better instigated for future expansion requirements.

A reliable combustion monitoring system is essential to satisfy global carbon neutrality trends. As the concentrations of emissions and flame stability are associated with the air–fuel ratio, the equivalence ratio should be continuously evaluated. In this study, a deep neural network- (DNN-) based regression model is proposed to predict the equivalence ratio of turbulent diffusion flames. Chemiluminescence signals from the OH ∗ , CH ∗ , and C2 ∗ radicals were acquired as input features. In addition, three different optical sensing views were applied to consider the future general measurement conditions. Furthermore, a loss function comparison for model training and hyperparameter tuning techniques, such as random search and Bayesian optimization, were used to improve the prediction performance. Consequently, the enhanced DNN model showed reductions in the mean absolute error and root mean square error of ~17.84% and ~12.06%, respectively, compared with the initial model. In addition, a mean absolute percentage error and R -squared value of ~3.61% and ~0.9311, respectively, were obtained. Thus, a novel sensing method has been proposed for flame monitoring systems to realize future digital transformations in the combustion industry.

This study explores the potential of the GPBoost approach for groundwater quality assessment in comparison to three other gradient boosting-based algorithms. Three methods, random search, grid search, and Bayesian optimization were used to find the optimal values of various hyperparameters with all four-gradient boosting-based algorithms. One hundred and two samples of Entropy weighted water quality index with 14 input parameters are used for assessing groundwater quality. The calculated EWQI values for drinking range between 80.4 and 394.96 in pre-monsoon and 39.6 to 338.79 during the post-monsoon period. Moreover, spatial distribution maps displayed that the central portions of the study area fall under medium water quality. The performances of models were compared based on multiple statistical criteria, including Correlation Coefficient (CC), root mean square error (RMSE), and mean absolute error (MAE). The results reveal that the CC value by all modeling approaches is more than 0.93, suggesting a comparable performance by all methods. Results in terms of RMSE values in predicting the EWQI values suggest GPBoost (random search) model performed better than the other three models, thus suggesting a competitive performance by GPBoost in comparison to other gradient boosting-based approaches. Relative importance analysis provided by random and grid search methods highlights the significance of NO3 −, Mg2+, TDS, EC, and TH as important input parameters for predicting EWQI.

The proliferation of cyber security attacks necessitates advanced and efficient detection methods. This study explores the application of Convolutional Neural Networks (CNNs) for classifying cyber security attacks using a comprehensive dataset containing various attack types and network traffic features. Emphasizing the role of hyperparameter optimization (HPO) techniques, this research aims to enhance the CNN model's performance in accurately detecting and classifying cyber attacks. Traditional machine learning approaches often need to catch up in capturing the complex patterns in such data, whereas CNNs excel in automatically extracting hierarchical features. Using the provided dataset, which includes attributes such as packet length, source and destination ports, protocol, and traffic type, we implemented various (HPO) techniques, including Grid Search, Random Search, and Bayesian Optimization, to identify the optimal CNN configurations. Our optimized CNN model significantly improved classification result. to baseline models without hyperparameter tuning. The results underline the importance of HPO in developing robust CNN models for cybersecurity applications. This study provides a practical framework for leveraging deep learning and optimization techniques to enhance cyber defense mechanisms, paving the way for future advancements in the field.

The prediction of rock bursts is of paramount importance in ensuring the safety of coal mine production. In order to enhance the precision of rock burst prediction, this paper utilizes a working face of the Gengcun Coal Mine as a case study. The paper employs a three-year microseismic monitoring data set from the working face and employs a sensitivity analysis to identify three monitoring indicators with a higher correlation with rock bursts: daily total energy, daily maximum energy, and daily frequency. Three subsets are created from the 10-day monitoring data: daily frequency, daily maximum energy, and daily total energy. The impact risk score of the next day is assessed as the sample label by the expert assessment system. Sample input and sample label define the data set. The long short-term memory (LSTM) neural network is employed to extract the features of time series. The Bayesian optimization algorithm is introduced to optimize the model, and the Bayesian optimization–long short-term memory (BO-LSTM) combination model is established. The prediction effect of the BO-LSTM model is compared with that of the gated recurrent unit (GRU) and the convolutional neural network (1DCNN). The results demonstrate that the BO-LSTM combined model has a practical application value because the four evaluation indexes of the model are mean absolute error (MAE), mean absolute percentage error (MAPE), variance accounted for (VAF), and mean squared error (MSE) of 0.026272, 0.226405, 0.870296, and 0.001102, respectively. These values are better than those of the other two single models. The rock explosion prediction model can make use of the research findings as a guide.