Random Search Optimization Research Articles

Multi-Step Ahead Water Level Forecasting Using Deep Neural Networks

Stream gauge height (water level) is a significant indicator for forecasting future floods. Flooding occurs when the water level exceeds the flood stage. Predicting imminent floods can save lives, protect infrastructure, and improve road traffic management and transportation. Deep neural networks have been increasingly used in this domain due to their predictive capabilities in capturing complex features and interdependencies. This study employs four distinct models—Multi-Layer Perceptron (MLP), Long Short-Term Memory (LSTM), transformer, and LSTNet—with MLP serving as the baseline model to forecast water levels. The models are trained using data from 20 distinct river gages across the state of Missouri to ensure consistent performance. Random search optimization is employed for hyperparameter tuning. The prediction intervals are set at 4, 6, 8, and 10 (each interval equivalent to 30 min) to ensure that performance results are robust and not due to random weight initialization or suboptimal hyperparameters and are consistent throughout different prediction intervals. The findings of this study indicate that the LSTNet model leads to a better performance than the other models, with a median RMSE of 0.00724, 0.00959, 0.01204, and 0.01230 for the 4, 6, 8, and 10 intervals, respectively. As climate change leads to localized storms driven by atmospheric shifts, water level fluctuations are becoming increasingly extreme, further exacerbating data drift in real-world datasets. The LSTNet model demonstrates superior performance in terms of RMSE, MAE, and the correlation coefficient across all prediction intervals when forecasting water levels under data drift conditions.

Research on reactor power prediction of nuclear power plant based on multivariate optimization GRU model

In the operation of nuclear power plants, the accurate prediction of power change trends is crucial for ensuring safety and stability. In this work, a ML-GRU-RS method, based on model-agnostic meta-learning (MAML), gate recurrent unit (GRU), and random search optimization, is proposed for the long-term prediction of key parameters of nuclear power plants. This method combines the fast adaptability of MAML, the time series data processing capability of GRU, and the optimization efficiency of random search to achieve high-precision predictions under varying power conditions. The results demonstrate that this method can effectively predict the future trends of key parameters in nuclear power plants. The ability of operators to anticipate these trends has been significantly enhanced, contributing to the overall safety of the nuclear power plants.

Ensemble and optimization algorithm in support vector machines for classification of wheat genotypes.

This study aimed to classifying wheat genotypes using support vector machines (SVMs) improved with ensemble algorithms and optimization techniques. Utilizing data from 302 wheat genotypes and 14 morphological attributes to evaluate six SVM kernels: linear, radial basis function (RBF), sigmoid, and polynomial degrees 1-3. Various optimization methods, including grid search, random search, genetic algorithms, differential evolution, and particle swarm optimization, were used. The radial basis function kernel achieves the highest accuracy at 93.2%, and the weighted accuracy ensemble further improves it to 94.9%. This study shows the effectiveness of these methods in agricultural research and crop improvement. Notably, optimization-based SVM classification, particularly with particle swarm optimization, saw a significant 1.7% accuracy gain in the test set, reaching 94.9% accuracy. These findings underscore the efficacy of RBF kernels and optimization techniques in improving wheat genotype classification accuracy and highlight the potential of SVMs in agricultural research and crop improvement endeavors.

Impact of crop management practices on maize yield: Insights from farming in tropical regions and predictive modeling using machine learning

Understanding the influence of crop management practices on maize yield is crucial amidst the changing environmental conditions. Smallholder farming in tropical regions has long puzzled decision-makers in terms of maize management. Balancing alternative management practices, environmental factors, and economic outcomes is essential. In this study, we examine the relationships between maize yield and various factors including climate variables, soil quality parameters, cultivars, tillage practices, and fertilizer usage in Chiapas, Mexico. Pearson's correlation coefficient and T-test were employed to determine the statistical significance of the correlations. Our findings reveal strong positive associations between maize yields and factors such as planting, cultivar, vapor pressure, temperature, solar radiations and fertilizer inputs for nitrogen, phosphorus, and potassium in lower elevation farms. Conversely, there is a negative correlation with elevation, slope, and system. Extensive data analysis was conducted to investigate the impact of different crop management practices on yield, utilizing both data visualization and analytics. Additionally, maize yield prediction was performed using six hybrid machine learning (ML) models, employing Grid Search and Random Search optimization techniques. Based on evaluation, Grid Search-based XGBoost exhibited the most accurate prediction results. Furthermore, explainable artificial intelligence (XAI) was employed to assess the individual impact of each input parameter on the ML model output.

A Comprehensive Review of Convolutional Neural Networks based Disease Detection Strategies in Potato Agriculture

AbstractThis review paper investigates the utilization of Convolutional Neural Networks (CNNs) for disease detection in potato agriculture, highlighting their pivotal role in efficiently analyzing large-scale agricultural datasets. The datasets used, preprocessing methodologies applied, specific data collection zones, and the efficacy of prominent algorithms like ResNet, VGG, and MobileNet variants for disease classification are scrutinized. Additionally, various hyperparameter optimization techniques such as grid search, random search, genetic algorithms, and Bayesian optimization are examined, and their impact on model performance is assessed. Challenges including dataset scarcity, variability in disease symptoms, and the generalization of models across diverse environmental conditions are addressed in the discussion section. Opportunities for advancing CNN-based disease detection, including the integration of multi-spectral imaging and remote sensing data, and the implementation of federated learning for collaborative model training, are explored. Future directions propose research into robust transfer learning techniques and the deployment of CNNs in real-time monitoring systems for proactive disease management in potato agriculture. Current knowledge is consolidated, research gaps are identified, and avenues for future research in CNN-based disease detection strategies to sustain potato farming effectively are proposed by this review. This study paves the way for future advancements in AI-driven disease detection, potentially revolutionizing agricultural practices and enhancing food security. Also, it aims to guide future research and development efforts in CNN-based disease detection for potato agriculture, potentially leading to improved crop management practices, increased yields, and enhanced food security.

A reliable breast cancer diagnosis approach using an optimized deep learning and conformal prediction

Accuracy of breast cancer detection using histopathological images is mainly subject to the expertise of physicians. For this reason, there are certain levels of uncertainty during the decision making of breast cancer diagnosis. To increase the certainty of physicians’ decisions, computer-aided medical systems can be used as additional knowledge for diagnosis from histopathological images. Early detection of breast cancer contributes greatly to the survival rate of the patients, which is essentially dependent on the analysis of breast images. In the literature review, few studies have been proposed for breast cancer classification based on histopathological images. However, the performance of current methods is still inefficient and they are incapable for extracting important features from the images due to ineffective hyper-parameters optimization. In this paper, a reliable deep learning approach for breast cancer diagnosis is proposed using a random search algorithm and DenseNet-121-based transfer-learning model. For the transfer learning technique, the pre-trained DenseNet-121 model is adopted due to its capability to strengthen the feature representation and reduce the number of trainable parameters. The use of random search optimization finds the best combinations of hyper-parameters values to improve the accuracy of the classification approach. Several experiments are conducted on a benchmark public dataset of histopathological images with different magnification factors: 40×, 100×, 200×, and 400×. The experiments showed that the proposed approach achieved 98.96 %, 97.62 %, 97.08 %, and 96.42 % of accuracy for 40×, 100×, 200×, and 400× test images, respectively, demonstrating its effectiveness compared to the state-of-the-art models and methods. The reliability of proposed approach is achieved by quantifying the uncertainty of model outcomes using conformal prediction method, guarantying user-chosen levels of confidence.

Time series prediction model using LSTM-Transformer neural network for mine water inflow

Mine flooding accidents have occurred frequently in recent years, and the predicting of mine water inflow is one of the most crucial flood warning indicators. Further, the mine water inflow is characterized by non-linearity and instability, making it difficult to predict. Accordingly, we propose a time series prediction model based on the fusion of the Transformer algorithm, which relies on self-attention, and the LSTM algorithm, which captures long-term dependencies. In this paper, Baotailong mine water inflow in Heilongjiang Province is used as sample data, and the sample data is divided into different ratios of the training set and test set in order to obtain optimal prediction results. In this study, we demonstrate that the LSTM-Transformer model exhibits the highest training accuracy when the ratio is 7:3. To improve the efficiency of search, the combination of random search and Bayesian optimization is used to determine the network model parameters and regularization parameters. Finally, in order to verify the accuracy of the LSTM-Transformer model, the LSTM-Transformer model is compared with LSTM, CNN, Transformer and CNN–LSTM models. The results prove that LSTM-Transformer has the highest prediction accuracy, and all the indicators of its model are well improved.

Exploring The Efficiency of Metaheuristics in Optimal Hyperparameter Tuning for Ensemble Models on Varied Data Modalities

Effective disease detection systems play an important role in healthcare by supporting diagnosis and treatment. This study provides a comparison of hyperparameter tuning methods for disease detection systems using four health datasets; kidney disease, diabetes detection, heart disease and breast cancer detection. The main objective of this research is to prepare datasets by normalizing the input and testing machine learning models such as Naive Bayes Support Vector Machine (SVM), Logistic Regression and k Nearest Neighbor (kNN). to identify effective models for each data set. After implementing the models, we apply three hyperparameter tuning techniques: Grid search, random search, and particle ensemble optimization (PSO). These methods are used to tune the model parameters. Improve overall performance metrics. The evaluation focuses on accuracy measurements to compare model performance before and after hyperparameter tuning. The results of this study illustrate how different tuning techniques can improve the performance of disease detection systems across a range of healthcare datasets. By conducting testing and analysis, we determine the appropriate tuning method for each data set, yielding valuable insights, to develop an accurate and effective disease detection system .These discoveries serve to advance the field of healthcare analytics and machine learning to deliver outcomes for patients and healthcare services.

Very short-term wind power forecasting for real-time operation using hybrid deep learning model with optimization algorithm

This paper proposes a new hybrid deep learning model to enhance the accuracy of forecasting very short-term wind power generation. The proposed model comprises a convolutional layer, a long-short-term memory (LSTM) unit, and fully connected neural network. Convolution layer can automatically learn complicated features from the raw input, whereas the LSTM layers can retain useful information through which gradient information may flow over extended periods. To obtain the best performance from the forecasting model, a random search optimization technique has been developed for tuning hyper-parameters of the model developed. The 5 min datasets from the White Rock wind farm, Australia are used to investigate the effectiveness of the proposed model as wind farms are participating in spot electricity market. To compare the effectiveness, the proposed model is compared with the existing models, such as convolution neural network (CNN), LSTM, gated recurrent unit (GRU), bidirectional LSTM (BiLSTM), artificial neural network (ANN), and support vector machine (SVM). The root-mean-square error (RMSE), mean absolute error (MAE), and Theil’s inequality coefficient (TIC) are used to analyze and compare the performances of the predictive models. Based on RMSE and MAE, the proposed model exhibits a higher accuracy of approximately 23.79% and 28.63% compared to other forecasting methods, respectively.

A Double Exponential Particle Swarm Optimization with non-uniform variates as stochastic tuning and guaranteed convergence to a global optimum with sample applications to finding optimal exact designs in biostatistics

Nature-inspired metaheuristic algorithms, like Particle Swarm Optimization (PSO), are powerful general-purpose optimization tools but they invariably do not come with rigorous theoretical justifications and can fail to find a global optimum. By treating PSO as a random search optimization process and repairing the famous Global Search Convergence Theorem by imposing an additional condition in the proof, we create a novel theory-based algorithm called the Double Exponential Particle Swarm optimization algorithm (DExPSO) that converges to a global optimum. In particular, we show the common practice of using uniform variates as stochastic components in PSO and related algorithms does not satisfy the sufficient conditions in DExPSO and hence may provide a reason why PSO and other nature-inspired algorithms, like QPSO, LcRiPSO, and CSO can fail to converge. Additionally, in more complicated design problems, we show that DExpSO tends to converge to the support points of the optimal design more frequently and faster than PSO and its variants do. Moreover, there is a possibility to modify other PSO variants to DExPSO variants, and such hybridization offers promising improvement in the quality of the global search. Our applications include finding designs that minimize the integrated mean squared prediction error and locally D-optimal exact designs for a 68-compartmental model to assess radioactive particles retained in the human lung after exposure. Because PSO, and more generally, metaheuristics are used across disciplines, including ecology, pharmacokinetics and pharmacodynamics studies, agriculture, engineering, and computer science, there are potentially broad and deep implications of our results.

A Comparison of Enhanced Ensemble Learning Techniques for Internet of Things Network Attack Detection

Over the past few decades, the Internet of Things (IoT) has become increasingly significant due to its capacity to enable low-cost device and sensor communication. Implementation has opened up many new opportunities in terms of efficiency, productivity, convenience, and security. However, it has also brought about new privacy and data security challenges, interoperability, and network reliability. The research issue is that IoT devices are frequently open to attacks. Certain machine learning (ML) algorithms still struggle to handle imbalanced data and have weak generalization skills when compared to ensemble learning. The research aims to develop security for IoT networks based on enhanced ensemble learning by using Grid Search and Random Search techniques. The method used is the ensemble learning approach, which consists of Random Forest (RF), Adaptive Boosting (AdaBoost), Gradient Boosting Machine (GBM), and Extreme Gradient Boosting (XGBoost). This study uses the UNSW-NB15 IoT dataset. The study's findings demonstrate that XGBoost performs better than other methods at identifying IoT network attacks. By employing Grid Search and Random Search optimization, XGBoost achieves an accuracy rate of 98.56% in binary model measurements and 97.47% on multi-class data. The findings underscore the efficacy of XGBoost in bolstering security within IoT networks.

VoteDroid: a new ensemble voting classifier for malware detection based on fine-tuned deep learning models

AbstractIn this work, VoteDroid a novel fine-tuned deep learning models-based ensemble voting classifier has been proposed for detecting malicious behavior in Android applications. To this end, we proposed adopting the random search optimization algorithm for deciding the structure of the models used as voter classifiers in the ensemble classifier. We specified the potential components that can be used in each model and left the random search algorithm taking a decision about the structure of the model including the number of each component that should be used and its location in the structure. This optimization method has been used to build three different deep learning models namely CNN-ANN, pure CNN, and pure ANN. After selecting the best structure for each DL model, the selected three models have been trained and tested using the constructed image dataset. Afterward, we suggested hybridizing the fine-tuned three deep-learning models to form one ensemble voting classifier with two different working modes namely MMR (Malware Minority Rule) and LMR (Label Majority Rule). To our knowledge, this is the first time that an ensemble classifier has been fine-tuned and hybridized in this way for malware detection. The results showed that the proposed models were promising, where the classification accuracy exceeded 97% in all experiments.

Application of a hybrid algorithm of LSTM and Transformer based on random search optimization for improving rainfall-runoff simulation

Flood forecasting using traditional physical hydrology models requires consideration of multiple complex physical processes including the spatio-temporal distribution of rainfall, the spatial heterogeneity of watershed sub-surface characteristics, and runoff generation and routing behaviours. Data-driven models offer novel solutions to these challenges, though they are hindered by difficulties in hyperparameter selection and a decline in prediction stability as the lead time extends. This study introduces a hybrid model, the RS-LSTM-Transformer, which combines Random Search (RS), Long Short-Term Memory networks (LSTM), and the Transformer architecture. Applied to the typical Jingle watershed in the middle reaches of the Yellow River, this model utilises rainfall and runoff data from basin sites to simulate flood processes, and its outcomes are compared against those from RS-LSTM, RS-Transformer, RS-BP, and RS-MLP models. It was evaluated against RS-LSTM, RS-Transformer, RS-BP, and RS-MLP models using the Nash–Sutcliffe Efficiency Coefficient (NSE), Root Mean Square Error (RMSE), Mean Absolute Error (MAE), and Bias percentage as metrics. At a 1-h lead time during calibration and validation, the RS-LSTM-Transformer model achieved NSE, RMSE, MAE, and Bias values of 0.970, 14.001m3/s, 5.304m3/s, 0.501% and 0.953, 14.124m3/s, 6.365m3/s, 0.523%, respectively. These results demonstrate the model's superior simulation capabilities and robustness, providing more accurate peak flow forecasts as the lead time increases. The study highlights the RS-LSTM-Transformer model's potential in flood forecasting and the advantages of integrating various data-driven approaches for innovative modelling.

Enhancing hydrogen production prediction from biomass gasification via data augmentation and explainable AI: A comparative analysis

Hydrogen production for clean energy is gaining a foothold, notably through the gasification of biomass. Machine learning aids in its accurate production predictions, yet its opaque nature limits explanation. This study looked into machine learning models' transparency and evaluated the impact of data augmentation on hydrogen production prediction accuracy using a publicly available numerical dataset. Fifteen (15) machine learning regression models including linear and tree-based were evaluated leading to the introduction of a hybrid model. The proposed hybrid model is based on ensemble and averaging techniques in mathematics. The deployed model's prediction was evaluated using the Mean Absolute Error (MAE), the Mean Square Error (MSE), the Root Mean Squared Error (RMSE), the Root Mean Squared Log Error (RMSLE), and R-squared (R2) based on K-Fold cross-validation. Furthermore, the effect of the grid search, random search, and Bayesian optimization techniques for model hyperparameter optimization were carried out. The evaluated models demonstrate superior predictions and explore data augmentation's potential in refining machine learning accuracy with the proposed hybrid model recording an MAE of 2.187, MSE of 9.788, RMSE of 3.129, R2 of 0.250, MSLE of 0.005, and RMSLE of 0.070 against the original data result of MAE of 2.665, MSE of 11.877, RMSE of 3.446, R2 of 0.112, MSLE of 0.006 and RMSLE of 0.076. However, when the proposed model hyperparameter training setting was optimized, the yielded result improved further with an MAE of 0.044, MSE of 0.040, RMSE of 0.215, R2 of 0.996, MSLE of 0.000 and RMSLE of 0.005. By unraveling the inner workings of black box models using the SHAP, LIME, and ELI-5 interpretable AI models, this study advances hydrogen production prediction using biomass gasification. Insights gained in this study foster the development of more transparent and effective models, aiming for the widespread adoption of sustainable hydrogen energy.

Hyperparameter Tunning and Feature Selection Methods for Malware Detection

Smartphones have started to take an essential place in every aspect of our lives with the developing technology. All kinds of transactions, from daily routine work to business meetings, payments, and personal transactions, started to be done via smartphones. Therefore, there is a significant amount of very important user information stored in these devices which makes them a target for malware developers. For these reasons, machine learning (ML) methods have been used to detect malicious software on android devices quickly and reliably. In this study, a machine learning-based Android malware detection system has been developed, optimized, and tested. To this end, firstly, the data in the dataset has been balanced with 3 different methods namely SMOTE, SMOTETomek and ClusterCentroids. Afterward, the obtained results have been tried to be optimized by using different feature selection approaches including mRMR, Mutual Information, Select From Model, and Select k Best. Finally, the most two successful methods from the five tested ML algorithms (i.e. RF, SVM, LR, XGBoost, and ETC) have been tuned using GridSearch, Random Search, and Bayesian Optimization algorithms in order to investigate the effects of hyperparameter tuning on the performance of ML algorithms.

A novel approach for prediction of groundwater quality using gradient boosting-based algorithms

ABSTRACT This study explores the potential of the GPBoost approach for groundwater quality assessment in comparison to three other gradient boosting-based algorithms. Three methods, random search, grid search, and Bayesian optimization were used to find the optimal values of various hyperparameters with all four-gradient boosting-based algorithms. One hundred and two samples of Entropy weighted water quality index with 14 input parameters are used for assessing groundwater quality. The calculated EWQI values for drinking range between 80.4 and 394.96 in pre-monsoon and 39.6 to 338.79 during the post-monsoon period. Moreover, spatial distribution maps displayed that the central portions of the study area fall under medium water quality. The performances of models were compared based on multiple statistical criteria, including Correlation Coefficient (CC), root mean square error (RMSE), and mean absolute error (MAE). The results reveal that the CC value by all modeling approaches is more than 0.93, suggesting a comparable performance by all methods. Results in terms of RMSE values in predicting the EWQI values suggest GPBoost (random search) model performed better than the other three models, thus suggesting a competitive performance by GPBoost in comparison to other gradient boosting-based approaches. Relative importance analysis provided by random and grid search methods highlights the significance of NO3 −, Mg2+, TDS, EC, and TH as important input parameters for predicting EWQI.

Prediction Stunting Analysis Using Random Forest Algorithm and Random Search Optimization

Stunting is a nutritional problem experienced by toddlers, characterized by a height below the average. This condition arises due to various factors, one of which is the nutritional issues faced by toddlers. Stunting cases in Indonesia are relatively high, reaching 21.6% in 2022, indicating a significant prevalence of stunting. The identification of stunting is carried out through a data mining approach, deemed more efficient. However, the classification algorithm in data mining often encounters data imbalance, leading to low accuracy and inaccurate prediction results. To address this, the study employs the Random Forest algorithm with optimization using the random search method. The test results demonstrate that Random Forest achieves a relatively high accuracy of 90.7%. After optimization using random search, accuracy further increases to 96.33%. The combination of the algorithm and optimization proves to be highly effective, resulting in a 5.63% increase in accuracy. These findings hold crucial implications in supporting decisions for preventing stunting in toddlers. This research serves as a valuable source of information for the Health sector in identifying and implementing more effective strategies for stunting prevention. The use of the Random Forest algorithm optimized with random search proves to be an efficient solution in addressing data imbalance

Software reliability prediction using ensemble learning with random hyperparameter optimization

The paper investigates software reliability prediction by using ensemble learning with random hyperparameter optimization. Software reliability is a significant problem with software quality that developers face. It involves accurately predicting the next failure. In recent years, machine learning techniques and ensemble learning approaches have been applied to improve software reliability prediction. These approaches aim to analyze historical data and develop models that can accurately forecast when failures are likely to occur. The article proposes an ensemble learning regression model using Ridge, Bayesian Ridge, Support Vector Regressor (SVR), K-Nearest Neighbors Algorithm (KNN), Regression tree, Random Forest, Neural network, and Decision Tree as base learners. Ridge is used as a combiner model. Each base learner hyperparameter is tuned using a random search algorithm automatically. A random hyperparameter search optimization algorithm selects the hyperparameter and adjusts it for overfitting and underfitting. The base models are tuned to minimize bias and variance. The performances of the models are evaluated using standard error measures such as Mean Squared Error (MSE), Sum of Squared Error (SSE), and Normalized Root Mean Square Error (NRMSE). The proposed ensemble model is compared with existing models using a benchmark dataset. The Iyer,and Lee, and Musa datasets are used for the experiment. The dataset is scaled using standard methods like logarithmic scaling, lagging, and linear interpolation. The results of the statistical comparison show better performance by our proposed model as compared to existing models.

Wafer Defect Identification with Optimal Hyper-Parameter Tuning of Support Vector Machine Using the Deep Feature of ResNet 101

As semiconductor processing technologies continue to advance, semiconductor wafers are becoming more densely packed and intricate, resulting in a higher incidence of surface imperfections. Therefore, it is crucial to detect these defects early and accurately classify them to pinpoint the root causes of the defects in the manufacturing process, ultimately leading to improved yield. Therefore, defect detection is critical in the industrial production of monocrystalline silicon. This study employs deep learning techniques to propose a framework for detecting defects on silicon wafers, focusing on optimizing the hyperparameters of support vector machines (SVM). Three methods were utilized to fine-tune the SVM parameters: Bayesian optimization, grid search, and random search techniques. This study demonstrates how selecting optimal values for SVM parameters can lead to better classification. Additionally, real manufacturing data were utilized to evaluate the performance of the proposed SVM classifier, with a comparison to state-of-the-art techniques in the field. By using deep features from ResNet 101 and a support vector machine, this work achieves 74.5% accuracy in identifying wafer defects without employing any optimization technique. However, the performance of the model was further improved by utilizing the random search optimization technique, which yielded the best result among the three optimization techniques tested, with an accuracy of 88.1%.

Algorithmic Proficiency in Spark Configuration Tuning: An Empirical Study using Execution Time Metrics across Varied Workloads

In the realm of big data, where datasets of immense scale pose processing challenges, distributed processing platforms like open-source Apache Spark have emerged to address these issues. Spark’s internal configuration parameters exert varying impacts on execution times based on job characteristics, making manual optimization daunting. The core focus of this study lies in optimizing Spark’s internal configurations, with specific attention directed towards three types of workloads: Iterative-intensive, Memory-intensive, and CPU-intensive. Employing Grid Search, Random Search, and Evolutionary Optimization algorithms yields substantial execution time reductions: 23.24% with Grid Search, 19.71% with Random Search, and 23.06% with Evolutionary Optimization. Notably, Evolutionary Optimization achieves optimal configurations approximately 29% faster than Grid Search. While Random Search and Evolutionary Optimization share similar time requirements, Random Search’s execution time reduction for a given Spark workload is relatively lower. This research sheds light on algorithmic configuration tuning intricacies and its influence on Spark workload execution times, contributing to the exploration of optimizing big data processing platforms.