Bayesian Optimization Algorithm Research Articles

A parallel constrained Bayesian optimization algorithm for high-dimensional expensive problems and its application in optimization of VRB structures

A parallel constrained Bayesian optimization algorithm for high-dimensional expensive problems and its application in optimization of VRB structures

Experimental analysis and prediction of radionuclide solubility using machine learning models: Effects of organic complexing agents

Radioactive wastes contain organic complexing agents that can form complexes with radionuclides and enhance the solubility of these radionuclides, increasing the mobility of radionuclides over great distances from a radioactive waste repository. In this study, four radionuclides (cobalt, strontium, iodine, and uranium) and three organic complexing agents (ethylenediaminetetraacetic acid, nitrilotriacetic acid, and iso-saccharic acid) were selected, and the solubility of these radionuclides was assessed under realistic environmental conditions such as different pHs (7, 9, 11, and 13), temperatures (10 °C, 20 °C, and 40 °C), and organic complexing agent concentrations (10−5–10−2 M). A total of 720 datasets were generated from solubility batch experiments. Four supervised machine learning models such as the Gaussian process regression (GPR), ensemble-boosted trees, artificial neural networks, and support vector machine were developed for predicting the radionuclide solubility. Each ML model was optimized using Bayesian optimization algorithm. The GPR evolved as a robust model that provided accurate predictions within the underlying solubility patterns by capturing the intricate relationships of the independent parameters of the dataset. At an uncertainty level of 95%, both the experimental results and GPR simulated estimations were closely correlated, confirming the suitability of the GPR model for future explorations.

An emission predictive system for CO and NOx from gas turbine based on ensemble machine learning approach

The gas turbine in a combined cyclic power plant (CCPP) produces harmful gases like carbon monoxide (CO) and nitrogen oxide (NOx) into the atmosphere. It is evident to monitor the rate at which these gases are produced during power generation to comply with the industrial standard for emission. Therefore, a system is required to continuously monitor the emission from the CCPP gas turbine. Hence, this work aims to design a stacked ensemble machine learning (SEM) based predictive model for CO and NOx emission from a CCPP gas turbine. The neural network for regression (NNR), a generalized additive model (GAM), and the bagging of regression trees (BT) act as the base learners. A generalized regression neural network (GRNN) is used as a meta-learner for SEM. The hyperparameters of SEM are optimized using a Bayesian optimization algorithm for CO and NOX prediction. In addition to this, the performance of SEM is compared with support vector regression (SVR), decision tree (DRT), and linear regression (LIR). Simulation results demonstrate that SEM can reduce the RMSE 5.7–93.8% for NOx and 1%-41.5% for CO compared to other ML techniques. Finally, comparing the results with ML techniques existing in the literature shows the higher predictive accuracy of the proposed SEM.

A data‐driven Bayesian approach for optimal dynamic product transitions

AbstractIn the processing industry, dynamic product transitions are essential for achieving high product quality, minimizing the use of raw materials and energy and reducing production costs. However, optimizing dynamic product transitions is a challenging task due to the complex dynamics of the process and the uncertainty in the measurements. In this article, a data‐driven Bayesian approach for optimal dynamic product transitions is proposed. The proposed approach is based on a dynamic optimization problem that is solved using a Bayesian optimization algorithm. One of the advantages of this approach for process optimization tasks is that it does not require a first‐principles dynamic mathematical model for drawing optimal solutions. The approach is applied to three case studies, and the results are comparable in performance quality with those obtained using a traditional gradient‐based optimization approach. The results show that the proposed approach is able to find optimal transition trajectories that meet the product composition requirements using smooth control actions. The approach is also able to cope with noisy measurements, which is an important feature in real‐world applications. The proposed approach has several advantages over traditional optimization approaches, including being data driven, able to cope with noisy measurements, computationally efficient, and it requires modest computational effort. Complex online optimal control problems can benefit from adopting a data‐driven Bayesian optimization scheme.

Improving the mechanical properties of Cantor-like alloys with Bayesian optimization

The search for better compositions in high entropy alloys is a formidable challenge in materials science. Here, we demonstrate a systematic Bayesian optimization method to enhance the mechanical properties of the paradigmatic five-element Cantor alloy in silico. This method utilizes an automated loop with an online database, a Bayesian optimization algorithm, thermodynamic modeling, and molecular dynamics simulations. Starting from the equiatomic Cantor composition, our approach optimizes the relative fractions of its constituent elements, searching for better compositions while maintaining the thermodynamic phase stability. With 24 steps, we find Fe21Cr20Mn5Co20Ni34 with a yield stress improvement of 58%, and with 72 steps, we find Fe6Cr22Mn5Co32Ni35 where the yield stress has improved by 74%. These optimized compositions correspond to Ni-rich medium entropy alloys with enhanced mechanical properties and superior face-centered-cubic phase stability compared to the traditional equiatomic Cantor alloy. The automatic approach devised here paves the way for designing high entropy alloys with tailored properties, opening avenues for numerous potential applications.

Optimal design of heat pipes for city gate station heaters by applying genetic and Bayesian optimization algorithms to an artificial neural network model

This study contains a novel idea to improve the performance of city gate station heaters, using pulsating heat pipes with optimized structure. In this research, a hybrid method including Bayesian and genetic algorithms is implemented to optimize the architecture of an artificial neural network (ANN) model and find the optimal operating point for heat pipes. In the previous study, the authors have proposed a neural network-based model to predict the thermal performance of heat pipes, based on a wide range of their design parameters, including dimensions of heat pipe, number of turns, working fluid, inclination angle, filling ratio, and heat input. In the present study, the prediction model architecture is optimized using Bayesian optimization algorithm. The best performance is achieved when iterations of the employed Bayesian optimization is set to 300 iterations or samples, which 150 samples are produced to train the Bayesian model and other 150 iterations are utilized in order to find the optimal ANN structure and hyper-parameter tuning of the model. The optimal ANN model with structure of 9-97-21-68-100-1 has evaluation parameters of 0.980 and 0.014 for R2 and MSE of total dataset, respectively. The performance metrics indicate an improvement in accuracy of the ANN model with a 53.33% reduction in MSE value and a 3.15% increase in R2 score. Then, the genetic algorithm is applied to the optimal model to find the optimum operating parameters of the heat pipe. In this regard, the design parameters of the best heat pipe in order to use in the city gate station heaters, are obtained by setting the genetic algorithm population equal to 300. Finally, as a practical case study, a real heater with capacity of 10,000 SCMH which is located in Iran is selected. Considering constraints and limitations of this heater, the proposed model indicates that a vertical pulsating heat pipe with four turns which is filled 55% with water as the working fluid, has the optimal thermal resistance of 0.104 K/W. Utilizing the optimal heat pipe in the mentioned heater is a promising solution for efficiency improvement and fuel consumption and CO2 emission reduction, which decreases the operating costs of the heater as well.

Hyperparameter Tunning and Feature Selection Methods for Malware Detection

Smartphones have started to take an essential place in every aspect of our lives with the developing technology. All kinds of transactions, from daily routine work to business meetings, payments, and personal transactions, started to be done via smartphones. Therefore, there is a significant amount of very important user information stored in these devices which makes them a target for malware developers. For these reasons, machine learning (ML) methods have been used to detect malicious software on android devices quickly and reliably. In this study, a machine learning-based Android malware detection system has been developed, optimized, and tested. To this end, firstly, the data in the dataset has been balanced with 3 different methods namely SMOTE, SMOTETomek and ClusterCentroids. Afterward, the obtained results have been tried to be optimized by using different feature selection approaches including mRMR, Mutual Information, Select From Model, and Select k Best. Finally, the most two successful methods from the five tested ML algorithms (i.e. RF, SVM, LR, XGBoost, and ETC) have been tuned using GridSearch, Random Search, and Bayesian Optimization algorithms in order to investigate the effects of hyperparameter tuning on the performance of ML algorithms.

Bayesian Optimization with Experience for Fast Development of Monolithic Tandem Solar Cells: Simulation Case Study

AbstractMachine learning (ML) is gaining more attention in photovoltaic research and will be a vital tool in reaching record‐high power conversion efficiencies (PCE) in the near future. One area, where ML is significantly beneficial is reducing the number of experiments needed to find the optimum combination of parameters in solar cell fabrication. Bayesian optimization (BO) provides routes for quickly identifying optimum parameters in problems with large parameter space. In this work, BO algorithms utilizing previous knowledge are demonstrated to result in faster optimization of tandem solar cells, a technology rapidly gaining more interest due to its potential to deliver record‐high performance. Namely, it is shown that in the space of all possible parameter combinations that take ≈88 years to evaluate, optimum PCE can be obtained in 20 min. Moreover, it is demonstrated that methods utilizing previous knowledge outperform those that do not by yielding an increase of ≈8.9%abs. in the 1st iteration and requiring 5× less time to reach a target PCE of 38.5% across 20 different trials. Results from this work help accelerate the development of tandem solar cells by removing the need for large numbers of experiments in identifying optimum parameters.

A novel hybrid forecasting model with feature selection and deep learning for wind speed research

AbstractAccurate wind speed prediction is of great importance for the operation of wind farms, and extensive efforts have been made to develop effective forecasting methods in this regard. However, the feature selection of data input as well as optimization of deep learning models have received comparatively less attention, leading to unreliable forecasting results. This research proposes a novel hybrid model that integrates data preprocessing, feature selection, and optimized forecasting for improved wind speed prediction. Specifically, a powerful preprocessing technique is utilized to reduce data noise disturbances, while an innovative two‐stage feature selection is designed to achieve the optimal input data format for forecasting purposes. Moreover, a hybrid forecasting module based on long‐short term memory, which is optimized by the Bayesian optimization algorithm, has been developed to enhance the efficiency and reliability of the model. The empirical study used 10‐min interval wind speed data of four seasons for presentation and evaluation results demonstrated its superior performance in effectively learning the volatility and irregularity features of wind speed series, which established a solid foundation for practical applications in wind power systems.

A physics-informed Bayesian optimization method for rapid development of electrical machines

Advanced slot and winding designs are imperative to create future high performance electrical machines (EM). As a result, the development of methods to design and improve slot filling factor (SFF) has attracted considerable research. Recent developments in manufacturing processes, such as additive manufacturing and alternative materials, has also highlighted a need for novel high-fidelity design techniques to develop high performance complex geometries and topologies. This study therefore introduces a novel physics-informed machine learning (PIML) design optimization process for improving SFF in traction electrical machines used in electric vehicles. A maximum entropy sampling algorithm (MESA) is used to seed a physics-informed Bayesian optimization (PIBO) algorithm, where the target function and its approximations are produced by Gaussian processes (GP)s. The proposed PIBO-MESA is coupled with a 2D finite element model (FEM) to perform a GP-based surrogate and provide the first demonstration of the optimal combination of complex design variables for an electrical machine. Significant computational gains were achieved using the new PIBO-MESA approach, which is 45% faster than existing stochastic methods, such as the non-dominated sorting genetic algorithm II (NSGA-II). The FEM results confirm that the new design optimization process and keystone shaped wires lead to a higher SFF (i.e. by 20%) and electromagnetic improvements (e.g. maximum torque by 12%) with similar resistivity. The newly developed PIBO-MESA design optimization process therefore presents significant benefits in the design of high-performance electric machines, with reduced development time and costs.

A Bayesian optimization R package for multitrait parental selection.

Selecting and mating parents in conventional phenotypic and genomic selection are crucial. Plant breeding programs aim to improve the economic value of crops, considering multiple traits simultaneously. When traits are negatively correlated and/or when there are missing records in some traits, selection becomes more complex. To address this problem, we propose a multitrait selection approach using the Multitrait Parental Selection (MPS) R package-an efficient tool for genetic improvement, precision breeding, and conservation genetics. The package employs Bayesian optimization algorithms and three loss functions (Kullback-Leibler, Energy Score, and Multivariate Asymmetric Loss) to identify parental candidates with desirable traits. The software's functionality includes three main functions-EvalMPS, FastMPS, and ApproxMPS-catering to different data availability scenarios. Through the presented application examples, the MPS R package proves effective in multitrait genomic selection, enabling breeders to make informed decisions and achieve strong performance across multiple traits.

Spatial distribution model of thermal load for high-speed dry hobbing tool based on probability density function

High-speed dry hobbing is a clean and environmental-friendly gear machining process, but thermal dissipation inefficiency accelerates tool wear and shorten tool life which is difficult to be effectively inhibited due to its complex distribution characteristics. This paper proposes a thermal load distribution model based on probability density function, describing the spatial distribution of thermal load on tool and providing theoretical support for predicting the distribution of thermal load. Firstly, conducting a finite element simulation analysis based on a mathematical model of high-speed dry gear hobbing process; Secondly, the influence of various parameters on the constructed model is analyzed, and the Bayesian optimization algorithm is utilized to optimize the model parameters. Finally, the model is validated at the micro and macro level. The results show that the average coefficient of determination of the model is greater than 0.86 in micro level. And the average fitting error is less than 6 °C in macro level, besides, the average relative error is less than 15%. The description of the uneven thermal distribution on tool is obtained with probability density function, it provides a theoretical support for the prediction and further optimization of thermal load on tool under multiple parameters in the future.

Artificial intelligence-based prediction of the rheological properties of hydrocolloids for plant-based meat analogues.

Methylcellulose has been applied as a primary binding agent to control the quality attributes of plant-based meat analogues. H owever, a great deal of effort has been made to search for hydrocolloids to replace methylcellulose because of increasing awareness of clean labels. In this study, a machine learning framework was proposed in order to describe and predict the flow behavior of six hydrocolloid solutions, and the predicted viscosities were correlated with the textural features of their corresponding plant-based meat analogues. Different shear-thinning and Newtonian behaviors were observed depending on the type of hydrocolloid and the shear rate. Methylcellulose exhibited an increasing viscosity pattern with increasing temperature, compared to the other hydrocolloids. The machine learning algorithms (random forest and multilayer perceptron models) showed a better viscosity fitting performance than the constitutive equations (power law and Cross models). In addition, three hyperparameters of the multilayer perceptron model (optimizer, learning rate, and the number of hidden layers) were tuned using the Bayesian optimization algorithm. The optimized multilayer perceptron model exhibited superior performance in viscosity prediction (R2 = 0.9944-0.9961/RMSE = 0.0545-0.0708). Furthermore, the machine learning-predicted viscosities overall showed similar patterns to the textural parameters of the meat analogues. © 2024 Society of Chemical Industry.

Automated Model Selection Using Bayesian Optimization and the Asynchronous Successive Halving Algorithm for Predicting Daily Minimum and Maximum Temperatures

This study addresses the crucial role of temperature forecasting, particularly in agricultural contexts, where daily maximum (Tmax) and minimum (Tmin) temperatures significantly impact crop growth and irrigation planning. While machine learning (ML) models offer a promising avenue for temperature forecasts, the challenge lies in efficiently training multiple models and optimizing their parameters. This research addresses a research gap by proposing advanced ML algorithms for multi-step-ahead Tmax and Tmin forecasting across various weather stations in Bangladesh. The study employs Bayesian optimization and the asynchronous successive halving algorithm (ASHA) to automatically select top-performing ML models by tuning hyperparameters. While both the Bayesian and ASHA optimizations yield satisfactory results, ASHA requires less computational time for convergence. Notably, different top-performing models emerge for Tmax and Tmin across various forecast horizons. The evaluation metrics on the test dataset confirm higher accuracy, efficiency coefficients, and agreement indices, along with lower error values for both Tmax and Tmin forecasts at different weather stations. Notably, the forecasting accuracy decreases with longer horizons, emphasizing the superiority of one-step-ahead predictions. The automated model selection approach using Bayesian and ASHA optimization algorithms proves promising for enhancing the precision of multi-step-ahead temperature forecasting, with potential applications in diverse geographical locations.

Brain tumor classification from MRI scans: a framework of hybrid deep learning model with Bayesian optimization and quantum theory-based marine predator algorithm.

Brain tumor classification is one of the most difficult tasks for clinical diagnosis and treatment in medical image analysis. Any errors that occur throughout the brain tumor diagnosis process may result in a shorter human life span. Nevertheless, most currently used techniques ignore certain features that have particular significance and relevance to the classification problem in favor of extracting and choosing deep significance features. One important area of research is the deep learning-based categorization of brain tumors using brain magnetic resonance imaging (MRI). This paper proposes an automated deep learning model and an optimal information fusion framework for classifying brain tumor from MRI images. The dataset used in this work was imbalanced, a key challenge for training selected networks. This imbalance in the training dataset impacts the performance of deep learning models because it causes the classifier performance to become biased in favor of the majority class. We designed a sparse autoencoder network to generate new images that resolve the problem of imbalance. After that, two pretrained neural networks were modified and the hyperparameters were initialized using Bayesian optimization, which was later utilized for the training process. After that, deep features were extracted from the global average pooling layer. The extracted features contain few irrelevant information; therefore, we proposed an improved Quantum Theory-based Marine Predator Optimization algorithm (QTbMPA). The proposed QTbMPA selects both networks' best features and finally fuses using a serial-based approach. The fused feature set is passed to neural network classifiers for the final classification. The proposed framework tested on an augmented Figshare dataset and an improved accuracy of 99.80%, a sensitivity rate of 99.83%, a false negative rate of 17%, and a precision rate of 99.83% is obtained. Comparison and ablation study show the improvement in the accuracy of this work.

Deep Gaussian process for enhanced Bayesian optimization and its application in additive manufacturing

ABSTARCT Engineering design problems typically require optimizing a quality measure by finding the right combination of controllable input parameters. In Additive Manufacturing (AM), the output characteristics of the process can often be non-stationary functions of the process parameters. Bayesian Optimization (BO) is a methodology to optimize such “black-box” functions, i.e., the input–output relationship is unknown and expensive to compute. Optimization tasks involving “black-box” functions widely use BO with Gaussian Process (GP) regression surrogate model. Using GPs with standard kernels is insufficient for modeling non-stationary functions, while GPs with non-stationary kernels are typically over-parameterized. On the other hand, a Deep Gaussian Process (DGP) can overcome GPs’ shortcomings by considering a composition of multiple GPs. Inference in a DGP is challenging due to its structure resulting in a non-Gaussian posterior, and using DGP as a surrogate model for BO is not straightforward. Stochastic Imputation (SI)-based inference is promising in speed and accuracy for BO. This work proposes a bootstrap aggregation-based procedure to effectively utilize the SI-based inference for BO with a DGP surrogate model. The proposed BO algorithm DGP-SI-BO is faster and empirically better than the state-of-the-art BO method in optimizing non-stationary functions. Several analytical test functions and a case study in metal AM simulation demonstrate the applicability of the proposed method.

Deep-GD: Deep Learning based Automatic Garment Defect Detection and Type Classification

Garment defect detection has been successfully implemented in the quality quick response system for textile manufacturing automation. Defects in the production of textiles waste a lot of resources and reduce the quality of the finished goods. It is challenging to detect garment defects automatically because of the complexity of images and variety of patterns in textiles. This study presented a novel deep learning-based Garment defect detection framework named as Deep-GD model for sequentially identifying image defects in patterned garments and classify the defect types. Initially, the images are gathered from the HKBU database and bilateral filters are used in the pre-processing of images to remove distortions. A squeeze-and-excitation network (SE-net) combined with Random Decision Forest Classifier with Bayesian Optimization (RDF-BO) algorithm is used to detect and classify garment defects. By analyzing the differences among the original and reconstruction images, the unsupervised technique trains to rebuild the fabric pattern. The SE-net is used to identify certain fabric flaws in the garments of the pre-processed images. Then, the defects-related garments are processed using RDF-BO algorithm for classifying the garment defect from these regions. Finally, the proposed Deep-GD model is used for classifying defected fabric into 12 classes such as Defect free, soil stain, oil stain, Double end, Snarls, Miss, Horizontal stripes, Lumpy, Dye spot, Fall out, Hairiness, tiny hole. The proposed Deep-GD model achieves the overall classification accuracy of 97.16%, which is comparatively superior to the existing techniques.

Mapping model between lateral responses of bridges under random traffic loads based on data-driven approach

Abstract The lateral distribution characteristic of a bridge is one of the important features reflecting its in-service state. Conventional deterministic indicators often struggle to consider this time-varying feature. In this study, a data-driven approach is employed to establish the mapping model among responses at the different lateral positions using the eXtreme Gradient Boosting model optimized by the Bayesian optimization algorithm. The proposed method is validated based on both data from a bridge health monitoring system equipped on an actual bridge and data from numerical simulation. At different lateral positions of the actual bridge, the overall coefficient of determination (R2) of the strain response mapped by the model is above 0.991, and the overall root mean squared error (RMSE) of the strain response mapped by the model are below 1.159 με. The numerical simulation method is used to consider a variety of working conditions with different road surface roughness and different traffic densities. Under different working conditions, the overall R2 of the deflection response mapped by the model is still above 0.961, and the overall RMSE of the deflection response mapped by the model is below 0.249 mm. These indicate that the proposed model can consider time-varying mapping relationships among responses at different lateral positions, and has good accuracy and applicability.

The Capacity-to-load ratio optimization method considering collaborative planning of multi-voltage level power grids

The capacity-to-load ratio is an important indicator to guide power grid planning and has a decisive influence on the selection of substation capacities and the construction of power lines. A capacity-to-load ratio optimization method concerning collaborative planning of multi-voltage level power grids is proposed in this paper. This method aims to minimize the construction cost and takes into account the collaborative planning of substations and power lines of multiple voltage levels. By combining the Bayesian optimization algorithm (BOA) and genetic algorithms (GA), a BOA-GA solving algorithm for the planning model is proposed. This algorithm automatically generates planning schemes for multi-voltage level power grids, and obtains the optimal capacity-to-load ratio, as well as the optimal investment of the power grid construction. Case studies demonstrate that the proposed model can optimize the capacity-to-load ratio and automatically plan the network structure of multi-voltage level power grids. The construction cost is significantly reduced while various technical indicators are satisfied. Additionally, the study reveals that the relationship between the capacity-to-load ratio and the overall investment of multi-voltage level power grid construction is a quadratic function over a long-time scale. And the curve is a parabola going upwards.

Optimization of medium components for protein production by Escherichia coli with a high-throughput pipeline that uses a deep neural network

To optimize rapidly the medium for green fluorescent protein expression by Escherichia coli with an introduced plasmid, pRSET/emGFP, a single-cycle optimization pipeline was applied. The pipeline included a deep neural network (DNN) and mathematical optimization algorithms with simultaneous optimization of 18 medium components. To evaluate the DNN data sampling method, two methods, orthogonal array (OA) and Latin hypercube sampling (LHS), were used to design 64 initial media for each sampling method. The OA- and LHS-based data sampling resulted in green fluorescent protein fluorescence intensities of 0.088× 103-1.85× 104 and 3.30× 103-1.50× 104, respectively. Fifty DNN models were built using the OA and LHS datasets. Hold-out validation was performed using 15% test of OA and LHS data. Mean square errors of the DNN models were 0.015-0.64, indicating the estimation accuracies were sufficient. However, the sensitivities of components in the DNN models varied and were grouped into six major classes by the index of k-means clustering. A representative model was selected for each class. Mathematical optimization algorithms using Bayesian optimization and genetic algorithm were applied to the representative models, and representative optimized medium (OM) compositions were selected by k-means clustering from the proposed OMs. A total of 54 OMs were obtained from the OA and LHS datasets. In the validating cultivation, the best OMs of OA and LHS were 2.12-fold and 2.13-fold higher, respectively, than those of the learning data.