Probability Of Improvement Research Articles

Enhancing solar energy conversion efficiency: Thermophysical property predicting of MXene/Graphene hybrid nanofluids via bayesian-optimized artificial neural networks

Accurately predicting thermo-physical properties (TPPs) of MXene/graphene-based nanofluids is crucial for photovoltaic/thermal solar systems, driving focused research on developing precise TPP predictive models. This study presents optimized multi-layer perceptron neural network (MLPNN) models, leveraging Bayesian optimization to refine architectural and training hyperparameters, including hidden layers, neurons, activation functions, standardization, and regularization terms. A comparative analysis of Bayesian acquisition functions—the probability of improvement (POI), lower confidence bound (LCB), expected improvement (EI), expected improvement plus (EIP), expected improvement per second plus (EIPSP), and expected improvement per second (EIPS)—demonstrated that the POI-MLPNN achieves the most accurate results, as evidenced by the lowest MAPE of 1.0923 % and exceptional consistency with an R-value of 0.99811. The EI-MLPNN and EIP-MLPNN models recorded the same outputs. The EI/EIP-MLPNN (R = 0.99668) model excels in consistency over LCB-MLPNN (R = 0.99529) and EIPSP-MLPNN (R = 0.99667). The optimized models offer a reliable, cost-efficient alternate for experimental and computational TPP analyses. Leveraging insights from these models enables better control over nanofluid TPPs in solar systems, enhancing energy conversion efficiency.

Bayesian Optimization for Contamination Source Identification in Water Distribution Networks

In the wake of the terrorist attacks of 11 September 2001, extensive research efforts have been dedicated to the development of computational algorithms for identifying contamination sources in water distribution systems (WDSs). Previous studies have extensively relied on evolutionary optimization techniques, which require the simulation of numerous contamination scenarios in order to solve the inverse-modeling contamination source identification (CSI) problem. This study presents a novel framework for CSI in WDSs using Bayesian optimization (BO) techniques. By constructing an explicit acquisition function to balance exploration with exploitation, BO requires only a few evaluations of the objective function to converge to near-optimal solutions, enabling CSI in real-time. The presented framework couples BO with EPANET to reveal the most likely contaminant injection/intrusion scenarios by minimizing the error between simulated and measured concentrations at a given number of water quality monitoring locations. The framework was tested on two benchmark WDSs under different contamination injection scenarios, and the algorithm successfully revealed the characteristics of the contamination source(s), i.e., the location, pattern, and concentration, for all scenarios. A sensitivity analysis was conducted to evaluate the performance of the framework using various BO techniques, including two different surrogate models, Gaussian Processes (GPs) and Random Forest (RF), and three different acquisition functions, namely expected improvement (EI), probability of improvement (PI), and upper confident bound (UCB). The results revealed that BO with the RF surrogate model and UCB acquisition function produced the most efficient and reliable CSI performance.

A comprehensive framework based on Bayesian optimization and skip connections artificial neural networks to predict buildings energy performance

Artificial Neural Networks (ANNs) are a powerful modeling and prediction tool. However, in the energy and buildings fields, certain articles have been highlighted for reducing ANNs' ability to aid in building energy prediction. This result stems from the author's limited ability to regulate a large number of hyper-parameters. As a result, this paper provides a comprehensive study of ANN challenges and solutions in a single framework. The study's research aims may be stated as defining the necessary data preparation stages; selecting the appropriate ANN design for each study scenario with its hyper-parameters; and presenting the approach for training and evaluating ANNs to obtain the best performance. All of these concerns are presented in this study, compiled into a comprehensive framework, and implemented on two different types of real-life data: 1) CBECS data for US commercial buildings, and 2) ASHRAE data for different buildings around the world. The framework optimization loop mainly relies on the Bayesian optimization technique, which employs Gaussian Processes (GP) as a surrogate function; a combination of two concepts (i.e., Expected Improvement (EI) and Probability Improvement (PI)) as an acquisition function; hyper-parameters as a dimension space optimization; and adjusted R2 value as a target optimization score to be improved. Various skip connection ANN architectures are applied to demonstrate the ability to modify ANNs' limitations in the energy and building fields, assuring excellent accuracy and reliability in forecasting building energy performance, with adjusted R2 of more than 0.93 for both data sets.

Analysis of the Effectiveness of Metaheuristic Methods on Bayesian Optimization in the Classification of Visual Field Defects.

Bayesian optimization (BO) is commonly used to optimize the hyperparameters of transfer learning models to improve the model's performance significantly. In BO, the acquisition functions direct the hyperparameter space exploration during the optimization. However, the computational cost of evaluating the acquisition function and updating the surrogate model can become prohibitively expensive due to increasing dimensionality, making it more challenging to achieve the global optimum, particularly in image classification tasks. Therefore, this study investigates and analyses the effect of incorporating metaheuristic methods into BO to improve the performance of acquisition functions in transfer learning. By incorporating four different metaheuristic methods, namely Particle Swarm Optimization (PSO), Artificial Bee Colony (ABC) Optimization, Harris Hawks Optimization, and Sailfish Optimization (SFO), the performance of acquisition function, Expected Improvement (EI), was observed in the VGGNet models for visual field defect multi-class classification. Other than EI, comparative observations were also conducted using different acquisition functions, such as Probability Improvement (PI), Upper Confidence Bound (UCB), and Lower Confidence Bound (LCB). The analysis demonstrates that SFO significantly enhanced BO optimization by increasing mean accuracy by 9.6% for VGG-16 and 27.54% for VGG-19. As a result, the best validation accuracy obtained for VGG-16 and VGG-19 is 98.6% and 98.34%, respectively.

AROPS: A Framework of Automated Reaction Optimization with Parallelized Scheduling.

With the development of automated experimental platforms and optimization algorithms, chemists can easily optimize chemical reactions in an automated and high-throughput fashion. However, the modules in existing automated experimental platforms are operated in a linear fashion without orchestrating with the optimization algorithm, thus leaving room for further efficiency improvement. Here, we introduced a framework of automated reaction optimization with parallelized scheduling (AROPS) to realize the integration of the optimization algorithm and module scheduling. AROPS relies on a customized Bayesian optimizer to solve multi-reactor/analyzer reaction optimization problems with three different scheduling modes to arrange tasks for various experimental modules. In addition, a mechanism based on probability of improvement (PI) for discarding unpromising ongoing experiments was developed to facilitate freeing up valuable experimental resources in parallelized optimization. We tested the performance of AROPS using a hardware emulator on three representative benchmark reactions encountered in organic synthesis, illustrating that AROPS can trade off optimization time and cost according to the chemists' preference.

A parallel technique for multi-objective Bayesian global optimization: Using a batch selection of probability of improvement

Bayesian global optimization (BGO) is an efficient surrogate-assisted technique for problems involving expensive evaluations. A parallel technique can be used to parallelly evaluate the true-expensive objective functions in one iteration to boost the execution time. An effective and straightforward approach is to design an acquisition function that can evaluate the performance of a bath of multiple solutions, instead of a single point/solution, in one iteration. This paper proposes five alternatives of Probability of Improvement (PoI) with multiple points in a batch (q-PoI) for multi-objective Bayesian global optimization (MOBGO), taking the covariance among multiple points into account. Both exact computational formulas and the Monte Carlo approximation algorithms for all proposed q-PoIs are provided. Based on the distribution of the multiple points relevant to the Pareto-front, the position-dependent behavior of the five q-PoIs is investigated. Moreover, the five q-PoIs are compared with the other nine state-of-the-art and recently proposed batch MOBGO algorithms on twenty bio-objective benchmarks. The empirical experiments on different variety of benchmarks are conducted to demonstrate the effectiveness of two greedy q-PoIs ( q-PoI best and q-PoI all ) on low-dimensional problems and the effectiveness of two explorative q-PoIs ( q-PoI one and q-PoI worst ) on high-dimensional problems with difficult-to-approximate Pareto front boundaries.

A novel Bayesian approach for multi-objective stochastic simulation optimization

Multi-objective stochastic simulation optimization plays an important role in designing complex engineering systems. To identify optimal solutions via simulations, Bayesian optimization, which utilizes metamodels and an acquisition function to determine the next design point, has been popular in machine learning. However, studies on Bayesian optimization of multi-objective stochastic simulation are limited and can give undesired performances in terms of convergence and diversity metrics. Moreover, direct extension of Bayesian optimization algorithms of deterministic responses is unrealistic as stochastic responses are observed with heterogeneous noise. To handle these issues, we propose a novel framework for Bayesian optimization of multiple stochastic responses. Stochastic kriging metamodels are constructed independently for multiple stochastic responses considering heterogeneous simulation noise. To remove undesired effects of random simulation noise on comparing solutions, quantiles of stochastic kriging metamodels are employed to define the Pareto dominance. Acquisition functions are proposed by integrating the hypervolume-based and probability-of-improvement (PoI)-based acquisition functions from deterministic multi-objective optimization with typical acquisition functions from univariate stochastic response optimization. This not only enhances the convergence and diversity of Pareto-optimal solutions but also properly handles heterogeneous noise. Exact calculation formulas of the proposed algorithms are further developed. Typical numerical cases and two engineering examples (a rocket injector design example and a real inventory design scenario) demonstrate that the proposed algorithms give better Pareto-optimal solutions of which the proximity to true Pareto fronts and the diversity are improved.

A surrogate-based parallel optimization of analog circuits using multi-acquisition functions

Surrogate-based optimization has been widely used in analog circuit sizing. Surrogate-based optimization efficiency in a parallel computing environment has become an emerging problem. In this paper, a surrogate-based parallel optimization of analog circuits using multi-acquisition functions (SPMA) is proposed. The Surrogate Model-Aware Search Mechanism (SMAS) is used as the basis. Under the SMAS framework, modeling and search are concentrated in promising areas. The probability of improvement (PI), expected improvement (EI), and lower confidence bound (LCB) combined in the multi-acquisition function each balance exploration and exploitation in its own way. It is necessary to configure enough training data points for the local model constructed in the promising area to improve the exploration ability of the multi-acquisition functions. Multiple mutation operations are used to provide more promising candidate designs and increase the probability that the surrogate model can prescreen to the optimal solutions. SPMA combines the advantages of multi-acquisition functions and multiple mutation operations, and has been verified by a two-stage amplifier and a three-stage amplifier, as well as three well-known benchmark functions. The optimization results show that SPMA is superior to state-of-the-art methods in terms of efficiency and effectiveness.

Comparison of parallel infill sampling criteria based on Kriging surrogate model

One of the key issues that affect the optimization effect of the efficient global optimization (EGO) algorithm is to determine the infill sampling criterion. Therefore, this paper compares the common efficient parallel infill sampling criterion. In addition, the pseudo-expected improvement (EI) criterion is introduced to minimizing the predicted (MP) criterion and the probability of improvement (PI) criterion, which helps to improve the problem of MP criterion that is easy to fall into local optimum. An adaptive distance function is proposed, which is used to avoid the concentration problem of update points and also improves the global search ability of the infill sampling criterion. Seven test problems were used to evaluate these criteria to verify the effectiveness of these methods. The results show that the pseudo method is also applicable to PI and MP criteria. The DMP and PEI criteria are the most efficient and robust. The actual engineering optimization problems can more directly show the effects of these methods. So these criteria are applied to the inverse design of RAE2822 airfoil. The results show the criterion including the MP has higher optimization efficiency.

Hybrid XGboost model with various Bayesian hyperparameter optimization algorithms for flood hazard susceptibility modeling

The purpose of this investigation is to develop an optimal model to flood susceptibility mapping in the Kan watershed, Tehran, Iran. Therefore, in this study, three Bayesian optimization hyper-parameter algorithms including Upper confidence bound (UCB), Probability of improvement (PI) and Expected improvement (EI) in order to Extreme Gradient Boosting (XGB) machine learning model optimization and Extreme randomize tree (ERT) model for modeling flood hazard were used. In order to perform flood susceptibility mapping, 118 historic flood locations were identified and analyzed using 17 geo-environmental explanatory variables to predict flooding susceptibility. Flood locations data were divided into 70% for training and 30% for testing of models developed. The receiver operating characteristic (ROC) curve parameters were used to evaluate the performance of the models. The evaluation results based on the criterion area under curve (AUC) in the testing stage showed that the ERT and XGB models have efficiencies of 91.37% and 91.95%, respectively. The evaluation of the efficiency of Bayesian hyperparameters optimization methods on the XGB model also showed that these methods increase the efficiency of the XGB model, so that the model efficiency using these methods EI-XGB, POI-XGB and UCB-XGB based on the AUC in the testing stage were 95.89%, 96.87% and 96.38%, respectively. The results of the relative importance of the five models shows that the variables of elevation and distance from the river are the significant compared to other variables in predicting flood hazard in the Kan watershed.

An Efficient Batch-Constrained Bayesian Optimization Approach for Analog Circuit Synthesis via Multiobjective Acquisition Ensemble

Bayesian optimization is a promising methodology for analog circuit synthesis. However, the sequential nature of the Bayesian optimization framework significantly limits its ability to fully utilize real-world computational resources. In this paper, we propose an efficient parallelizable Bayesian optimization algorithm via Multi-objective ACquisition function Ensemble (MACE) to further accelerate the optimization procedure. By sampling query points from the Pareto front of the probability of improvement (PI), expected improvement (EI) and lower confidence bound (LCB), we combine the benefits of state-of-the-art acquisition functions to achieve a delicate tradeoff between exploration and exploitation for the unconstrained optimization problem. Based on this batch design, we further adjust the algorithm for the constrained optimization problem. By dividing the optimization procedure into two stages and first focusing on finding an initial feasible point, we manage to gain more information about the valid region and can better avoid sampling around the infeasible area. After achieving the first feasible point, we favor the feasible region by adopting a specially designed penalization term to the acquisition function ensemble. The experimental results quantitatively demonstrate that our proposed algorithm can reduce the overall simulation time by up to 74 times compared to differential evolution (DE) for the unconstrained optimization problem when the batch size is 15. For the constrained optimization problem, our proposed algorithm can speed up the optimization process by up to 15 times compared to the weighted expected improvement based Bayesian optimization (WEIBO) approach, when the batch size is 15.

Objective functions from Bayesian optimization to locate additional drillholes

The key available information to choose new locations for drilling are the estimated ore grade values and the corresponding uncertainties at the tentative locations. These pieces of information are combined to generate a single objective function. The mathematical form of the objective function should reflect the effect of these values and their relative importance. Traditional objective function use multiplication of these parameters by different powering values. In this study, we develop two novel objective functions from the Bayesian optimization: the probability of improvement (PI), and the expected improvement (EI). These two objective functions seek new drillholes while considering the effect of the used value and their relative importance. Therefore, they can provide a trade-off between exploration and exploitation. All the objective functions have adjustable parameters. These parameters are typically tuned using expert knowledge or heuristic rules. Here, a statistical method based on cross-validation is proposed to adjust the parameters of the traditional and novel objective functions. The performance of the novel objective functions is validated against other ones using a distance based ranking method, in a phosphate deposit. The obtained results demonstrate the robustness of the EI and PI, the newly introduced objective functions from the Bayesian optimization framework.

Efficient Aerodynamic Optimization of Propeller using Hierarchical Kriging Models

The number of aerodynamic analysis in the design optimization of propeller has been reduced significantly by using the surrogate model, such as kriging. In this paper, a more efficient aerodynamic design optimization method of propeller is proposed by using the HK (hierarchical kriging) model. The high-fidelity model is defined as a RANS (Reynolds- Averaged Navier-Stokes) simulation for propeller and the low-fidelity model is defined as the results by blade-element/vortex theory. Initial samples are selected for two levels of fidelity via LHS (Latin Hypercube Sampling). The aerodynamic performance of varying fidelity is used to build variable-fidelity surrogate models for functions of objective (e.g. thrust) and constraint (e.g. shaft power). Infill sampling criteria, including MSP (minimizing of surrogate prediction), EI (expected improvement), PI (probability of improvement), LCB (lower-confidence bounding) and MSE (mean-squared error) are used to obtain new samples, and the surrogate models are repetitively updated until a global optimum is found. High-altitude propellers are optimized by the kriging model and hierarchical kriging model, respectively. Compared to kriging model, the number of RANS solving by the hierarchical kriging model is reduced 37.5%, and the optimization time is reduced 24.3%. The results have shown that the proposed design method for propeller can significantly improve the optimization efficiency.

A Fast-Converging Ensemble Infilling Approach Balancing Global Exploration and Local Exploitation: The Go-Inspired Hybrid Infilling Strategy

AbstractInfilling strategies have been proposed for decades and are widely used in engineering problems. It is still challenging to achieve an effective trade-off between global exploration and local exploitation. In this paper, a novel decision-making infilling strategy named the Go-inspired hybrid infilling (Go-HI) strategy is proposed. The Go-HI strategy combines multiple individual infilling strategies, such as the mean square error (MSE), expected improvement (EI), and probability of improvement (PoI) strategies. The Go-HI strategy consists of two major parts. In the first part, a tree-like structure consisting of several subtrees is built. In the second part, the decision value for each subtree is calculated using a cross-validation (CV)-based criterion. Key factors that significantly influence the performance of the Go-HI strategy, such as the number of component infilling strategies and the tree depth, are explored. Go-HI strategies with different component strategies and tree depths are investigated and also compared with four baseline adaptive sampling strategies through three numerical functions and one engineering case. Results show that the number of component infilling strategies exerts a larger influence on the global and local performance than the tree depth; the Go-HI strategy with two component strategies performs better than the ones with three; the Go-HI strategy always outperforms the three component infilling strategies and the other four benchmark strategies in global performance and robustness and saves much computational cost.

Efficient computation of expected hypervolume improvement using box decomposition algorithms

In the field of multi-objective optimization algorithms, multi-objective Bayesian Global Optimization (MOBGO) is an important branch, in addition to evolutionary multi-objective optimization algorithms. MOBGO utilizes Gaussian Process models learned from previous objective function evaluations to decide the next evaluation site by maximizing or minimizing an infill criterion. A commonly used criterion in MOBGO is the Expected Hypervolume Improvement (EHVI), which shows a good performance on a wide range of problems, with respect to exploration and exploitation. However, so far, it has been a challenge to calculate exact EHVI values efficiently. This paper proposes an efficient algorithm for the exact calculation of the EHVI for in a generic case. This efficient algorithm is based on partitioning the integration volume into a set of axis-parallel slices. Theoretically, the upper bound time complexities can be improved from previously O (n^2) and O(n^3), for two- and three-objective problems respectively, to varTheta (nlog n), which is asymptotically optimal. This article generalizes the scheme in higher dimensional cases by utilizing a new hyperbox decomposition technique, which is proposed by Dächert et al. (Eur J Oper Res 260(3):841–855, 2017). It also utilizes a generalization of the multilayered integration scheme that scales linearly in the number of hyperboxes of the decomposition. The speed comparison shows that the proposed algorithm in this paper significantly reduces computation time. Finally, this decomposition technique is applied in the calculation of the Probability of Improvement (PoI).

Enhanced surrogate assisted framework for constrained global optimization of expensive black-box functions

An enhanced surrogate assisted framework, based on Probability of Improvement (PI) method, is proposed in this paper. We made some modifications to the original PI approach to enhance the performance of the modeling and optimization framework, leading to fewer rigorous simulations to find the optimal solution without loss of accuracy. We also extended the algorithm for handling general constraints using a fully probabilistic approach. The behavior of the proposed framework was investigated through a set of 9 Unconstrained Test Functions (UTF), 7 Constrained Optimization Problems (COP) and 3 Chemical Engineering Problems (CEP). The numerical results indicate that a lower number of rigorous model simulations were needed for optimizing UTF compared to the classic PI method and that the proposed framework was capable of achieving sustained near optimal solutions for COP and CEP. These results indicate that the proposed framework is suitable for solving computationally expensive constrained black-box optimization problems.