Multi-objective Bayesian Optimization Research Articles

A Bayesian Multiobjective Approach Based on GMPPT for PV Arrays

This paper considers the problem of tracking the global maximum power point (GMPP) in partially shaded conditions (PSCs) as a multiobjective optimization problem and solves it using a novel multiobjective optimization algorithm on the basis of Bayesian optimization formulation. Bayesian optimization is a metamodel-based global optimization method that is able to balance exploration and exploitation. The Pareto solutions are obtained by using a multiobjective Bayesian optimization algorithm. Also, a new acquisition function is proposed to improve the diversity and convergence of the Pareto solutions. Two objective functions are introduced to remove the large tracking errors and oscillations of the operating point around the GMPP. The suggested method is implemented online for GMPP tracking so that the suggested method monitors any change in environmental conditions and generates the optimal duty cycle for the DC-DC converter for the GMPP tracking (GMPPT) by the PV array. Several multipeak PSC scenarios are implemented and simulated to show efficiency of the suggested approach. The MATLAB/SIMULINK is employed to implement a photovoltaic (PV) system comprising a PV array, a boost converter, and the proposed multiobjective Bayesian optimization algorithm (MOBOA). The simulation results show a very satisfactory performance of the MOBOA in terms of transient state and steady-state oscillations and tracking speed.

Bayesian Optimization of Computer-Proposed Multistep Synthetic Routes on an Automated Robotic Flow Platform.

Computer-aided synthesis planning (CASP) tools can propose retrosynthetic pathways and forward reaction conditions for the synthesis of organic compounds, but the limited availability of context-specific data currently necessitates experimental development to fully specify process details. We plan and optimize a CASP-proposed and human-refined multistep synthesis route toward an exemplary small molecule, sonidegib, on a modular, robotic flow synthesis platform with integrated process analytical technology (PAT) for data-rich experimentation. Human insights address catalyst deactivation and improve yield by strategic choices of order of addition. Multi-objective Bayesian optimization identifies optimal values for categorical and continuous process variables in the multistep route involving 3 reactions (including heterogeneous hydrogenation) and 1 separation. The platform’s modularity, robotic reconfigurability, and flexibility for convergent synthesis are shown to be essential for allowing variation of downstream residence time in multistep flow processes and controlling the order of addition to minimize undesired reactivity. Overall, the work demonstrates how automation, machine learning, and robotics enhance manual experimentation through assistance with idea generation, experimental design, execution, and optimization.

Development of Aluminum Scandium Nitride Molecular Dynamics Force Fields with Scalable Multi-Objective Bayesian Optimization

Development of Aluminum Scandium Nitride Molecular Dynamics Force Fields with Scalable Multi-Objective Bayesian Optimization

Multi-Objective Bayesian Optimization of Lithium-Ion Battery Cells for Electric Vehicle Operational Scenarios

The development of lithium-ion batteries (LIBs) based on current practice allows an energy density increase estimated at 10% per year. However, the required power for portable electronic devices is predicted to increase at a much faster rate, namely 20% per year. Similarly, the global electric vehicle battery capacity is expected to increase from around 170 GWh per year today to 1.5 TWh per year in 2030—this is an increase of 125% per year. Without a breakthrough in battery design technology, it will be difficult to keep up with their increasing energy demand. The objective of this investigation is to develop a design methodology to accelerate the LIB development through the integration of electro-chemical numerical simulations and machine learning algorithms. In this work, the Gaussian process (GP) regression model is used as a fast approximation of numerical simulation (conducted using Simcenter Battery Design Studio®). The GP regression models are systematically updated through a multi-objective Bayesian optimization algorithm, which enables the exploration of innovative designs as well as the determination of optimal configurations. The results reported in this work include optimal thickness and porosities of LIB electrodes for several practical charge–discharge scenarios which maximize energy density and minimize capacity fade.

A Nested Weighted Tchebycheff Multi-Objective Bayesian Optimization Approach for Flexibility of Unknown Utopia Estimation in Expensive Black-Box Design Problems

Abstract We propose a nested weighted Tchebycheff Multi-objective Bayesian optimization (WTB MOBO) framework where we built a regression model selection procedure from the ensemble of models, toward better estimation of the uncertain parameters (utopia) of the weighted Tchebycheff expensive black-box multi-objective function. In our previous work, a weighted Tchebycheff MOBO approach has been demonstrated which attempts to estimate the model parameters (utopia) in formulating the acquisition function of the weighted Tchebycheff multi-objective black-box functions, through calibration using an a priori selected regression model. However, the existing MOBO model lacks flexibility in selecting the appropriate regression models given the guided sampled data and, therefore, can under-fit or over-fit as the iterations of the MOBO progress. This ultimately can reduce the overall MOBO performance. As, in general, it is too complex to a priori guarantee a best model, this motivates us to consider a portfolio of different families (simple-to-complex) of predictive models that have been fitted with current training data guided by the WTB MOBO, and the best model is selected following a user-defined prediction root-mean-square error-based approach. The proposed approach is implemented in optimizing a thin tube design under constant loading of temperature and pressure, minimizing the risk of creep-fatigue failure and design cost. Finally, the nested WTB MOBO model performance is compared with different MOBO frameworks with respect to accuracy in parameter estimation, Pareto-optimal solutions, and function evaluation cost. This approach is generalized enough to consider different families of predictive models in the portfolio for best model selection, where the overall design architecture allows for solving any high-dimensional (multiple functions) complex black-box problems and can be extended to any other global criterion multi-objective optimization methods where prior knowledge of utopia is required.

Vortex-model-based Multi-objective Optimization of Winglets for Wind Turbines using Machine Learning

Different Design Driving Load constraints (DDLs), are explored in this work to determine under which constraints and conditions a winglet can have an added value to the wind turbine blade design. Multi-objective Bayesian optimization is used to maximize the rotor’s power production while minimizing the flapwise DDLs. Surrogate models, created using machine learning techniques such as Gaussian Processes and Bayesian Neural Networks, are used in combination with an acquisition function, to determine what designs should be evaluated by the lifting line model AWSM, with the goal to obtain designs that lie on the Pareto front of two or more objectives. The recent Bayesian Neural Networks as surrogate model were able to find the Pareto-front most effectively in this work. Furthermore, the results show that different DDL constraints led to different winglet designs, with noticeable differences between upwind and downwind winglet designs. Winglet designs were found to be able to increase power without increasing the thrust, root flapwise bending moment and flapwise bending moment at radial locations on the blade. A noticeable increase in power was found when introducing sweep to the winglet design.

Discovery of Pb-free hybrid organic\u2013inorganic 2D perovskites using a stepwise optimization strategy

The current status of 2D organic–inorganic hybrid perovskites for use in photovoltaic (PV) and light-emitting diode (LED) applications lags far behind their 3D counterparts. Here, we propose a computational strategy for discovering novel perovskites with as few computing resources as possible. A tandem optimization algorithm consisting of an elitism-reinforced nondominated sorting genetic algorithm (NSGA-II) and a multiobjective Bayesian optimization (MOBO) algorithm was used for density functional theory (DFT) calculations. The DFT-calculated band gap and effective mass were taken as objective functions to be optimized, and the constituent molecules and elements of a Ruddlesden–Popper (RP) structure (n = 2) were taken as decision variables. Fourteen previously unknown RP perovskite candidates for PV and LED applications were discovered as a result of the NSGA-II/MOBO algorithm. Thereafter, more accurate DFT calculations based on the HSE06 exchange correlation functional and ab initio molecular dynamics (AIMD) were conducted for the discovered 2D perovskites to ensure their validity.

Multiobjective Tree-Structured Parzen Estimator

Practitioners often encounter challenging real-world problems that involve a simultaneous optimization of multiple objectives in a complex search space. To address these problems, we propose a practical multiobjective Bayesian optimization algorithm. It is an extension of the widely used Tree-structured Parzen Estimator (TPE) algorithm, called Multiobjective Tree-structured Parzen Estimator (MOTPE). We demonstrate that MOTPE approximates the Pareto fronts of a variety of benchmark problems and a convolutional neural network design problem better than existing methods through the numerical results. We also investigate how the configuration of MOTPE affects the behavior and the performance of the method and the effectiveness of asynchronous parallelization of the method based on the empirical results.

A hybrid of Bayesian-based global search with Hooke–Jeeves local refinement for multi-objective optimization problems

The proposed multi-objective optimization algorithm hybridizes random global search with a local refinement algorithm. The global search algorithm mimics the Bayesian multi-objective optimization algorithm. The site of current computation of the objective functions by the proposed algorithm is selected by randomized simulation of the bi-objective selection by the Bayesian-based algorithm. The advantage of the new algorithm is that it avoids the inner complexity of Bayesian algorithms. A version of the Hooke–Jeeves algorithm is adapted for the local refinement of the approximation of the Pareto front. The developed hybrid algorithm is tested under conditions previously applied to test other Bayesian algorithms so that performance could be compared. Other experiments were performed to assess the efficiency of the proposed algorithm under conditions where the previous versions of Bayesian algorithms were not appropriate because of the number of objectives and/or dimensionality of the decision space.

Comparison of conceptually different multi-objective Bayesian optimization methods for material design problems

For real-world applications, material properties must usually meet multiple requirements, and researchers often spend considerable time designing such materials by trial and error. Multi-objective Bayesian optimization (MOBO) constitutes a promising data-driven solution to accelerate such design problems. As things stand, conceptually different MOBO methods exist for material design problems, such as scalarization- and hypervolume-based methods. However, no standard approach exists to compare how these methods perform and the appropriate choice of MOBO method in each case remains unclear. Herein, a benchmark protocol to compare how conceptually different MOBO methods perform was introduced, based on which the performances of MOBO methods were comprehensively compared using multiple design problems and performance metrics. The benchmark results showed that there was no method that performed best for all combinations of design problems and performance metrics. Moreover, when multiple MOBO methods were compared, the opportunity cost of using each method emerged and it was shown that an inappropriately chosen method can hinder MOBO efficiency. The benchmark results shown here highlight the importance of choosing the right MOBO method and provide guidelines for how this can be done.

Performance-oriented model learning for control via multi-objective Bayesian optimization

The closed-loop performance of model-based controllers, such as model predictive control, largely depends on the quality of their underlying model of system dynamics. Inspired by the notion of identification for control, this paper presents a strategy for performance-oriented, data-driven model adaptation for control. The fundamental idea is to mitigate plant-model mismatch via improving the model’s control-oriented predictive quality for optimizing closed-loop performance measures of interest, as opposed to enhancing its general predictive accuracy. To this end, we leverage a composite model structure that consists of a prior system model (physics-based or data-driven) and a data-driven model that can be efficiently adapted in a performance-oriented manner towards maximization of the closed-loop control performance. To solve the performance-oriented model adaptation problem, we use multi-objective Bayesian optimization (MOBO) that can directly handle black-box and expensive-to-evaluate functions that are computed from noisy observations of closed-loop performance measures. The MOBO approach is demonstrated on a benchmark bioreactor case study. Simulation results indicate that, given a fixed budget of process runs, performance-oriented model adaptation can yield control-oriented models that result in a significant improvement in closed-loop performance, compared to model identification using closed-loop data, while systematically accounting for multiple performance measures. The proposed approach can be particularly useful for multi-objective model learning and auto-tuning of model-based controllers for processes with finite-time control objectives, where each process run is associated with a high monetary value.

Machine Learning Enabled Design Automation and Multi-Objective Optimization for Electric Transportation Power Systems

This article presents an automated design and optimization framework for electric transportation power systems (ETPS) enabled by machine learning (ML). The use of physical models, simulations, and optimization methods can greatly aid the engineering design process. However, when considering the optimal co-design of multiple interdependent subsystems that span multiple physical domains, such model-based simulations can be computationally expensive, and traditional metaheuristic optimization methods can be unreliable. Bayesian optimization (BO), an ML framework, paves one feasible pathway to realize an efficient design process practically. However, current state-of-the-art BO algorithms are non-compatible or perform poorly when applied to system-level ETPS design with multiple objectives and constraints. This article proposes a novel BO algorithm referred to as max-value entropy search for multiobjective optimization with constraints (MESMOC) to solve multiobjective optimization (MOO) problems with black-box constraints that can only be evaluated through design simulations. After a full presentation of the algorithm, MESMOC is applied to a realistic ETPS design case using a heavy-duty electric vertical-takeoff-landing (eVTOL) urban aerial vehicle (UAV) power system. Two MOO experimental trials show a drastic reduction in the number of design simulations to discover a high-quality Pareto front. In Trial 1, MESMOC uncovered the entire Pareto front while only requiring to explore ~4% of the design space. With expanded design parameters and larger design space in Trial 2, a near complete but high-quality Pareto front was uncovered. Both trials compared MESMOC to the popular genetic algorithm NSGA-II and another BO algorithm predictive entropy search for multi-objective Bayesian optimization with constraints (PESMOC), showing superior performance.

Longitudinal bending of corrugated sandwich panels with cores of various shapes

The mechanical response of aluminium corrugated sandwich panels with cores of different shapes subjected to longitudinal three-point bending is investigated experimentally and numerically in this paper. The shapes of the core are sinusoid, triangle, trapezoid, and rectangle, respectively. The experimental and simulated results match well with each other. Subsequently, numerical models with specific boundary conditions are developed to simulate the corrugate sandwich panels with an infinite number of core cells. Parametric studies are conducted using these numerical models. The effects of core web thickness, core length, and corrugation angle are investigated numerically. Two deformation modes (Mode I and Mode II) are observed. The panels that deform in Mode I have a larger indentation area and minor global bending. Mode I tends to happen in panels with weaker core webs. Finally, the multiobjective Bayesian optimization method based on lower confidence bound (LCB) criterion is adopted to optimize the geometry of trapezoidal corrugated sandwich panels to improve the specific energy absorption (SEA) and reduce the intrusion depth during the bending.

A Broadband Asymmetric Doherty Power Amplifier Design Based on Multiobjective Bayesian Optimization: Theoretical and Experimental Validation

In this paper, a new multi-objective Bayesian optimization (BO) algorithm is applied to design a broadband gallium nitride (GaN) based Doherty power amplifier (DPA). The optimization process can be automatically implemented by combing the proposed methodology with a commercial simulation software. The performance of the DPA optimized by the proposed method is compared with those obtained with the initial designed DPA, the DPA optimized by existing BO method, and DPA optimized by the optimizers built-in using the commercial software advanced design system (ADS). The comparison results reveal that the DPA designed with the proposed method can achieve better performance with less optimization time than other optimization methods. The measured results show that the optimized DPA with the proposed method achieves a 9-dB back-off efficiency of 44.6%-65% and a saturated efficiency of 60.5%-78% from 2.0 GHz to 2.6 GHz, with saturated output power varying from 43.8 dBm to 45.2 dBm.

Multi-objective Bayesian topology optimization of a lattice-structured heat sink in natural convection

Additive manufacturing (AM) has an affinity with topology optimization to think of various designs with complex structures. Hence, this paper aims to optimize the design of a lattice-structured heat sink, which can be manufactured by AM. The design objectives are to maximize the thermal performance of convective heat transfer in natural convection simulated by computational fluid dynamics (CFD) and to minimize the material cost required for AM process at the same time. The lattice structure is represented as a node/edge system via graph theory with a moderate number of design variables. Bayesian optimization, which employs the non-dominated sorting genetic algorithm II and the Kriging surrogate model, is conducted to search for better designs with the minimum CFD cost. The present topology optimization successfully finds better lattice-structured heat sink designs than a reference fin-structured design regarding thermal performance and material cost. Also, several optimized lattice-structured designs outperform reference pin-fin-structured designs regarding thermal performance though the pin-fin structure is still advantageous for a material cost-oriented design. This paper also discusses the flow mechanism observed in the heat sink to explain how the optimized heat sink structure satisfies the competing design objectives simultaneously.

A Multi-Objective Bayesian Optimization Environment for Systematic Design of Numerical Schemes for Compressible Flow

A Multi-Objective Bayesian Optimization Environment for Systematic Design of Numerical Schemes for Compressible Flow

Reduced-order kinetic Monte Carlo model to simulate water diffusion in biodegradable polymers

Water is a driving factor in the degradation process of biodegradable polymers. However, water diffusion is typically not incorporated in the kinetic models of biodegradation such as kinetic Monte Carlo (kMC), because water diffusion events occur at a much shorter time scale than the hydrolysis reactions. As such, there is a need to improve the computational efficiency of water diffusion in the kMC models of diffusion processes in biodegradation. In this work, a new dimensionality reduction scheme for kMC diffusion models is developed to significantly reduce the computation time, where a two-dimensional kMC diffusion model for porous microstructures is reduced to one-dimensional ones. The dimensionality reduction is accomplished by calibrating the model parameters with multi-objective Bayesian optimization. The reduced-order diffusion model shows a 675-fold faster computation compared to the original two-dimensional model.

Multi-objective Bayesian optimization of ferroelectric materials with interfacial control for memory and energy storage applications

Optimization of materials’ performance for specific applications often requires balancing multiple aspects of materials’ functionality. Even for the cases where a generative physical model of material behavior is known and reliable, this often requires search over multidimensional function space to identify low-dimensional manifold corresponding to the required Pareto front. Here, we introduce the multi-objective Bayesian optimization (MOBO) workflow for the ferroelectric/antiferroelectric performance optimization for memory and energy storage applications based on the numerical solution of the Ginzburg–Landau equation with electrochemical or semiconducting boundary conditions. MOBO is a low computational cost optimization tool for expensive multi-objective functions, where we update posterior surrogate Gaussian process models from prior evaluations and then select future evaluations from maximizing an acquisition function. Using the parameters for a prototype bulk antiferroelectric (PbZrO3), we first develop a physics-driven decision tree of target functions from the loop structures. We further develop a physics-driven MOBO architecture to explore multidimensional parameter space and build Pareto-frontiers by maximizing two target functions jointly—energy storage and loss. This approach allows for rapid initial materials and device parameter selection for a given application and can be further expanded toward the active experiment setting. The associated notebooks provide both the tutorial on MOBO and allow us to reproduce the reported analyses and apply them to other systems (https://github.com/arpanbiswas52/MOBO_AFI_Supplements).

Output Space Entropy Search Framework for Multi-Objective Bayesian Optimization

We consider the problem of black-box multi-objective optimization (MOO) using expensive function evaluations (also referred to as experiments), where the goal is to approximate the true Pareto set of solutions by minimizing the total resource cost of experiments. For example, in hardware design optimization, we need to find the designs that trade-off performance, energy, and area overhead using expensive computational simulations. The key challenge is to select the sequence of experiments to uncover high-quality solutions using minimal resources. In this paper, we propose a general framework for solving MOO problems based on the principle of output space entropy (OSE) search: select the experiment that maximizes the information gained per unit resource cost about the true Pareto front. We appropriately instantiate the principle of OSE search to derive efficient algorithms for the following four MOO problem settings: 1) The most basic single-fidelity setting, where experiments are expensive and accurate; 2) Handling black-box constraints which cannot be evaluated without performing experiments; 3) The discrete multi-fidelity setting, where experiments can vary in the amount of resources consumed and their evaluation accuracy; and 4) The continuous-fidelity setting, where continuous function approximations result in a huge space of experiments. Experiments on diverse synthetic and real-world benchmarks show that our OSE search based algorithms improve over state-of-the-art methods in terms of both computational-efficiency and accuracy of MOO solutions.

ERGO: A New Robust Design Optimization Technique Combining Multi-Objective Bayesian Optimization With Analytical Uncertainty Quantification

Abstract In this work, robust design optimization (RDO) is treated, motivated by the increasing desire to account for variability in the design phase. The problem is formulated in a multi-objective setting with the objective of simultaneously minimizing the mean of the objective and its variance due to variability of design variables and/or parameters. This allows the designer to choose its robustness level without the need to repeat the optimization as typically encountered when formulated as a single objective. To account for the computational cost that is often encountered in RDO problems, the problem is fitted in a Bayesian optimization framework. The use of surrogate modeling techniques to efficiently solve problems under uncertainty has effectively found its way in the optimization community leading to surrogate-assisted optimization-under-uncertainty schemes. The Gaussian processes, the surrogates on which Bayesian optimization builds, are often considered cheap-to-sample black-boxes and are sampled to obtain the desired quantities of interest. However, since the analytical formulation of these surrogates is known, an analytical treatment of the problem is available. To obtain the quantities of interest without sampling an analytical uncertainty, propagation through the surrogate is presented. The multi-objective Bayesian optimization framework and the analytical uncertainty quantification are linked together through the formulation of the robust expected improvement, obtaining the novel efficient robust global optimization scheme. The method is tested on a series of test cases to examine its behavior for varying difficulties and validated on an aerodynamic test function which proves the effectiveness of the novel scheme.