Multi-fidelity Bayesian Optimization Research Articles

Parametric shape optimization of pin fin arrays using a multi-fidelity surrogate model based Bayesian method

The current work aims to develop a multi-fidelity Bayesian optimization methodology to reduce computation time in CFD based design optimization of an internal cooling channel of a gas turbine blade. To build energy efficient gas turbine engines, it is important to increase the turbine inlet temperature, while preventing material failure of hot gas path components. Power lost due to pumping compressed air in turbine blade internal cooling channels can affect the overall efficiency of a power cycle. While designing such a cooling channel, it is important to ensure high heat transfer within a given pressure drop budget. Design optimization using CFD typically needs a sufficient number of computationally expensive high fidelity simulations to search for an optimum design. In the current work, a multi-fidelity Gaussian process surrogate based Bayesian optimization framework has been used to design a pin fin array channel, typically used for gas turbine blade trailing edge cooling, with the goal of maximizing Nusselt number at a constrained pressure loss. Data from a cheaper, low fidelity computation model has been leveraged, along with an expensive high fidelity one, in order to search for a design configuration that maximizes heat transfer at a given pressure drop constraint. The multi-fidelity approach improves upon single fidelity optimization results, at a low additional computational cost due to usage of simulation data from the cheap lowfidelity CFD model. A multifidelity approach outperformed the previous single fidelity case, with 6% higher objective while satisfying the pressure constraint, with 2% lower computational expense and 8 fewer high fidelity data point computations.

A multifidelity Bayesian optimization method for inertial confinement fusion design

Due to their cost, experiments for inertial confinement fusion (ICF) heavily rely on numerical simulations to guide design. As simulation technology progresses, so too can the fidelity of models used to plan for new experiments. However, these high-fidelity models are by themselves insufficient for optimal experimental design, because their computational cost remains too high to efficiently and effectively explore the numerous parameters required to describe a typical experiment. Therefore, traditionally, ICF design has relied on low-fidelity modeling to initially identify potentially interesting design regions, which are then subsequently explored via selected high-fidelity modeling. In this paper, we demonstrate that this two-step approach can be insufficient: even for simple design problems, a two-step optimization strategy can lead high-fidelity searching toward incorrect regions and consequently waste computational resources on parameter regimes far away from the true optimal solution. We reveal that a primary cause of this behavior in ICF design problems is the presence of low-fidelity optima in different regions of the parameter space far away from high-fidelity optima. To address this issue, we propose an iterative multifidelity Bayesian optimization method based on Gaussian Process Regression that leverages both low- and high-fidelity models simultaneously. We demonstrate, using both two- and eight-dimensional ICF test problems, that our algorithm can effectively utilize both low-fidelity and high-fidelity models to refine the designs. This approach proves to be more efficient than relying solely on high-fidelity modeling for optimization.

Physics-aware multifidelity Bayesian optimization: A generalized formulation

The adoption of high-fidelity models for many-query optimization problems is majorly limited by the significant computational cost required for their evaluation at every query. Multifidelity Bayesian methods (MFBO) allow to include costly high-fidelity responses for a sub-selection of queries only, and use fast lower-fidelity models to accelerate the optimization process. State-of-the-art methods rely on a purely data-driven search and do not include explicit information about the physical context. This paper acknowledges that prior knowledge about the physical domains of engineering problems can be leveraged to accelerate these data-driven searches, and proposes a generalized formulation for MFBO to embed a form of domain awareness during the optimization procedure. In particular, we formalize a bias as a multifidelity acquisition function that captures the physical structure of the domain. This permits to partially alleviate the data-driven search from learning the domain properties on-the-fly, and sensitively enhances the management of multiple sources of information. The method allows to efficiently include high-fidelity simulations to guide the optimization search while containing the overall computational expense. Our physics-aware multifidelity Bayesian optimization is presented and illustrated for two classes of optimization problems frequently met in science and engineering, namely design optimization and health monitoring problems.

A Multi-Fidelity Bayesian Optimization Approach for Constrained Multi-Objective Optimization Problems

Abstract In this paper, a multi-fidelity Bayesian optimization approach is presented to tackle computationally expensive constrained multiobjective optimization problems (MOPs). The proposed approach consists of a three-stage optimization framework designed to search for promising candidate points. In the first stage, an acquisition function is proposed to identify a feasible solution if none is available in the current set of sampling points. Subsequently, a new multi-fidelity weighted expected hypervolume improvement function is developed to find better solutions. In the third stage, a constrained weighted lower confidence bound acquisition function is presented to enhance the constraint predictions and refine the solutions near the constraint boundary. Additionally, a filter strategy is suggested to determine whether constraint updating is necessary, aiming to save computational resources and improve optimization efficiency. Moreover, to expedite the optimization process, a parallel optimization approach is further developed based on the suggested three-stage optimization framework. To achieve this, a multi-fidelity influence function is introduced, allowing the proposed approach to determine a desired number of candidate points within a single iteration. Lastly, the proposed approach is demonstrated through six numerical benchmark examples, which verifies its significant advantages in addressing expensive constrained MOPs. Besides, the proposed approach is applied to the multiobjective optimization of a metamaterial vibration isolator, resulting in the attainment of satisfactory solutions.

Non-myopic multipoint multifidelity Bayesian framework for multidisciplinary design

The adoption of high-fidelity models in multidisciplinary design optimization (MDO) permits to enhance the identification of superior design configurations, but would prohibitively rise the demand for computational resources and time. Multifidelity Bayesian Optimization (MFBO) efficiently combines information from multiple models at different levels of fidelity to accelerate the MDO procedure. State-of-the-art MFBO methods currently meet two major limitations: (i) the sequential adaptive sampling precludes parallel computations of high-fidelity models, and (ii) the search scheme measures the utility of new design evaluations only at the immediate next iteration. This paper proposes a Non-Myopic Multipoint Multifidelity Bayesian Optimization (NM3-BO) algorithm to sensitively accelerate MDO overcoming the limitations of standard methods. NM3-BO selects a batch of promising design configurations to be evaluated in parallel, and quantifies the expected long-term improvement of these designs at future steps of the optimization. Our learning scheme leverages an original acquisition function based on the combination of a two-step lookahead policy and a local penalization strategy to measure the future utility achieved evaluating multiple design configurations simultaneously. We observe that the proposed framework permits to sensitively accelerate the MDO of a space vehicle and outperforms popular algorithms.

Safeguarding Multi-Fidelity Bayesian Optimization Against Large Model Form Errors and Heterogeneous Noise

Abstract Bayesian optimization (BO) is a sequential optimization strategy that is increasingly employed in a wide range of areas such as materials design. In real-world applications, acquiring high-fidelity (HF) data through physical experiments or HF simulations is the major cost component of BO. To alleviate this bottleneck, multi-fidelity (MF) methods are used to forgo the sole reliance on the expensive HF data and reduce the sampling costs by querying inexpensive low-fidelity (LF) sources whose data are correlated with HF samples. However, existing multi-fidelity BO (MFBO) methods operate under the following two assumptions that rarely hold in practical applications: (1) LF sources provide data that are well correlated with the HF data on a global scale, and (2) a single random process can model the noise in the MF data. These assumptions dramatically reduce the performance of MFBO when LF sources are only locally correlated with the HF source or when the noise variance varies across the data sources. In this paper, we view these two limitations and uncertainty sources and address them by building an emulator that more accurately quantifies uncertainties. Specifically, our emulator (1) learns a separate noise model for each data source, and (2) leverages strictly proper scoring rules in regularizing itself. We illustrate the performance of our method through analytical examples and engineering problems in materials design. The comparative studies indicate that our MFBO method outperforms existing technologies, provides interpretable results, and can leverage LF sources which are only locally correlated with the HF source.

High-dimensional multi-fidelity Bayesian optimization for quantum control

We present the first multi-fidelity Bayesian optimization (BO) approach for solving inverse problems in the quantum control of prototypical quantum systems. Our approach automatically constructs time-dependent control fields that enable transitions between initial and desired final quantum states. Most importantly, our BO approach gives impressive performance in constructing time-dependent control fields, even for cases that are difficult to converge with existing gradient-based approaches. We provide detailed descriptions of our machine learning methods as well as performance metrics for a variety of machine learning algorithms. Taken together, our results demonstrate that BO is a promising approach to efficiently and autonomously design control fields in general quantum dynamical systems.

Towards a multi-fidelity & multi-objective Bayesian optimization efficient algorithm

Black-box optimization methods like Bayesian optimization are often employed in cases where the underlying objective functions and their gradient are complex, expensive to evaluate, or unavailable in closed form, making it difficult or impossible to use traditional optimization techniques. Fixed-wing drone design problems often face this kind of situations. Moreover in the literature multi-fidelity strategies allow to consistently reduce the optimization cost for mono-objective problems. The purpose of this paper is to propose a multi-fidelity Bayesian optimization method that suits to multi-objective problem solving. In this approach, low-fidelity and high-fidelity objective functions are used to build co-Kriging surrogate models which are then optimized using a Bayesian framework. By combining multiple fidelity levels and objectives, this approach efficiently explores the solution space and identifies the set of Pareto-optimal solutions. First, four analytical problems were solved to assess the methodology. The approach was then used to solve a more realistic problem involving the design of a fixed-wing drone for a specific mission. Compared to the mono-fidelity strategy, the multi-fidelity one significantly improved optimization performance. On the drone test case, using a fixed budget, it allows to divide the inverted generational distance metric by 6.87 on average.

An Analog Circuit Building Block Generator via Nested Multi-Fidelity Modeling

In this paper, we propose an analog circuit building block generator, which is composed of a layout-aware analog circuit sizing scheme and an automated analog circuit layout generator. We reformulate the analog circuit sizing problem as a novel constrained multi-objective optimization problem and propose a multi-objective Bayesian optimization scheme that can find multiple different qualified designs. We further leverage a nested multi-fidelity Bayesian optimization method in layout-aware sizing to counterbalance the schematic-level simulation and the expensive post-layout simulation without losing efficiency. The automated layout generator enables the in-loop layout generation, and thus it is possible to find a set of valid post-layout results directly. The experimental results on three real-world analog circuits have demonstrated the efficiency of our proposed approach.

Coupling multi-fidelity xRAGE with machine learning for graded inner shell design optimization in double shell capsules

Bayesian optimization has shown promise for the design optimization of inertial confinement fusion targets. Specifically, in Vazirani et al. [Phys. Plasmas 28, 122709 (2021)], optimal designs for double shell capsules with graded inner shells were identified using one-dimensional xRAGE simulation yield calculations. While the machine learning models were able to accurately learn and predict one-dimensional simulation target performance, using simulations with higher fidelity would improve design optimization and better match with the expected experimental performance. However, higher fidelity physics modeling, i.e., two-dimensional xRAGE simulations, requires significantly larger computational time/cost, usually at least an order of magnitude, in comparison with one-dimensional simulations. This study presents a multi-fidelity Bayesian optimization, in which the machine learning model leverages low-fidelity (one-dimensional xRAGE) and high-fidelity (two-dimensional xRAGE) simulations to more accurately predict “pre-shot” target performance with respect to the expected experimental performance. By building a multi-fidelity Bayesian optimization framework coupled with xRAGE, the low-fidelity and high-fidelity simulations are able to inform one another, such that we have: (1) improved physics modeling in comparison with using low-fidelity simulations alone, (2) reduced computational time/cost in comparison with using high-fidelity simulations alone, and (3) more confidence in the expected performance of optimized targets during real-world experiments. In the future, we plan to use this robust multi-fidelity Bayesian optimization methodology to expedite the design of graded inner shells further and eventually full capsules as a part of the current double shell campaign at the National Ignition Facility.

Accelerated Design of Architected Materials with Multifidelity Bayesian Optimization

In this work, we present a multifidelity Bayesian optimization framework for designing architected materials with optimal energy absorption during compression. Data from both physical experiments (high fidelity) and numerical simulations (low fidelity) are fed in parallel to train the surrogate model, which iteratively decides the next sets of experiments and simulations to run in order to find the optimal structural parameters. We show that having multifidelity data sources allows the optimization framework to find the optimum after fewer iterations relative to using a single high-fidelity source. This saves both material costs and time in the optimization process. Finally, we also apply constraints (on relative density and stress variations) to the optimization process, finding optimal structures within the bounds of the constraints. This framework can be translated to other problems that require complex, high-fidelity, labor-intensive experiments while automating low-fidelity simulations.

Multi-fidelity Bayesian optimization to solve the inverse Stefan problem

In this work, we propose an efficient solution of the inverse Stefan problem by multi-fidelity Bayesian optimization. We construct a multi-fidelity Gaussian process surrogate model by combining many low-fidelity estimates of a solidification problem with only a few high-fidelity measurements. To solve the inverse problem, we employ the Gaussian process model in a Bayesian optimization approach based on a multi-fidelity knowledge gradient acquisition function. To account for the specific structure of the target function, we reformulate it as a composite function and thus significantly improve the stability of the optimization procedure. Target values can be switched easily, and previously obtained samples and surrogate models can be reused. The proposed method iteratively improves the recommended solution of the inverse problem. Explicitly adding recommended points of previous iterations to the solution procedure enhances the convergence properties of the algorithm.We demonstrate the applicability of the algorithm by solving the inverse problem for a planar solidification front in a single-fidelity setting. Process parameters are identified for targeted crystal-growth velocities during directional dendritic solidification. The relation between these velocities and process parameters, such as undercooling, thermal diffusivity, capillarity, and capillary anisotropy, defines the thermo-mechanical properties of the solidified part in metal-based additive manufacturing. Material design is based on the corresponding inverse problem. Cost-efficiency of solving the inverse problem is improved by introducing a fidelity hierarchy based on coarse-grid approximations of high-fidelity numerical simulations. The open-source simulation framework ALPACA for multiphase flows allows to generate data at all fidelities. We demonstrate the superior convergence properties of the presented multi-fidelity approach by comparison with an approach solely based on high-fidelity measurements of the tip velocity.

Combining multi-fidelity modelling and asynchronous batch Bayesian Optimization

Bayesian Optimization is a useful tool for experiment design. Unfortunately, the classical, sequential setting of Bayesian Optimization does not translate well into laboratory experiments, for instance battery design, where measurements may come from different sources and their evaluations may require significant waiting times. Multi-fidelity Bayesian Optimization addresses the setting with measurements from different sources. Asynchronous batch Bayesian Optimization provides a framework to select new experiments before the results of the prior experiments are revealed. This paper proposes an algorithm combining multi-fidelity and asynchronous batch methods. We empirically study the algorithm behaviour, and show it can outperform single-fidelity batch methods and multi-fidelity sequential methods. As an application, we consider designing electrode materials for optimal performance in pouch cells using experiments with coin cells to approximate battery performance.

Multi-objective and multi-fidelity Bayesian optimization of laser-plasma acceleration

Beam parameter optimization in accelerators involves multiple, sometimes competing, objectives. Condensing these individual objectives into a single figure of merit unavoidably results in a bias towards particular outcomes, often in an undesired way in the absence of prior knowledge. Finding an optimal objective definition then requires operators to iterate over many possible objective weights and definitions, a process that can take many times longer than the optimization itself. A more versatile approach is multi-objective optimization, which establishes the trade-off curve or Pareto front between objectives. Here we present the first results on multi-objective Bayesian optimization of a simulated laser-plasma accelerator. We find that multi-objective optimization reaches comparable performance to its single-objective counterparts while allowing for instant evaluation of entirely new objectives. This dramatically reduces the time required to find appropriate objective definitions for new problems. Additionally, our multi-objective, multi-fidelity method reduces the time required for an optimization run by an order of magnitude. It does so by dynamically choosing simulation resolution and box size, requiring fewer slow and expensive simulations as it learns about the Pareto-optimal solutions from fast low-resolution runs. The techniques demonstrated in this paper can easily be translated into many different computational and experimental use cases beyond accelerator optimization.

Multi-fidelity Bayesian optimization of covalent organic frameworks for xenon/krypton separations

We employ multi-fidelity Bayesian optimization to search a large candidate set of covalent organic frameworks (COFs) for the one with the largest [simulated] equilibrium adsorptive selectivity for xenon (Xe) over krypton (Kr) at room temperature.

Multifidelity multiobjective optimization for wake-steering strategies

Abstract. Wake steering is an emerging wind power plant control strategy where upstream turbines are intentionally yawed out of perpendicular alignment with the incoming wind, thereby “steering” wakes away from downstream turbines. However, trade-offs between the gains in power production and fatigue loads induced by this control strategy are the subject of continuing investigation. In this study, we present a multifidelity multiobjective optimization approach for exploring the Pareto front of trade-offs between power and loading during wake steering. A large eddy simulation is used as the high-fidelity model, where an actuator line representation is used to model wind turbine blades and a rainflow-counting algorithm is used to compute damage equivalent loads. A coarser simulation with a simpler loads model is employed as a supplementary low-fidelity model. Multifidelity Bayesian optimization is performed to iteratively learn both a surrogate of the low-fidelity model and an additive discrepancy function, which maps the low-fidelity model to the high-fidelity model. Each optimization uses the expected hypervolume improvement acquisition function, weighted by the total cost of a proposed model evaluation in the multifidelity case. The multifidelity approach is able to capture the logit function shape of the Pareto frontier at a computational cost only 30 % that of the single-fidelity approach. Additionally, we provide physical insights into the vortical structures in the wake that contribute to the Pareto front shape.

A Generalized Framework of Multifidelity Max-Value Entropy Search Through Joint Entropy.

Bayesian optimization (BO) is a popular method for expensive black-box optimization problems; however, querying the objective function at every iteration can be a bottleneck that hinders efficient search capabilities. In this regard, multifidelity Bayesian optimization (MFBO) aims to accelerate BO by incorporating lower-fidelity observations available with a lower sampling cost. In our previous work, we proposed an information-theoretic approach to MFBO, referred to as multifidelity max-value entropy search (MF-MES), which inherits practical effectiveness and computational simplicity of the well-known max-value entropy search (MES) for the single-fidelity BO. However, the applicability of MF-MES is still limited to the case that a single observation is sequentially obtained. In this letter, we generalize MF-MES so that information gain can be evaluated even when multiple observations are simultaneously obtained. This generalization enables MF-MES to address two practical problem settings: synchronous parallelization and trace-aware querying. We show that the acquisition functions for these extensions inherit the simplicity of MF-MES without introducing additional assumptions. We also provide computational techniques for entropy evaluation and posterior sampling in the acquisition functions, which can be commonly used for all variants of MF-MES. The effectiveness of MF-MES is demonstrated using benchmark functions and real-world applications such as materials science data and hyperparameter tuning of machine-learning algorithms.

Nonlinear Multi-Fidelity Bayesian Optimization: An Application in the Design of Blast Mitigating Structures

<div class="section abstract"><div class="htmlview paragraph">A common scenario in engineering design is the availability of several black-box functions that describe an event with different levels of accuracy and evaluation cost. Solely employing the highest fidelity, often the most expensive, black-box function leads to lengthy and costly design cycles. Multi-fidelity modeling improves the efficiency of the design cycle by combining information from a small set of observations of the high-fidelity function and large sets of observations of the low-fidelity, fast-to-evaluate functions. In the context of Bayesian optimization, the most popular multi-fidelity model is the auto-regressive (AR) model, also known as the co-kriging surrogate. The main building block of the AR model is a weighted sum of two Gaussian processes (GPs). Therefore, the AR model is well suited to exploit information generated by sources that present strong linear correlations. Recently, the non-linear auto-regressive Gaussian process (NARGP) model has appeared as an alternative to integrate information generated by non-linearly correlated black-box functions. The performance of the NARGP model in structural optimization has remained largely unexplored. This investigation presents a Bayesian optimization approach that implements the NARGP model as the multi-fidelity surrogate model. The optimization strategy is utilized in the design sandwich composite armors for blast mitigation. The armors are made of four layers: steel, carbon fiber reinforced polymer (CFRP), aluminum honeycomb (HC), and CFRP. The optimization problem has three design variables, which are the thickness of the CFRP and aluminum HC layers. Two objectives are minimized: the armor’s penetrations and the reaction force at the armor’s supports. The black-box functions are two finite element models with different levels of fidelity. The low-fidelity model assumes elastic behavior of the sandwich composite. The high-fidelity model considers the nonlinear behavior of each layer of the armor. The results show that the proposed non-linear multi-fidelity Bayesian optimization approach produces a more stable expansion of the Pareto front than an optimization strategy that employs the AR model. This outcome suggests that the NARGP model is an appealing alternative in design problems with a limited number of function evaluations of the high-fidelity source.</div></div>

Learning cardiac activation maps from 12-lead ECG with multi-fidelity Bayesian optimization on manifolds

We propose a method for identifying an ectopic activation in the heart non-invasively. Ectopic activity in the heart can trigger deadly arrhythmias. The localization of the ectopic foci or earliest activation sites (EASs) is therefore a critical information for cardiologists in deciding the optimal treatment. In this work, we formulate the identification problem as a global optimization problem, by minimizing the mismatch between the ECG predicted by a cardiac model, when paced at a given EAS, and the observed ECG during the ectopic activity. Our cardiac model amounts at solving an anisotropic eikonal equation for cardiac activation and the forward bidomain model in the torso with the lead field approach for computing the ECG. We build a Gaussian process surrogate model of the loss function on the heart surface to perform Bayesian optimization. In this procedure, we iteratively evaluate the loss function following the lower confidence bound criterion, which combines exploring the surface with exploitation of the minimum region. We also extend this framework to incorporate multiple levels of fidelity of the model. We show that our procedure converges to the minimum only after 11.7 ± 10.4 iterations (20 independent runs) for the single-fidelity case and 3.5 ± 1.7 iterations for the multi-fidelity case. We envision that this tool could be applied in real time in a clinical setting to identify potentially dangerous EASs.

On the importance of microstructure information in materials design: PSP vs PP

The focus of goal-oriented materials design is to find the necessary chemistry/processing conditions to achieve the desired properties. In this setting, a material’s microstructure is either only used to carry out multiscale simulations to establish an invertible quantitative process-structure-property (PSP) relationship, or to rationalize a posteriori the underlying microstructural features responsible for the properties achieved. The materials design process itself, however, tends to be microstructure-agnostic: the microstructure only mediates the process-property (PP) connection and is—with some exceptions such as architected materials—seldom used for the optimization itself. While the existence of PSP relationships is the central paradigm of materials science, it would seem that for materials design, one only needs to focus on PP relationships. In this work, we attempt to resolve the issue whether ‘PSP’ is a superior paradigm for materials design in cases where the microstructure itself cannot be (directly) manipulated to optimize materials’ properties. To this end, we formulate a novel microstructure-aware closed-loop multi-fidelity Bayesian optimization framework for materials design and rigorously demonstrate the importance of the microstructure information in the design process. The problem considered here involves finding the right combination of chemistry and processing parameters that maximizes a targeted mechanical property of a model dual-phase steel. Our results clearly show that an explicit incorporation of microstructure knowledge in the materials design framework significantly enhances the materials optimization process. We thus prove, in a computational setting, and for a particular representative problem where microstructure intervenes to influence properties of interest, that ‘PSP’ is superior to ‘PP’ when it comes to materials design.