Outer Optimization Loop Research Articles

Hazard-consistent scenario selection for long-term storm surge risk assessment over extended coastal regions

The recent destructive hurricane seasons and concerns related to the future influence of climate change have increased the relevance of coastal storm hazards and, in particular, of storm surge hazard estimation when discussing the resilience of coastal communities. This hazard is generally represented as surge inundation probabilities over a large number of individual locations in the geographic domain of interest, and is typically assessed utilizing an ensemble of storm scenarios (i.e., storm events) that are representative of the regional climatology. This paper investigates the storm ensemble selection within this setting, with the objective of identifying a small number of storm scenarios that are consistent with some chosen hazard descriptions over a large geographic region. Beyond the storm events, the occurrence rates (i.e., weights) are also updated. Following past works, a two-stage optimization is adopted for the storm selection. The inner-loop identifies the occurrence rates for a given storm subset, formulating the problem as a linear programming optimization for the sum of absolute deviation for the predicted hazard. The outer-loop searches for the best subset with the desired number of storms, adopting a genetic algorithm integer optimization for minimizing the aforementioned deviation. This work extends this implementation to extended coastal regions, with many locations of interest. In this case, it is computationally intractable to consider all the locations in the domain within the linear programming formulation, and for this reason, a subset of representative locations is chosen through cluster analysis. The hazard description for only these locations is used in the storm ensemble selection. For clustering, different strategies using correlation between locations based on geospatial information, surge response, or a combination of both are examined. Additionally, the correlation in the hazard description is better integrated into the storm selection by establishing a modification of the objective function adopted for the outer optimization loop. Applications to different North Atlantic coastal domains are presented as case studies.

IMAML-IDCG: Optimization-based meta-learning with ImageNet feature reusing for few-shot invasive ductal carcinoma grading

Model-Agnostic Meta-Learning (MAML) is a widely used few-shot learning (FSL) technique that reduces reliance on large, labeled datasets in deep learning for medical imaging analysis. However, MAML requires backpropagating through all feature layers for task adaptation, leading to suboptimal computational efficiency. We propose IMAML-IDCG (ImageNet Model-Agnostic Meta-Learning in Invasive Ductal Carcinoma Grading), which enhances computational efficiency for few-shot grading of Invasive Ductal Carcinoma (IDC) through three key techniques: (1) ImageNet feature reusing, (2) ImageNet partial freezing strategy, and (3) adaptive inner learning rate. IMAML-IDCG is initialized with ImageNet pre-trained weights. During the inner optimization loop, only the model’s classifier head layer is optimized, leveraging prior ImageNet knowledge (ImageNet feature reusing) and employing an adaptive learning rate for improved task adaptation. In the outer optimization loop, IMAML-IDCG selectively fine-tunes the last few model layers to enhance efficiency and reduce overfitting (ImageNet partial freezing strategy). We evaluated IMAML-IDCG using the BreaKHis dataset (7,909 images) as the base dataset, and the BCHI (282 images) and PathoIDCG (3,744 images) datasets as the novel datasets. Our empirical results demonstrate that IMAML-IDCG outperforms MAML and other FSL methods in few-shot IDC grading tasks across various cross-magnification domain settings. Notably, IMAML-IDCG achieves a 14.64% improvement over MAML on the BCHI dataset and a 6.04% improvement on the PathoIDCG 40X dataset when meta-trained with the BreaKHis 40X dataset in the 3-way 5-shot scenario.

Machine learning enhanced control co-design optimization of an immersion cooled battery thermal management system

The development of lithium-ion battery technology has ensured that battery thermal management systems are an essential component of the battery pack for next-generation energy storage systems. Using dielectric immersion cooling, researchers have demonstrated the ability to attain high heat transfer rates due to the direct contact between cells and the coolant. However, feedback control has not been widely applied to immersion cooling schemes. Furthermore, current research has not considered battery pack plant design when optimizing feedback control. Uncertainties are inherent in the cooling equipment, resulting in temperature and flow rate fluctuations. Hence, it is crucial to systematically consider these uncertainties during cooling system design to improve the performance and reliability of the battery pack. To fill this gap, we established a reliability-based control co-design optimization framework using machine learning for immersion cooled battery packs. We first developed an experimental setup for 21700 battery immersion cooling, and the experiment data were used to build a high-fidelity multiphysics finite element model. The model can precisely represent the electrical and thermal profile of the battery. We then developed surrogate models based on the finite element simulations in order to reduce computational cost. The reliability-based control co-design optimization was employed to find the best plant and control design for the cooling system, in which an outer optimization loop minimized the cooling system cost while an inner loop ensured battery pack reliability. Finally, an optimal cooling system design was obtained and validated, which showed a 90% saving in cooling system energy consumption.

Reliability-Based Design Optimization of Additive Manufacturing for Lithium Battery Silicon Anode

Abstract With the blooming of the electric vehicle market and the advancement in the lithium-ion battery industry, silicon anode has shown great potential for the next-generation battery. Using the state-of-the-art additive manufacturing technique (three-dimensional (3D) holographic lithography), researchers have demonstrated that silicon anode can be fabricated as a three-dimensional bicontinuous porous microstructure. However, the volume fluctuation of the silicon anode caused by lithiation during the discharging process causes continuous capacity decay and poor cycling life. Besides, uncertainties are inherent in the manufacturing and usage processes, making it crucial to systematically consider them in the silicon anode design to improve its performance and reliability. To fill the gap between current silicon anode research and future industrial need, this study established a digital twin to investigate the optimal design for silicon anode under the uncertainties of additive manufacturing and battery usage. This study started with developing multiphysics finite element models of the silicon anode lithiation process to investigate the volume fluctuation of silicon. Then, surrogate models were built based on the results from the finite element models to reduce computational cost. The reliability-based design optimization (RBDO) was employed to find the best design point for the silicon anode, in which an outer optimization loop maximized the objective function and an inner loop dedicated to reliability analysis. Finally, the Pareto optimal front of the silicon anode designs was obtained and validated, which shows over 10% improvements in the silicon anode's total capacity and rate capability.

Comparative assessment and improvement of the powertrain supplied by hybrid energy source in fuel cell vehicles

The energy distribution and conversion between the fuel cell (FC) and the battery are realized by the powertrain in fuel cell vehicles (FCVs). In this study, the cost and efficiency of a novel powertrain based on the dual winding motor are assessed and compared with the mainstream powertrain. To shorten the energy transfer chain from FC to motor, a dual winding motor is employed and supplied by the energy source (ES) directly through inverters. The DC/DC converter is eliminated from the powertrain to avoid its cost and energy loss. The powertrain based on the dual winding motor is further optimized by multiple objectives, including FCV economy and durability. The double-loop optimization is proposed to solve the coupling between hybrid energy source sizing (HESS) and energy management strategy (EMS). Particle swarm optimization is adopted in outer-loop optimization to get the optimal size of the ES. Dynamic programming is used in inner-loop optimization to get the optimal energy distribution between ESs. Compared with the prevalent DC/DC converter-based powertrain, the optimized powertrain based on the dual winding motor is lower in manufacturing cost and fuel consumption.

MULTILEVEL MONTE CARLO ESTIMATORS FOR DERIVATIVE-FREE OPTIMIZATION UNDER UNCERTAINTY

Optimization is a key tool for scientific and engineering applications; however, in the presence of models affected by uncertainty, the optimization formulation needs to be extended to consider statistics of the quantity of interest. Optimization under uncertainty (OUU) deals with this endeavor and requires uncertainty quantification analyses at several design locations; i.e., its overall computational cost is proportional to the cost of performing a forward uncertainty analysis at each design location. An OUU workflow has two main components: an inner loop strategy for the computation of statistics of the quantity of interest, and an outer loop optimization strategy tasked with finding the optimal design, given a merit function based on the inner loop statistics. In this work, we propose to alleviate the cost of the inner loop uncertainty analysis by leveraging the so-called multilevel Monte Carlo (MLMC) method, which is able to allocate resources over multiple models with varying accuracy and cost. The resource allocation problem in MLMC is formulated by minimizing the computational cost given a target variance for the estimator. We consider MLMC estimators for statistics usually employed in OUU workflows and solve the corresponding allocation problem. For the outer loop, we consider a derivative-free optimization strategy implemented in the SNOWPAC library; our novel strategy is implemented and released in the Dakota software toolkit. We discuss several numerical test cases to showcase the features and performance of our approach with respect to its Monte Carlo single fidelity counterpart.

Non-Invasive Electroanatomical Mapping: A State-Space Approach for Myocardial Current Density Estimation.

Electroanatomical mapping is a method for creating a model of the electrophysiology of the human heart. Medical professionals routinely locate and ablate the site of origin of cardiac arrhythmias with invasive catheterization. Non-invasive localization takes the form of electrocardiographic (ECG) or magnetocardiographic (MCG) imaging, where the goal is to reconstruct the electrical activity of the human heart. Non-invasive alternatives to catheter electroanatomical mapping would reduce patients' risks and open new venues for treatment planning and prevention. This work introduces a new system state-based method for estimating the electrical activity of the human heart from MCG measurements. Our model enables arbitrary propagation paths and velocities. A Kalman filter optimally estimates the current densities under the given measurements and model parameters. In an outer optimization loop, these model parameters are then optimized via gradient descent. This paper aims to establish the foundation for future research by providing a detailed mathematical explanation of the algorithm. We demonstrate the feasibility of our method through a simplified one-layer simulation. Our results show that the algorithm can learn the propagation paths from the magnetic measurements. A threshold-based segmentation into healthy and pathological tissue yields a DICE score of 0.84, a recall of 0.77, and a precision of 0.93.

Problem-size-independent angles for a Grover-driven quantum approximate optimization algorithm

The quantum approximate optimization algorithm (QAOA) requires that circuit parameters are determined that allow one to sample from high-quality solutions to combinatorial optimization problems. Such parameters can be obtained using either costly outer-loop optimization procedures and repeated calls to a quantum computer or, alternatively, via analytical means. In this work, we consider a context in which one knows a probability density function describing how the objective function of a combinatorial optimization problem is distributed. We show that, if one knows this distribution, then the expected value of strings, sampled by measuring a Grover-driven, QAOA-prepared state, can be calculated independently of the size of the problem in question. By optimizing this quantity, optimal circuit parameters for average-case problems can be obtained on a classical computer. Such calculations can help deliver insights into the performance of and predictability of angles in QAOA in the limit of large problem sizes, in particular, for the number partitioning problem.

Input-to-State Stability of a Bilevel Proximal Gradient Descent Algorithm*

This paper studies convergence properties of inexact iterative solution schemes for bilevel optimization problems. Bilevel optimization problems emerge in control-aware design optimization, where the system design parameters are optimized in the outer loop and a discrete-time control trajectory is optimized in the inner loop, but also arise in other domains including machine learning. In the paper, an interconnection of proximal gradient algorithms is proposed to solve the inner loop and outer loop optimization problems in the setting of control-aware design optimization, and its robustness is analyzed from a control-theoretic perspective. By employing input-to-state stability arguments, conditions are derived that ensure convergence of the interconnected scheme to the optimal solution for a class of the bilevel optimization problems.

Adaptive machine learning-based surrogate modeling to accelerate PDE-constrained optimization in enhanced oil recovery

In this contribution, we develop an efficient surrogate modeling framework for simulation-based optimization of enhanced oil recovery, where we particularly focus on polymer flooding. The computational approach is based on an adaptive training procedure of a neural network that directly approximates an input-output map of the underlying PDE-constrained optimization problem. The training process thereby focuses on the construction of an accurate surrogate model solely related to the optimization path of an outer iterative optimization loop. True evaluations of the objective function are used to finally obtain certified results. Numerical experiments are given to evaluate the accuracy and efficiency of the approach for a heterogeneous five-spot benchmark problem.

Integrated design for the forming process and structural performance of variable-thickness laminates

The manufacturing process of composite laminated parts has a significant influence on the final performance, especially since the draping process can change the predefined fiber orientation at local sites. With the maturing of draping-process simulation technology, an integrated structure–process–performance design has become possible. This article reports a computational framework for the integrated design of the draping process and structural performance of variable-thickness (VT) laminates. The framework is a two-loop optimization workflow. Specifically, a genetic draping optimization method features the inner optimization loop to find the optimal draping strategy and obtain the resulting fiber orientations and manufacturing layout. Structural performance optimization outlines the outer optimization loop, where a parametric method for the physical configuration of VT laminates is proposed to evaluate the structural performance considering the draping strategy. The entire framework is demonstrated by an automotive B-pillar reinforcement. The benefits of incorporating the draping design versus no draping design are illustrated by comparing the corresponding design results. The comparison results show that the integrated design not only contributes to pretreating manufacturing defects but also allows the use of reoriented fiber directions caused by draping to improve the mechanical properties and engage the load bearing. It thus achieves a further weight reduction of 18.63%. This framework facilitates resource-friendly process design, comprehensive performance evaluation, and structurally efficient laminate design.

Constrained data smoothing via optimal control

AbstractThe article considers a problem of best smoothing in a strip, where the objective is to find a function that satisfies bilateral constraints on its values, for all and minimizes a weighted sum of the ‐norm of the second derivative and squared deviations from specified values, , at discrete points . We assume that constraints and are continuous functions that are linear in each interval , . We connect this problem to a state‐constrained optimal control problem for the double integrator, and give conditions for the existence and uniqueness of the solution under which we also show that the solution is a cubic spline with knots at and no more than two additional knots in each interval . We propose a numerical algorithm for solving this problem based on a two stage minimization, where the outer loop optimization problem is finite‐dimensional and convex, while the inner loop optimization problem admits a solution which is easy to compute. Numerical results that show the efficacy of the proposed approach are reported.

Evolutionary Machine Learning With Minions: A Case Study in Feature Selection

Many decisions in a machine learning (ML) pipeline involve nondifferentiable and discontinuous objectives and search spaces. Examples include feature selection, model selection, and hyperparameter tuning, where candidate solutions in an outer optimization loop must be evaluated via a learning subsystem. Evolutionary algorithms (EAs) are prominent gradient-free methods to handle such tasks. However, EAs are known to pose steep computational challenges, especially when dealing with large-instance datasets. As opposed to prior works that often fall back on parallel computing hardware to resolve this big data problem of EAs, in this article, we propose a novel algorithm-centric solution based on <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">evolutionary multitasking</i> . Our approach involves the creation of a band of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">minions</i> , i.e., small data proxies to the main target task, that are constructed by subsampling a fraction of the large dataset. We then combine the minions with the main task in a single multitask optimization framework, boosting evolutionary search by using small data to quickly optimize for the large dataset. Our key algorithmic contribution in this setting is to allocate computational resources to each of the tasks in a principled manner. The article considers wrapper-based feature selection as an illustrative case study of the broader idea of using multitasking to speedup outer loop evolutionary configurations of any ML subsystem. The experiments reveal that multitasking can indeed speedup baseline EAs, by more than 40% on some datasets.

A novel optimization approach for material screening in the adsorption process separation context based on rigorous models: Separation of a mixture of N2 and CO2 in a Pressure Swing Adsorption Unit as a case study

Adsorption material screening simultaneously with optimizing an adsorption-based process is a challenging problem with increasing attention in the literature. This is usually done in an uncoupled way, separated into two separated steps. The first step focuses on the materials' properties, followed by an optimization step where the focus is on the process behavior given a pre-selected material. It is observed in the literature an increasing demand for systematic optimization approaches that make use of materials science databases in a process systems engineering context. Therefore, providing a synergy between these two disciplines. Hence, the present work proposes the design of a looped optimization problem where the material and process operating conditions are used as decision variables to optimize a Pressure Swing Adsorption. An inner loop is used to screen the best material among the 12 options. An outer optimization loop is used to optimize de process operating variables. A case study presents the separation of the raffinate stream produced in the gaseous Simulated Moving Bed process to separate CH4/N2 using CO2 as the desorbent gas. Thus, the SMB raffinate stream is composed of a mixture of N2 and CO2. The results demonstrate that the proposed methodology can efficiently perform both tasks, identify the most suitable material and optimize the Pressure Swing Adsorption unit.

Efficient well placement optimization under uncertainty using a virtual drilling procedure

An Automatic Well Planner (AWP) is used to efficiently adjust pre-determined well paths to honor near-well properties and increase overall production. AWP replicates modern geosteering decision-making where adjustments to pre-programmed well paths are driven by continuous integration of data obtained from logging-while-drilling and look-ahead technology. In this work, AWP is combined into a robust optimization scheme to develop trajectories that follow reservoir properties in a more realistic manner compared to common well representations for optimization purposes. Core AWP operation relies on an artificial neural network coupled with a geology-based feedback mechanism. Specifically, for each well path candidate obtained from an outer-loop optimization procedure, AWP customizes trajectories according to the particular geological near-well properties of each realization in an ensemble of models. While well placement searches typically rely on linear well path representations, AWP develops customized trajectories by moving sequentially from heel to the toe. Analog to realistic drilling operations, AWP determines subsequent trajectory points by efficiently processing neighboring geological information. Studies are performed using the Olympus ensemble. AWP and the two derivative-free algorithms used in this work, Asynchronous Parallel Pattern Search (APPS) and Particle Swarm Optimization (PSO), are implemented using NTNU’s open-source optimization framework FieldOpt. Results show that, with both APPS and PSO, the AWP solutions outperform the solutions obtained with a straight-line parameterization in all the three tested well placement optimization scenarios, which varied from the simplest scenario with a sole producer in a single-realization environment to a scenario with the full ensemble and multiple producers.

Goal-oriented mesh adaptivity for inverse problems in linear micromorphic elasticity

In this work, we extend goal-oriented mesh adaptivity to parameter identification for a class of linear micromorphic elasticity problems. Starting from a compact formulation in our previous work (Ju and Mahnkhen, 2017), we propose a two-level optimization framework based on goal-oriented error estimation. By means of a sensitivity analysis of the generalized constitutive relations, we establish a gradient-based solver for the inverse problem, where parameters are optimized within an inner optimization loop for a given mesh. Exact error representations are derived from a Lagrange method, aiming at a quantity of interest as a user-defined functional of the parameters. By using a patch recovery technique for enhanced solutions, a computable error estimator is presented and used to drive an adaptive refinement algorithm, which forms an outer optimization loop. For a numerical study, we investigate the performance of the resulting adaptive procedure in case of perfect, incomplete and perturbed data. The results confirm the effectiveness of the proposed adaptive procedure.

Braess paradox and double-loop optimization method to enhance power grid resilience

Multiple physical failures and severe power disruptions occur in power grid under extreme operation conditions. In this paper, we study power grid resilience and derive quick recovery methods through adjusting the operating modes of available components and reconfiguring the remaining network. The largest amount of power that is available to the loads after reorganizing the remaining undamaged components in the post-disaster stage is identified as an important resilience indicator. An interior point method is firstly used to find the largest amount of power supply (LPS) of fixed topology. The post-disaster network should contain as many available components as possible in order to give the best topological connection. However, disconnecting some undamaged components proactively can further increase the LPS. This phenomenon can be interpreted as the Breass paradox and is effectively a combinatorial network reconfiguration. A double-loop optimization strategy is proposed to achieve the LPS available to the post-disaster network, where the interior point method serves the inner optimization loop and the outer optimization loop generates an optimal topology using a genetic algorithm. Simulation results verify the efficacy of the proposed method in achieving a quick power recovery in extreme events. Our work provides useful advice to power grid operators on how to effectively coordinate available resources after extreme events occur.

A non-conforming dual approach for adaptive Trust-Region reduced basis approximation of PDE-constrained parameter optimization

In this contribution we propose and rigorously analyze new variants of adaptive Trust-Region methods for parameter optimization with PDE constraints and bilateral parameter constraints. The approach employs successively enriched Reduced Basis surrogate models that are constructed during the outer optimization loop and used as model function for the Trust-Region method. Each Trust-Region sub-problem is solved with the projected BFGS method. Moreover, we propose a non-conforming dual (NCD) approach to improve the standard RB approximation of the optimality system. Rigorous improved a posteriori error bounds are derived and used to prove convergence of the resulting NCD-corrected adaptive Trust-Region Reduced Basis algorithm. Numerical experiments demonstrate that this approach enables to reduce the computational demand for large scale or multi-scale PDE constrained optimization problems significantly.

Linear buckling topology optimization of reinforced thin-walled structures considering uncertain geometrical imperfections

Geometrical imperfections significantly affect the load-carrying capacity of thin-walled structures (TWSs). Herein, we develop a topology optimization method for the stiffeners of thin-walled structures considering the worst-case critical buckling load with spatially varying geometrical uncertainties. The thickness imperfections of the thin-walled structures are modeled using a non-probabilistic bounded field model because of a lack of sufficient probability information. The bounded field uncertainty is discretized using series expansion and represented as a set of uncorrelated uncertain coefficients. Then, as an inner loop of the topology optimization problem, the worst-case critical buckling load is assessed under the non-probabilistic field description. The outer loop optimization problem is expressed as determining the optimum stiffener topology that maximizes the worst-case critical buckling load under constrained material volume, and the nested optimization problem is solved via a gradient-based algorithm. Numerical examples demonstrate that the proposed method for stiffener optimization improves the stability of thin-walled structures with uncertain geometrical imperfections.

Machine Learning-Based Reverse Modeling Approach for Rapid Tool Shape Optimization in Die-Sinking Micro Electro Discharge Machining

Abstract This paper focuses on efficient computational optimization algorithms for the generation of micro electro discharge machining (µEDM) tool shapes. In a previous paper, the authors presented a reliable reverse modeling approach to perform such tasks based on a crater-by-crater simulation model and an outer optimization loop. Two-dimensional results were obtained but 3D tool shapes proved difficult to generate due to the high numerical cost of the simulation strategy. In this paper, a new reduced modeling optimization framework is proposed, whereby the computational optimizer is replaced by an inexpensive surrogate that is trained by examples. More precisely, an artificial neural network (ANN) is trained using a small number of full reverse simulations and subsequently used to directly generate optimal tool shapes, given the geometry of the desired workpiece cavity. In order to train the ANN efficiently, a method of data augmentation is developed, whereby multiple features from fully simulated EDM cavities are used as separate instances. The performances of two ANN are evaluated, one trained without modification of process parameters (gap size and crater shape) and the second trained with a range of process parameter instances. It is shown that in both cases, the ANN can produce unseen tool shape geometries with less than 6% deviation compared to the full computational optimization process and at virtually no cost. Our results demonstrate that optimized tool shapes can be generated almost instantaneously, opening the door to the rapid virtual design and manufacturability assessment of µEDM die-sinking operations.