Expensive Computational Models Research Articles

Physics-constrained polynomial chaos expansion for scientific machine learning and uncertainty quantification

We present a novel physics-constrained polynomial chaos expansion as a surrogate modeling method capable of performing both scientific machine learning (SciML) and uncertainty quantification (UQ) tasks. The proposed method possesses a unique capability: it seamlessly integrates SciML into UQ and vice versa, which allows it to quantify the uncertainties in SciML tasks effectively and leverage SciML for improved uncertainty assessment during UQ-related tasks. The proposed surrogate model can effectively incorporate a variety of physical constraints, such as governing partial differential equations (PDEs) with associated initial and boundary conditions constraints, inequality-type constraints (e.g., monotonicity, convexity, non-negativity, among others), and additional a priori information in the training process to supplement limited data. This ensures physically realistic predictions and significantly reduces the need for expensive computational model evaluations to train the surrogate model. Furthermore, the proposed method has a built-in uncertainty quantification (UQ) feature to efficiently estimate output uncertainties. To demonstrate the effectiveness of the proposed method, we apply it to a diverse set of problems, including linear/non-linear PDEs with deterministic and stochastic parameters, data-driven surrogate modeling of a complex physical system, and UQ of a stochastic system with parameters modeled as random fields.

Quantifying spit growth and its hydrodynamic drivers in wind-dominated lake environments

Many sand spits are morphodynamically complex landforms, that are either analysed with complex and expensive computational models or at a conceptual level. Therefore, most case studies on spits in different environments are descriptive. A novel method based on the use of polar coordinates was devised to quantitatively analyse spit morphodynamics in a non-tidal, wind-dominated lake environment, using the Marker Wadden islands in Lake Markermeer, the Netherlands, as a case study. A high-resolution morphological data set allowed for the quantification of sedimentation processes around two spits, in two distinctive depth zones. Spit-platform growth is governed by alongshore currents that transport sediment over the spit-platform into deeper waters; the size of the spit-platform in turn affects the growth of the spit around the mean water level. Insight in this complex interplay of processes is crucial to understand spit behaviour in low-energy lake environments. At the Marker Wadden the submerged spit-platform grows during high energy wind events while the emerged spit part grows under mild to moderate energy conditions. With this new method we can quantitatively explore the role of different wave and flow conditions and predict spit growth direction in non-tidal, wind-dominated environments, beyond the level of conceptual descriptions.

Multivariate sensitivity analysis for a large-scale climate impact and adaptation model

Abstract We apply a new efficient methodology for Bayesian global sensitivity analysis for large-scale multivariate data. A multivariate Gaussian process is used as a surrogate model to replace the expensive computer model. To improve the computational efficiency and performance of the model, compactly supported correlation functions are used. The goal is to generate sparse matrices, which give crucial advantages when dealing with large data sets. The method was applied to multivariate data from the IMPRESSIONS Integrated Assessment Platform version 2. Our empirical results on Integrated Assessment Platform version 2 data show that the proposed methods are efficient and accurate for global sensitivity analysis of complex models.

Learning non-stationary and discontinuous functions using clustering, classification and Gaussian process modelling

Surrogate models have shown to be an extremely efficient aid in solving engineering problems that require repeated evaluations of an expensive computational model. They are built by sparsely evaluating the costly original model and have provided a way to solve otherwise intractable problems. A crucial aspect in surrogate modelling is the assumption of smoothness and regularity of the model to approximate. This assumption is however not always met in reality. For instance in civil or mechanical engineering, some models may present discontinuities or non-smoothness e.g., in case of instability patterns such as buckling or snap-through. Building a single surrogate model capable of accounting for these fundamentally different behaviours or discontinuities is not an easy task. In this paper, we propose a three-stage approach for the approximation of non-smooth functions which combines clustering, classification and regression. The idea is to split the space following the localized behaviors or regimes of the system and build local surrogates that are eventually assembled. A sequence of well-known machine learning techniques are used: Dirichlet process mixtures models (DPMM), support vector machines and Gaussian process modelling. The approach is tested and validated on two analytical functions and a finite element model of a tensile membrane structure.

Updating an uncertain and expensive computational model in structural dynamics based on one single target FRF using a probabilistic learning tool

Updating an uncertain and expensive computational model in structural dynamics based on one single target FRF using a probabilistic learning tool

Data-driven flow-map models for data-efficient discovery of dynamics and fast uncertainty quantification of biological and biochemical systems.

Computational models are increasingly used to investigate and predict the complex dynamics of biological and biochemical systems. Nevertheless, governing equations of a biochemical system may not be (fully) known, which would necessitate learning the system dynamics directly from, often limited and noisy, observed data. On the other hand, when expensive models are available, systematic and efficient quantification of the effects of model uncertainties on quantities of interest can be an arduous task. This paper leverages the notion of flow-map (de)compositions to present a framework that can address both of these challenges via learning data-driven models useful for capturing the dynamical behavior of biochemical systems. Data-driven flow-map models seek to directly learn the integration operators of the governing differential equations in a black-box manner, irrespective of structure of the underlying equations. As such, they can serve as a flexible approach for deriving fast-to-evaluate surrogates for expensive computational models of system dynamics, or, alternatively, for reconstructing the long-term system dynamics via experimental observations. We present a data-efficient approach to data-driven flow-map modeling based on polynomial chaos Kriging. The approach is demonstrated for discovery of the dynamics of various benchmark systems and a coculture bioreactor subject to external forcing, as well as for uncertainty quantification of a microbial electrosynthesis reactor. Such data-driven models and analyses of dynamical systems can be paramount in the design and optimization of bioprocesses and integrated biomanufacturing systems.

Adaptive learning for reliability analysis using Support Vector Machines

Given an expensive computational model of a system subject to reliability requirements, this work shows how to approximate the failure probability by learning adaptively the high-likelihood regions of the Limit State Function using Support Vector Machines. To this end, an algorithm is proposed that selects informative parameter points to add to training data at each iteration to improve the accuracy of the approximation. Furthermore, we provide a means to quantify the uncertainty in the Limit State Function, using geometrical arguments to estimate an upper bound to the failure probability.

Sequential Active Learning of Low-Dimensional Model Representations for Reliability Analysis

To date, the analysis of high-dimensional, computationally expensive engineering models remains a difficult challenge in risk and reliability engineering. We use a combination of dimensionality reduction and surrogate modeling termed partial least squares--driven polynomial chaos expansion (PLS-PCE) to render such problems feasible. Standalone surrogate models typically perform poorly for reliability analysis. Therefore, in a previous work, we have used PLS-PCEs to reconstruct the intermediate densities of a sequential importance sampling approach to reliability analysis. Here, we extend this approach with an active learning procedure that allows for improved error control at each importance sampling level. To this end, we formulate an estimate of the combined estimation error for both the subspace identified in the dimension reduction step and the surrogate model constructed therein. With this, it is possible to adapt the training set so as to optimally learn the subspace representation and the surrogate model constructed therein. The approach is gradient-free and thus can be directly applied to black box--type models. We demonstrate the performance of this approach with a series of low- (2 dimensions) to high- (869 dimensions) dimensional example problems featuring a number of well-known caveats for reliability methods besides high dimensions and expensive computational models: strongly nonlinear limit-state functions, multiple relevant failure regions, and small probabilities of failure.

Bayesian optimization for inverse calibration of expensive computer models: A case study for Johnson-Cook model in machining

Inverse model calibration for identifying the constitutive model parameters can be computationally demanding for expensive-to-evaluate simulation models. This paper presents a modified Bayesian optimization (BO) method, denoted as BO-bound, that incorporates theoretical bounds on the quantity of interest. A case study for the inverse calibration of the Johnson Cook (J-C) flow stress model parameters is presented using machining (cutting) force data. The results show fast calibration of the five J-C parameters within 25 simulations. In general, the BO-bound method is applicable for inverse calibration of any expensive simulation models as well as optimization problems with known bounds.

EFFICIENT CALIBRATION FOR HIGH-DIMENSIONAL COMPUTER MODEL OUTPUT USING BASIS METHODS

Calibration of expensive computer models with high-dimensional output fields can be approached via history matching. If the entire output field is matched, with patterns or correlations between locations or time points represented, calculating the distance metric between observational data and model output for a single input setting requires a time intensive inversion of a high-dimensional matrix. By using a low-dimensional basis representation rather than emulating each output individually, we define a metric in the reduced space that allows the implausibility for the field to be calculated efficiently, with only small matrix inversions required, using projection that is consistent with the variance specifications in the implausibility. We show that projection using the $L_2$ norm can result in different conclusions, with the ordering of points not maintained on the basis, with implications for both history matching and probabilistic methods. We demonstrate the scalability of our method through history matching of the Canadian atmosphere model, CanAM4, comparing basis methods to emulation of each output individually, showing that the basis approach can be more accurate, whilst also being more efficient.

Accelerated HKUST-1 Thin-Film Property Optimization Using Active Learning.

A flow-coating method termed solution shearing has been shown to grow large-area thin films with no void spaces. Attaining full coverage is one of the key prerequisites for the adoption of any metal-organic framework (MOF) thin film for a variety of practical applications, including separation, membranes and sensors. However, the solution-shearing process has multiple discrete and continuous parameters that can be varied, including the metal ion and linker concentrations, solvents, substrate temperature, coating speed, and the number of coating passes. Optimization of these parameters for full coverage is a time-consuming and daunting process due to vast parameter space. Here, we incorporate an active learning approach into the solution-sheared HKUST-1 thin-film-processing parameters to control the coverage and extend the approach to gain control over the thickness. The understanding of high-quality MOF thin-film formation using solution shearing is improved by correlating the processing parameter sets and their corresponding film coverage. A large area and fully covered HKUST-1 thin film with a minimized thickness of 2.2 μm is fabricated by using guidance from active learning. To confirm full coverage, a redox-active molecule, called 7,7,8,8-tetracyanoquinodimethane (TCNQ), is incorporated along with the HKUST-1 thin film. The TCNQ@HKUST-1 thin film with a minimized thickness has the same order of magnitude of electrical conductivity as that of the TCNQ@HKUST-1 thin film created previously while reducing the film thickness by 60%. We show that active learning has the potential to rapidly navigate the vast processing space in multicomponent systems, especially when experiments are expensive and traditional computational models are not readily available for process optimization.

Derivation of Kinematic Equations Based on Full-Scale Racking Tests for Seismic Performance Evaluation of Unitized Four-Sided Structural Sealant Glazing Curtain Wall Systems

Curtain wall glazing systems are a major part of a building due to the multiple roles they have, including occupant protection against environmental effects and the transfer of loads to the structural system. From a structural perspective, limited analytical guidelines and methods exist to aid designers in their determination of the curtain wall performance without extensive simulation or laboratory testing. This study takes experimental data from full-scale, “unitized”, four-sided structural sealant glazing (4SSG) curtain wall system mockups featuring a re-entrant corner subjected to cyclic racking displacements in accordance with the American Architectural Manufacturers Association AAMA 501.6 protocol to derive and establish equations that predict the relative displacements of the glass relative to the glazing frame, based on the amount of inter-story drift. Through derivation and testing, sealant cohesive failure and glass cracking were identified as limit states and corresponding drift levels were determined to control many of the equations. Displacements from the newly derived equations were correlated to the effective shear strain value experienced by the structural silicone in the mockup concurrently with the curtain wall’s drift capacity. This paper provides detailed derivation of the kinematic equations for possible use by glazing design professionals. Such equations can help designers to more easily predict the drifts that cause damage to such systems by manual calculations without the need for expensive mockup testing or time-intensive computer models.

Fast, Efficient and Flexible Particle Accelerator Optimisation Using Densely Connected and Invertible Neural Networks

Particle accelerators are enabling tools for scientific exploration and discovery in various disciplines. However, finding optimised operation points for these complex machines is a challenging task due to the large number of parameters involved and the underlying non-linear dynamics. Here, we introduce two families of data-driven surrogate models, based on deep and invertible neural networks, that can replace the expensive physics computer models. These models are employed in multi-objective optimisations to find Pareto optimal operation points for two fundamentally different types of particle accelerators. Our approach reduces the time-to-solution for a multi-objective accelerator optimisation up to a factor of 640 and the computational cost up to 98%. The framework established here should pave the way for future online and real-time multi-objective optimisation of particle accelerators.

Computing Marginal Cost of Durability of Energy Systems Components by Structural Optimization With Fatigue Constraints

Abstract There is a relationship between product durability and the effect the product has on the environment and economy. One approach that impacts this dynamic involves a circular economy. The idea of a circular economy is gaining more traction as some businesses have begun to shift toward a product as a service model in which they, the businesses, maintain ownership of the product. One example of this business model is emerging in the energy sector. Given this shift, the life of the product becomes more important as it directly impacts the bottom line of the business. This gives rise to the marginal cost of durability (MCD) metric. The MCD determines the cost of the product in relation to the life of the system. For longer life, the design generally necessitates more cost-intensive measures to ensure durability. In the context of sustainable design, system life is particularly important for renewable energy systems that promote sustainable living. These large structures often require a high volume of materials and the end-of-life disposal for those materials. The design requirements for material also increase as the design life increases. The additional materials provide a safeguard against failure phenomena, such as fatigue. The MCD metric has been used in previous studies. However, there is no formal method for determining the MCD. This article examines a method for measuring the MCD for the commercial class of wind energy production systems. A metamodel of the damage response is built in lieu of expensive computational models. Design optimization is used to search for the design parameters having fatigue damage as a constraint. This process is repeated for a set of system life values, yielding a set of designs. Curve fitting is used to find a mathematical relationship between life and cost. An example of this method is applied to the study of a wind turbine tower life. The study indicates that the wind turbine tower design for 80 years has 34% more mass and cost than a 20-year design. The results from the proposed method provide information that can be used to determine the design life of a system.

Robust optimisation of computationally expensive models using adaptive multi-fidelity emulation

Computationally expensive models are increasingly employed in the design process of engineering products and systems. Robust design in particular aims to obtain designs that exhibit near-optimal performance and low variability under uncertainty. Surrogate models are often employed to imitate the behaviour of expensive computational models. Surrogates are trained from a reduced number of samples of the expensive model. A crucial component of the performance of a surrogate is the quality of the training set. Problems occur when sampling fails to obtain points located in an area of interest and/or where the computational budget only allows for a very limited number of runs of the expensive model. This paper employs a Gaussian process emulation approach to perform efficient single-loop robust optimisation of expensive models. The emulator is enhanced to propagate input uncertainty to the emulator output, allowing single-loop robust optimisation. Further, the emulator is trained with multi-fidelity data obtained via adaptive sampling to maximise the quality of the training set for the given computational budget. An illustrative example is presented to highlight how the method works, before it is applied to two industrial case studies.

Interplay of stereo-electronic, vibronic and environmental effects in tuning the chiroptical properties of an Ir(III) cyclometalated N-heterocyclic carbene

Chiroptical spectra are among the most suitable techniques for investigating the ground and excited electronic states of chiral systems, but their interpretation is not straightforward and strongly benefits from quantum chemical simulations, provided that the employed computational model is sufficiently accurate and deals properly with stereo-electronic, vibrational averaging and environmental effects. Since the synergy among all these effects is only rarely accounted for, especially for large and flexible organometallic systems, the main aim of this contribution is to illustrate the latest developments of computational approaches rooted into the density functional theory for describing stereo-electronic effects and complemented by effective techniques to deal with vibrational modulation effects and solvatochromic shifts. In this connection, chiral iridium complexes offer an especially suitable case study in view of their bright phosphorescence, which is particularly significant for building effective light emitting diodes (OLEDs) and biomarkers and can be finely tuned by the nature of the metal ligands. For instance, a recently synthesized family of cycloiridiated complexes, KC and KD, bearing a pentahelicenic N-heterocyclic carbene (KB), has shown an enhanced long-lasting, bright phosphorescence. Deeper insights into the still unclear nature and origin of the enhancement could be gained by the interpretation of the chiroptical spectra, which is quite challenging in view of the presence of two sources of chirality, the chiral center on Ir and the chiral axis related to the helicene ligand, in addition to the relativistic effects related to the presence of the Ir center. At the same time, the large dimensions of KC and KD hamper the use of the most sophisticated (but prohibitively expensive) computational models, so that more approximate approaches must be validated on a suitable model compound. To this end, after optimizing the computational scheme on a model system devoid of the helicene moiety (KA), we have performed a comprehensive investigation of the KC and KD spectra, whose interpretation is further aided by novel graphical tools. The discussion and analysis of the results will not be focused on the theoretical background, but, rather, on practical details (specific functional, basis set, vibronic model, solvent regime) with the aim of providing general guidelines for the use of last-generation computational spectroscopy tools also by non-specialists.

COVID-19 Detection by Optimizing Deep Residual Features with Improved Clustering-Based Golden Ratio Optimizer.

The COVID-19 virus is spreading across the world very rapidly. The World Health Organization (WHO) declared it a global pandemic on 11 March 2020. Early detection of this virus is necessary because of the unavailability of any specific drug. The researchers have developed different techniques for COVID-19 detection, but only a few of them have achieved satisfactory results. There are three ways for COVID-19 detection to date, those are real-time reverse transcription-polymerize chain reaction (RT-PCR), Computed Tomography (CT), and X-ray plays. In this work, we have proposed a less expensive computational model for automatic COVID-19 detection from Chest X-ray and CT-scan images. Our paper has a two-fold contribution. Initially, we have extracted deep features from the image dataset and then introduced a completely novel meta-heuristic feature selection approach, named Clustering-based Golden Ratio Optimizer (CGRO). The model has been implemented on three publicly available datasets, namely the COVID CT-dataset, SARS-Cov-2 dataset, and Chest X-Ray dataset, and attained state-of-the-art accuracies of 99.31%, 98.65%, and 99.44%, respectively.

A Bayesian framework for functional calibration of expensive computational models through non-isometric matching

We study statistical calibration, i.e., adjusting features of a computational model that are not observable or controllable in its associated physical system. We focus on functional calibration, which arises in many manufacturing processes where the unobservable features, called calibration variables, are a function of the input variables. A major challenge in many applications is that computational models are expensive and can only be evaluated a limited number of times. Furthermore, without making strong assumptions, the calibration variables are not identifiable. We propose Bayesian Non-isometric Matching Calibration (BNMC) that allows calibration of expensive computational models with only a limited number of samples taken from a computational model and its associated physical system. BNMC replaces the computational model with a dynamic Gaussian process whose parameters are trained in the calibration procedure. To resolve the identifiability issue, we present the calibration problem from a geometric perspective of non-isometric curve to surface matching, which enables us to take advantage of combinatorial optimization techniques to extract necessary information for constructing prior distributions. Our numerical experiments demonstrate that in terms of prediction accuracy BNMC outperforms, or is comparable to, other existing calibration frameworks.

Efficient Aerodynamic Shape Optimization with Deep-Learning-Based Geometric Filtering

Surrogate-based optimization has been used in aerodynamic shape optimization, but it has been limited due to the curse of dimensionality. Although a large number of variables are required for the shape parameterization, many of the shapes that the parameterization can produce are abnormal and do not add meaningful information to a surrogate model. To improve the efficiency of surrogate-based optimization, recent machine learning techniques are applied in this study to reduce the abnormality of both initial and infill samples. This paper proposes a new sampling method for airfoils and wings, which is based on a deep convolutional generative adversarial network. This network is trained to learn the underlying features among the existing airfoils and is able to generate sample airfoils that are notably more realistic than those generated by other sampling methods. In addition, a discriminative model is developed based on convolutional neural networks. This model detects the geometric abnormality of airfoils or wing sections quickly without using expensive computational fluid dynamic models. These machine learning models are embedded in a surrogate-based aerodynamic optimization framework and perform aerodynamic shape optimization for airfoils and wings. The results demonstrate that, compared with the conventional methods, our proposed models can double the optimization efficiency.

Toward accurate prediction of amino acid derivatives structure and energetics from DFT: glycine conformers and their interconversions.

This work provides the accurate reference data for structural and energetic properties relevant for computational studies of amino acids and polypeptides. Glycine due to its small size allows for detailed theoretical explorations of its whole conformational space. The reference energies are computed at the CCSD(T)/CBS+CV level on the best estimated geometries of all local minima and the transition states. For the minima, we complete the set of reference structures reported in Phys. Chem. Chem. Phys., 2013, 15, 10094, by determining the CCSD(T)/(CBS+CV)MP2 quality geometries for the two highest energy conformers VIIp/tcc and VIIIn/gtc. These data stand as the reference to asses reliability and accuracy of less expensive computational models, with particular focus on dispersion-corrected hybrid and double-hybrid DFT approaches, considering several basis sets of double- and triple-ζ quality. Based on results for minima the B2LYP-D3(BJ)/aug-cc-pVTZ level is set as the reference for transition states geometries. Considering accuracy of both single point energies and structural parameters B2PLYP(-D3BJ) and DSDPBEP86 in conjunction with aug-cc-pVTZ basis set can be recommended for reference studies of amino-acids and small poly-peptides, while B3LYP(-D3(BJ)) shown to be the most robust from considered hybrid DFT approaches. Graphical abstract TOC: Benchmarking DFT for amino acid analogues: highly accurate characterization of energies and structures from full glycine conformational space.