Adaptive Importance Sampling Method Research Articles

Two-Phase Adaptive Kriging Model Based Importance Sampling Method for Estimating Time-Dependent Failure Probability

To improve the computational efficiency of the time-dependent failure probability (TDFP) to system with dynamic loads, materials degeneration and other time-varying factors, a two-phase adaptive Kriging model-based importance sampling (IS) method is proposed. The main contribution of this article is that a single Kriging model is established and adaptively refined to gradually approach the optimal IS probability density function (PDF) and estimate TDFP in two sequential phases. In the first phase, a generally approximate IS PDF is constructed by Kriging model based on its prediction characteristic and subsequently updated until convergence. Then, a simple rejection sampling technique is used to robustly generate IS samples for the estimation of TDFP. In the second phase, current Kriging model is subsequently refined for accurately identifying the classification of the IS samples generated in the last phase. Finally, the TDFP can be estimated using the products obtained in the above two phases. As the IS PDF established by Kriging model is used for directly approximating the optimal one, the proposed method has the capacity to deal with the complicated problem containing multiple failure regions. Additionally, since IS procedure can dramatically improve the sampling efficiency, the proposed method is an effective technique to the time-consuming TDFP analysis problems.

Probabilistic analysis of bolted joint anti-loosening under cyclic transverse load

The loosening of bolted joints under cyclic transverse load is a crucial failure mode, and the uncertainty of the relevant parameters of the bolted joint will affect the loosening state. However, there are few bolt-loosening probability analysis and evaluation methods based on parameter uncertainty in previous studies. In this paper, different loosening states are analyzed based on the relationship between pitch torque, thread friction torque, and bearing friction torque. Considering the uncertainty of process parameters, a probabilistic analysis method of the bolted joint anti-loosening under cyclic transverse load is proposed. In the proposed method, the finite element model verified by the fastening experiment is used to verify the accuracy of the bolted joints’ loosening criterion. Then, the loosening criterion is used as the Limit State Function (LSF). According to the LSF, an Adaptive Kriging-Kernel Density Importance Sampling (AK-KDIS) method is proposed for solving the anti-loosening reliability. This method combines adaptive Kriging model with the importance sampling method, effectively reducing the number of sampling samples and the number of LSF calls. The classical and robust Quasi-Monte Carlo (QMC) method is also used to evaluate the reliability of anti-loosening to verify the accuracy of the proposed AK-KDIS method. Finally, the practical application of probabilistic analysis of anti-loosening is presented, and the effectiveness of the proposed method is analyzed in detail. The results indicate that the AK-KDIS method has higher efficiency while ensuring calculation accuracy compared with the QMC method.

A multi-mode failure boundary exploration and exploitation framework using adaptive kriging model for system reliability assessment

Various adaptive reliability analysis methods based on surrogate models have recently been developed. A multi-mode failure boundary exploration and exploitation framework (MFBEEF) was proposed for system reliability assessment using the adaptive kriging model based on sample space partitioning to reduce computational cost and use the characteristics of the failure boundary in multiple failure mode systems. The efficiency of the adaptive construction of kriging model can be improved by using the characteristics of the center sample of the small space to represent the characteristics of all samples in the small space. This method proposes a failure boundary exploration and exploitation strategy and a convergence criterion based on the maximum failure probability error for a system with multiple failure modes to adaptively approximate the failure boundary of a system with multiple failure modes. A multiple-failure-mode learning function was used to identify the optimal training sample to gradually update the kriging model during the failure boundary exploration and exploitation stages. In addition, a complex failure boundary-oriented adaptive hybrid importance sampling method was developed to improve the applicability of the MFBEEF method to small failure probability assessments. Finally, the MFBEEF method was proven to be effective using five system reliability analysis examples: a series system, a parallel system, a series–parallel hybrid system, a multi-dimensional series system with multiple failure modes, and an engineering problem with multiple implicit performance functions.

A Kriging-assisted two-stage adaptive radial-based importance sampling method for random-interval hybrid reliability analysis

A Kriging-assisted two-stage adaptive radial-based importance sampling method for random-interval hybrid reliability analysis

Reliability-based design tool for gas storage in lined rock caverns

ABSTRACT The transition to a fossil-free energy matrix may require large quantities of hydrogen gas, which could be stored efficiently in underground lined rock caverns (LRCs). Since the consequences of failure can be catastrophic, the LRC design needs to have a small failure probability. However, the current deterministic design practice for LRCs limits the possibility to stringently address geotechnical uncertainties. In this paper, a reliability-based design tool is presented for LRCs. The adaptive directional importance sampling (ADIS) method, which requires a relatively small number of samples, is used with a 3D finite element (FE) model to evaluate small probabilities of failure. An illustrative example based on the LRC in Skallen, southwestern Sweden, demonstrates the implementation and applicability of the developed design tool. The considered uncertainties are related to the geological conditions and the steel lining. The results show that the reliability of this LRC design meets the expected safety requirements. Considering different geological conditions with correlations, at least “good” quality rock mass is needed for the LRC design. An additional sensitivity analysis is performed to study the potential influence of corrosion and hydrogen embrittlement, showing that the LRC design could meet safety requirements for a lower category of the weld quality.

Bayesian updating of reliability by cross entropy-based importance sampling

Bayesian analysis enables a consistent updating of the failure probability of engineering systems when new data is available. To this end, we introduce an adaptive importance sampling (IS) method based on the principle of cross entropy (CE) minimization. The key contribution is a novel IS density associated with the posterior probability density function (PDF) of the uncertain parameters, that facilitates efficient sampling from the important region of the failure domain, especially when the failure event is rare. The IS density is designed via a two-step procedure. The first step involves construction of a sample-based approximation of the posterior, which we build using the CE method. Here the aim is to determine the parameters of a chosen parametric distribution family that minimize its Kullback–Leibler divergence from the posterior PDF. The second step of the proposed method constructs the desired IS density for sampling the failure domain through a second round of CE minimization, starting from the approximate posterior obtained in the first step. An adaptive, multi-level approach is employed to solve the two CE optimization problems. The IS densities deduced in the two steps are then applied to construct an efficient estimator for the posterior probability of failure. Through numerical studies, we investigate and demonstrate the efficacy of the method in accurately estimating the reliability of engineering systems with rare failure events.

Sequential Reliability Analysis for the Adjusting Mechanism of Tail Nozzle Considering Wear Degradation

The adjusting mechanism is an important part of an aero engine, and the wear degradation of clearance is widely present in its hinges. In this work, an adjusting mechanism with hinge clearance is analyzed by dynamic simulation and the wear depth is predicted precisely using a wear model. Based on that, a sequential reliability analysis of motion accuracy is carried out. In order to avoid the expensive computational cost of simulation, the adaptive radial-based importance sampling method combined with the adaptive Kriging model (AK-ARBIS) is employed, which describes the decrease of reliability in the standard normal space sphere by sphere with the updated Kriging model. To further utilize the information about each state of wear degradation, the advanced AK-ARBIS method is investigated. Through analytical examples of two typical mechanisms and the engineering application of the adjustment mechanism, the results show that the calculation cost of the sequential reliability analysis under different states can be effectively reduced.

Enhanced Adaptive Kriging Combined Adaptive Radial-Based Importance Sampling Method for Reliability Analysis

The surrogate model-based sampling method has been an active area of research due to its evident efficiency over direct sampling method. The fusion of the adaptive kriging (AK) model with the adaptive radial-based importance sampling (ARBIS) method is developed as an improvement of the AK model combined with Monte Carlo simulation (MCS) method especially for estimating the small failure probabilities. In the AK–ARBIS method, the number of candidate samples in each updating process of the kriging model is reduced through partitioning the whole MCS sample domain into different subdomains and removing the samples inside the last subdomain from the participating candidate sampling pool, which can avoid the time-consuming matrix operation involved in kriging predictions in order to find the best training sample in each updating step. In the existing AK–ARBIS method, kriging model stops updating according to the learning function-based stopping criterion, which cannot directly reflect the error of the estimated failure probability. In this regard, this paper proposes the error-based stopping criterion (ESC) enhanced AK–ARBIS method, where the processes of finding the optimal hypersphere and accurately identifying the sign of candidate samples are decoupled in order to successfully embed the ESC. In addition to embedding the ESC into the AK–ARBIS method, two target-oriented candidate sampling reduction strategies are investigated, respectively, to improve the efficiency of finding the optimal hypersphere and accurately identifying the sign of the participating candidate samples. Based on the candidate sampling pool reduction strategies and ESC, both the computational time and the required number of calls to the real limit state function are reduced over the existing AK–ARBIS method. Three case studies are analyzed to demonstrate the accuracy and efficiency of the proposed enhanced AK–ARBIS method, especially for estimating the small failure probabilities.

Optimized Population Monte Carlo

Adaptive importance sampling (AIS) methods are increasingly used for the approximation of distributions and related intractable integrals in the context of Bayesian inference. Population Monte Carlo (PMC) algorithms are a subclass of AIS methods, widely used due to their ease in the adaptation. In this paper, we propose a novel algorithm that exploits the benefits of the PMC framework and includes more efficient adaptive mechanisms, exploiting geometric information of the target distribution. In particular, the novel algorithm adapts the location and scale parameters of a set of importance densities (proposals). At each iteration, the location parameters are adapted by combining a versatile resampling strategy (i.e., using the information of previous weighted samples) with an advanced optimization-based scheme. Local second-order information of the target distribution is incorporated through a preconditioning matrix acting as a scaling metric onto a gradient direction. A damped Newton approach is adopted to ensure robustness of the scheme. The resulting metric is also used to update the scale parameters of the proposals. We discuss several key theoretical foundations for the proposed approach. Finally, we show the successful performance of the proposed method in three numerical examples, involving challenging distributions.

Series System Reliability of Uncertain Linear Structures under Gaussian Excitation by Cross Entropy–Based Importance Sampling

We present an adaptive importance sampling (IS) method to estimate the reliability of linear structures with parameter uncertainties that are subjected to Gaussian process excitation. Structural failure is defined as a union of multiple first-passage failure events. The main contribution is the introduction of an efficient IS density for the uncertain structural parameters. This density is determined by minimizing the cross-entropy (CE) between the theoretically optimal IS density of the structural parameters and a chosen parametric family of probability distributions. The CE minimization procedure requires evaluation of the system failure probability conditional on fixed values of the uncertain parameters. A closed-form analytical approximation of this conditional failure probability was derived based on an upper bound on the out-crossing rate. Finally, a joint IS density of the random excitation and the uncertain structural parameters was introduced to estimate the series system failure probability involving parameter uncertainties. The accuracy and efficiency of the proposed method was demonstrated by means of two examples: a Gaussian white noise–excited two-story linear shear frame; and a six-story, three-bay moment-resisting steel frame subjected to a filtered nonstationary Gaussian excitation.

Product path guiding with semi-adaptive spatio-directional tree

Monte Carlo integration is an established method in simulating light transport. However, due to its stochastic nature, it requires copious amounts of samples to eliminate the estimation error, and variance reduction techniques such as importance sampling are used to improve the convergence speed as a remedy. Path guiding is a class of adaptive importance sampling methods devised specifically for light transport simulation, which demonstrates significant improvements over classical sampling techniques. Yet most of the previous path guiding methods only account for the radiance term, omitting the bidirectional scattering distribution function (BSDF) term. This results in suboptimal sample quality for non-diffuse light transport. This paper presents a path guiding method which guides paths according to the full product of the BSDF and radiance terms in light transport equation. Extending a spatio-directional tree based path guiding method, we use a semi-adaptive grid-quadtree hybrid subdividing the directional domain in a spatial subdomain to learn and represent the radiance field. With the help of BSDF lookup tables used to accelerate BSDF evaluations, this grid-quadtree allows us to efficiently construct approximate product distributions and guide paths according to the product of incident radiance and cosine-weighted BSDF at each path vertex. The resulting method is relatively simple to implement and enables more robust sampling on scenes with many glossy surfaces.

A Bayesian inference and model selection algorithm with an optimization scheme to infer the model noise power

ABSTRACT Model fitting is possibly the most extended problem in science. Classical approaches include the use of least-squares fitting procedures and maximum likelihood methods to estimate the value of the parameters in the model. However, in recent years, Bayesian inference tools have gained traction. Usually, Markov chain Monte Carlo (MCMC) methods are applied to inference problems, but they present some disadvantages, particularly when comparing different models fitted to the same data set. Other Bayesian methods can deal with this issue in a natural and effective way. We have implemented an importance sampling (IS) algorithm adapted to Bayesian inference problems in which the power of the noise in the observations is not known a priori. The main advantage of IS is that the model evidence can be derived directly from the so-called importance weights – while MCMC methods demand considerable postprocessing. The use of our adaptive target adaptive importance sampling (ATAIS) method is shown by inferring, on the one hand, the parameters of a simulated flaring event that includes a damped oscillation and, on the other hand, real data from the Kepler mission. ATAIS includes a novel automatic adaptation of the target distribution. It automatically estimates the variance of the noise in the model. ATAIS admits parallelization, which decreases the computational run-times notably. We compare our method against a nested sampling method within a model selection problem.

ADAPTIVE IMPORTANCE SAMPLING VIA AUTO-REGRESSIVE GENERATIVE MODELS AND GAUSSIAN PROCESSES.

The quality of importance distribution is vital to adaptive importance sampling, especially in high dimensional sampling spaces where the target distributions are sparse and hard to approximate. This requires that the proposal distributions are expressive and easily adaptable. Because of the need for weight calculation, point evaluation of the proposal distributions is also needed. The Gaussian process has been proven to be a highly expressive non-parametric model for conditional density estimation whose training process is also straightforward. In this paper, we introduce a class of adaptive importance sampling methods where the proposal distribution is constructed in a way that Gaussian processes are combined autoregressively. By numerical experiments of sampling from a high dimensional target distribution, we demonstrate that the method is accurate and efficient compared to existing methods.

Error-based stopping criterion for the combined adaptive Kriging and importance sampling method for reliability analysis

Metamodel-based method is a wise reliability analysis technique because it uses the metamodel to substitute the actual limit state function under the predefined accuracy. Adaptive Kriging (AK) is a famous metamodel in reliability analysis for its flexibility and efficiency. AK combined with the importance sampling (IS) method abbreviate as AK–IS can extremely reduce the size of candidate sampling pool in the updating process of Kriging model, which makes the AK-based reliability method more suitable for estimating the small failure probability. In this paper, an error-based stopping criterion of updating the Kriging model in the AK–IS method is constructed and two considerable maximum relative error estimation methods between the failure probability estimated by the current Kriging model and the limit state function are derived. By controlling the maximum relative error, the accuracy of the estimate can be adjusted flexibly. Results in three case studies show that the error-based stopping criterion based AK–IS method can achieve the predefined accuracy level and simultaneously enhance the efficiency of updating the Kriging model.

Implicitly adaptive importance sampling

Adaptive importance sampling is a class of techniques for finding good proposal distributions for importance sampling. Often the proposal distributions are standard probability distributions whose parameters are adapted based on the mismatch between the current proposal and a target distribution. In this work, we present an implicit adaptive importance sampling method that applies to complicated distributions which are not available in closed form. The method iteratively matches the moments of a set of Monte Carlo draws to weighted moments based on importance weights. We apply the method to Bayesian leave-one-out cross-validation and show that it performs better than many existing parametric adaptive importance sampling methods while being computationally inexpensive.

Enhanced Robust Design Optimization in Seat Belt Anchorage Strength for Front Crash Safety of Multi-Purpose Vehicle

This paper deals with an enhanced robust design optimization (RDO) method and its application to the strength design problem of seat belt anchorage, related to the front crash safety of multi-purpose vehicles. In order to determine the rational design safety of the newly developed automotive part, such as the seat, in which the reliability of the evaluation data is not sufficient at the design stage, it is necessary to implement a probabilistic design considering uncertainties. Thickness size variables of the seat frame structure’s members were considered random design variables, including uncertainties such as manufacturing tolerance, which are an inevitable hazard in the design of automotive parts. Probabilistic constraints were selected from the strength performances of the seat belt anchorage test, which are regulated in Economic Commission for Europe (ECE) and Federal Motor Vehicle Safety Standard (FMVSS), and the strength performances were evaluated by finite element analyses. The RDO problem was formulated such that the random design variables were determined by minimizing the seat frame weight subject to the probabilistic strength performance constraints evaluated from the reliability analyses. Three sigma level quality was considered for robustness in side constraints. The mean value reliability method (MVRM) and adaptive importance sampling method (AISM) were used for the reliability analyses in the RDO, and reliability probabilities from the MVRM and the AISM on the probabilistic optimum design were assessed by Monte Carlo simulation (MCS). The RDO results according to the reliability analysis methods were compared to determine the optimum design results. In the case of the RDO with the AISM, the structure reliability was fully satisfied for all the constraint functions, so the most reliable structural safety was guaranteed for the seat frame design.

Hamiltonian Adaptive Importance Sampling

Importance sampling (IS) is a powerful Monte Carlo (MC) methodology for approximating integrals, for instance in the context of Bayesian inference. In IS, the samples are simulated from the so-called proposal distribution, and the choice of this proposal is key for achieving a high performance. In adaptive IS (AIS) methods, a set of proposals is iteratively improved. AIS is a relevant and timely methodology although many limitations remain yet to be overcome, e.g., the curse of dimensionality in high-dimensional and multi-modal problems. Moreover, the Hamiltonian Monte Carlo (HMC) algorithm has become increasingly popular in machine learning and statistics. HMC has several appealing features such as its exploratory behavior, especially in high-dimensional targets, when other methods suffer. In this letter, we introduce the novel Hamiltonian adaptive importance sampling (HAIS) method. HAIS implements a two-step adaptive process with parallel HMC chains that cooperate at each iteration. The proposed HAIS efficiently adapts a population of proposals, extracting the advantages of HMC. HAIS can be understood as a particular instance of the generic layered AIS family with an additional resampling step. HAIS achieves a significant performance improvement in high-dimensional problems w.r.t. state-of-the-art algorithms. We discuss the statistical properties of HAIS and show its high performance in two challenging examples.

A Likely Magnetic Activity Cycle for the Exoplanet Host M Dwarf GJ 3512

Abstract Current radial velocity data from specialized instruments contain a large amount of information that may pass unnoticed if their analysis is not accurate. The joint use of Bayesian inference tools and frequency analysis has been shown as effective in revealing exoplanets but they have been used less frequently to investigate stellar activity. We intend to use radial velocity data of the exoplanet host star GJ 3512 to investigate its magnetic activity. Our study includes the analysis of the photometric data available. The main objectives of our work are to constrain the orbital parameters of the exoplanets in the system, to determine the current level of activity of the star and to derive an activity cycle length for it. An adaptive importance sampling method was used to determine the parameters of the exoplanets orbit. Generalized Lomb–Scargle periodograms were constructed with both radial velocity curve and photometric data. A careful analysis of the harmonic frequencies was conducted in each periodogram. Our fit to multiple Keplerian orbits constrained the orbital parameters of two giant gas planets orbiting the star GJ 3512. The host star showed an increase of its magnetic activity during the last observing campaign. The accurate fit of the radial velocity curve data to the multi-Keplerian orbit permitted to reveal the star rotation in the residuals of the best fit and estimate an activity cycle length of ∼14 yr.

A new reliability method for small failure probability problems by combining the adaptive importance sampling and surrogate models

Reliability analysis for structural systems with multiple failure modes and expensive-to-evaluate simulations is challenging. In this paper, a new and efficient system reliability method is proposed based on the adaptive importance sampling and kriging models. The Metropolis–Hastings (M–H) algorithm is used to construct several Markov chains to fully explore complex failure regions. A number of Markov chain states are selected as the center of the component importance sampling functions to generate samples for reliability analysis. Based on the component importance sampling function of each selected chain state, the system importance sampling function is constructed with the weighting index. The system importance sampling function can be constructed effectively because it does not involve time-consuming simulations and the most probable point (MPP) search. The new learning function, which is directly linked to the system failure probability, is developed to adaptively select the best added samples for refining the kriging models at each iteration. The adaptive importance sampling method and kriging models are well-combined for system reliability analysis in the proposed method. Compared with existing methods, the proposed method, generally, offers the following advantages: (1) The learning function and stopping criterion are directly linked to system failure probability; (2)the adaptive importance sampling and kriging models are well-combined to yield accurate results based on a small sample size for small failure probability problems; (3) the weights of sampling centers are considered, and the MPP search is not required at each iteration; (4) it is applicable for complex systems regardless of the structure and system failure probability level. Three numerical examples are analyzed, which demonstrate that the proposed method is effective for complex system reliability analysis.

Using Generative Adversarial Networks as a Fast Forward Operator for Hydrogeological Inverse Problems.

Subsurface characterization using inverse techniques constitutes one of the fundamental elements of hydrogeological modeling applications. Available methods to solve inverse problems rely on a forward operator that predicts state variables for a given set of subsurface parameters. As the number of model parameters to be estimated increases, forward operations incur a significant computational demand. In this paper, we investigate the use of conditional generative adversarial networks (cGAN) as an emulator for the forward operator in the context of a hydrogeological inverse problem. We particularly investigate if the cGAN can be used to replace the forward operator used in the adaptive importance sampling method posterior population expansion (PoPEx) with reasonable accuracy and feasible computation requirement. The cGAN model trained on channelized geological structures has shown that the cGAN is able to reproduce the state variables corresponding to a certain parameter field. Hence, its integration in PoPEx yielded satisfactory results. In terms of the computational demand, the use of cGAN as a surrogate forward model reduces the required computational time up to 80% for the problem defined in the study. However, the training time required to create a model seems to be the major drawback of the method.