Importance Sampling Simulation Research Articles

An improved sequential importance sampling method for structural reliability analysis of high dimensional problems

Reliability analysis for rare events has proven to be a significant challenge, particularly in non-linear and complex scenarios. Despite the effectiveness of Sequential Importance Sampling (SIS) and Subset Simulation (SUS) in addressing high-dimensional small failure probability problems, the computational burden associated with mechanical simulations for complex structures remains substantial. To address this issue, this paper introduces a novel approach referred to as SIS and Kriging Metamodel Integration (SISAK), designed for computing small failure probabilities in high dimensions. This method involves generating a small number of samples from the original distribution to construct an adaptive metamodel, which is then iteratively optimized using a series of intermediate distributions introduced by SIS. By substituting the function for intermediate distributions with a continuously optimized metamodel, the enhanced approach significantly reduces the computational load associated with mechanical model evaluations compared to the direct use of SIS. This approach does not depend on determining the most probable failure point and does not require consideration of the failure domain shape. Additionally, it inherits the advantages of SIS in addressing high-dimensional small failure probabilities, making it well-suited for structural reliability analysis of nonlinear systems, discontinuous failure domains, cascading functions, and high-dimensional problems.

Estimating Tail Probability in MMPP/D/1 Queue with Importance Sampling by Service Rate Adjustments

The Asynchronous Transfer Mode (ATM) is an efficient technology for call relays, and it transmits information from multiple services including data, video, or voice. This information is conveyed at ATM multiplexers in small fixed-size packets called cells. The acceptable cell loss probability at ATM multiplexers is about 10−12. Important Sampling (IS) is an efficient method for estimating tiny probabilities that cannot be achieved by traditional Monte Carlo (MC) methods. This research presents a novel approach for evaluating the tail probability in the MMPP/D/1 queue system utilizing importance sampling simulation in the ATM network. To generate more rare events, a virtual queue is implemented in the dequeue process by decreasing the processing rate in the queue. In this way, the tail probability can be estimated on a real-time network.

Meta-model based sequential importance sampling method for structural reliability analysis under high dimensional small failure probability

Reliability analysis poses a significant challenge for complex structures with stringent reliability requirements. While Sequential Importance Sampling (SIS) and Subset Simulation (SUS) have proven highly effective in addressing high-dimensional problems with small failure probabilities, the computational burden of mechanical simulations remains substantial due to the time-consuming nature of numerical simulation processes. Consequently, this paper introduces a novel approach, denoted as AK-SIS, which combines SIS with Kriging metamodeling specifically designed to address computational challenges associated with small failure probabilities. The fundamental principle of this approach involves utilizing AK-MCS technology (Echard et al., 2011) [1] as a precursor to the SIS approach to initially generate metamodels. These metamodels are then employed in lieu of performance functions in subsequent steps, significantly reducing the number of function calls required to simulate complex engineering problems when applying SIS techniques directly. By inheriting the advantages of SIS, AK-SIS has demonstrated its suitability for reliability analysis in scenarios involving high-dimensional spaces and small fault probabilities. Furthermore, AK-SIS is not limited by the shape of the failure domain, eliminates the need to solve the design point, and is particularly well-suited for analyzing reliability in cases of discontinuous failure domains, multiple failure domains, as well as complex failure domains and rare events. The efficacy of AK-SIS is substantiated through rigorous evaluation encompassing nonlinear, high-dimensional examples, and an engineering application. These empirical validations collectively contribute to a robust methodological framework for reliability analysis of intricate structures characterized by stringent reliability requirements.

Generic fully coupled framework for reliability assessment of offshore wind turbines under typical limit states

Offshore wind turbines (OWTs) generally experience harsh marine environments with uncertainties in marine loads, soil properties, and material properties; therefore, the structural integrity assessment of OWTs is a challenging task. Most of the previously proposed methods for reliability assessment are time-consuming and neglect the coupling effects of aerodynamics, hydrodynamics, and OWT structural dynamics under complex environmental loads. In this study, a fully coupled framework for the reliability assessment of OWTs is proposed, in which a dynamic simulation tool in the time domain is combined with surrogate models. The established Kriging, multivariate regression, and support vector regression surrogate models are compared to determine the most feasible model. Furthermore, the influence of different solving methods, such as first-order reliability method (FORM), importance sampling (IS), and subset simulation (SS), on the estimation of structural reliability indexes are discussed. The superiority of the estimated reliability indexes in the safety assessment of OWTs is proved using the SS method based on the Kriging surrogate model. Subsequently, the potential failure modes of an OWT under different operational states are revealed, indicating that excessive tower vibrations must be eliminated for stable power production from OWTs. Furthermore, the SS method is found to be feasible as it ensures the estimation accuracy of the reliability indexes of the parked OWT based on Kriging and the other surrogate models. Finally, the significance of soil–structure interactions in the reliability index estimation is investigated, and the equivalent coupled spring boundary is proposed from the perspectives of structural safety and simulation accuracy.

Accurate Multi-segment Probability Density Estimation Through Moment Matching

The impact of process variations continues to grow as transistor feature size shrinks. Such variations in transistor parameters lead to variations and unpredictability in circuit output and may ultimately cause them to violate specifications leading to circuit failure. In fact, timely failures in critical circuits may lead to catastrophic failures in the entire chip. As such, statistical modeling of circuit behavior is becoming increasingly important. However, existing statistical circuit simulation approaches fail to accurately and efficiently analyze the high sigma behavior of probabilistic circuit output. To this end, we propose PDM (Piecewise Distribution Model) - a piecewise distribution modeling approach via moment matching using maximum entropy to model the high sigma behavior of analog/mixed-signal (AMS) circuit probability distributions. PDM is independent of the number of input dimensions and matches region specific probabilistic moments which allows for significantly greater accuracy compared to other moment matching approaches. PDM also utilizes Spearman’s rank correlation coefficient to select the optimal approximation for the tail of the distribution. Experiments on a known mathematical distribution and various circuits obtain accurate results up to 4.8 sigma with 2-3 orders of speedup relative to Monte Carlo. PDM also demonstrates better accuracy while compared against other state-of-the-art statistical modeling approaches, such as maximum entropy, importance sampling, and subset simulation.

Modified control variates method based on second-order saddle-point approximation for practical reliability analysis

Abstract. A novel method is presented for efficiently analyzing the reliability of engineering components and systems with highly nonlinear complex limit state functions. The proposed method begins by transforming the integral of the limit state function into an integral of a highly correlated limit state function using the control variates method. The second-order reliability method is then employed within the control variates framework to approximate the highly correlated limit state function as a quadratic polynomial. Subsequently, the probability of failure is obtained through the estimation of the saddle-point approximation and a small number of samples generated by Latin hypercube sampling. To demonstrate the effectiveness of the proposed method, four examples involving mathematical functions and mechanical problems are solved. The results are compared with those obtained using the second-order reliability method (SORM), control variates based on Monte Carlo simulation (CVMCS) with second-order saddle-point approximation (SOSPA), importance sampling (IS) and Monte Carlo simulation (MCS). The findings demonstrate that, while maintaining high-precision reliability results, the proposed method significantly reduces the number of evaluations of the limit state function through a small number of initial samples. The method is capable of efficiently and accurately solving complex practical engineering reliability problems.

A Best Balance Ratio Ordered Feature Selection Methodology for Robust and Fast Statistical Analysis of Memory Designs

Recently, machine learning yield models for Integrated Circuit (IC) have gained widespread prominence in the EDA community, and are very promising in terms of emulating memory design functionality and thereby speeding up circuit simulation based variance reduction methods. A main challenge that arises in this area is class imbalance that occurs naturally due to the high targeted manufacturing yield. Thus, the imbalanced nature of the sampled memory datasets can compromise the model performance. In this work, we attain deep insights into the memory classification problem for modeling rare fail events in the context of importance sampling based yield analysis. We propose a comprehensive and computationally efficient method that addresses the joint considerations of the best combination of relevant features and class balance ratios, which are key for classifier generalization capability. The methodology relies on synthetic minority oversampling techniques to enforce the minority class while probing for the best data balance ratio in conjunction with an iterative L1-SVM based approach that qualifies as an approximation to the L0-norm regularization for the best feature subset selection. We compare the proposed methodology against standalone L1-SVM solutions, unbalanced L0-norm approximation as well as an algorithmic data balancing method in the context of yield estimation methodology. The methodology is shown to result in high fidelity classifiers as demonstrated when analyzing the yield of a 14nm FinFET SRAM cross-section with speedup of 179x for the importance sampling simulations compared to pure circuit simulation based approaches and an average error of 0.19 sigma.

Reliability analysis of bridge girders based on regular vine Gaussian copula model and monitored data

For improving the reliability analysis of bridge girders, a consideration of the nonlinear dependence among multivariable random variables is essential. Thus, this study presents a new reliability analysis method for bridge girders, and the regular vine Gaussian copula model (RVGCM) combining the regular vine structure with multiple bivariate Gaussian copulas is established for modeling the nonlinear dependence between the failure modes at different control monitoring points. With the established RVGCM, the failure problem of the bridge girder can be simplified into a series of bivariate failure problems. The nonlinear dependence among different pairs of variables, which may affect the reliability evaluation, can be captured and described. The feasibility of the proposed method is demonstrated using a practical bridge girder with multiple control monitoring points. The reliability analysis results using the RVGCM, statistical independent method (statistical independence between the failure modes), and Importance Sampling (IS) simulation approach are compared. It is proven that the RVGCM is superior in modeling dependence between the failure modes for reliability analysis of bridge girder. The reliability analysis results of the RVGCM are closer to the benchmark results obtained from IS than the statistical independent method, and the calculation efficiency is better than IS. In addition, the results of the proposed method are consistent with the inspection results of actual bridges, which proves the practical application potential of the proposed method.

MLCV: Bridging Machine-Learning-Based Dimensionality Reduction and Free-Energy Calculation.

Importance-sampling algorithms leaning on the definition of a model reaction coordinate (RC) are widely employed to probe processes relevant to chemistry and biology alike, spanning time scales not amenable to common, brute-force molecular dynamics (MD) simulations. In practice, the model RC often consists of a handful of collective variables (CVs) chosen on the basis of chemical intuition. However, constructing manually a low-dimensional RC model to describe an intricate geometrical transformation for the purpose of free-energy calculations and analyses remains a daunting challenge due to the inherent complexity of the conformational transitions at play. To solve this issue, remarkable progress has been made in employing machine-learning techniques, such as autoencoders, to extract the low-dimensional RC model from a large set of CVs. Implementation of the differentiable, nonlinear machine-learned CVs in common MD engines to perform free-energy calculations is, however, particularly cumbersome. To address this issue, we present here a user-friendly tool (called MLCV) that facilitates the use of machine-learned CVs in importance-sampling simulations through the popular Colvars module. Our approach is critically probed with three case examples consisting of small peptides, showcasing that through hard-coded neural network in Colvars, deep-learning and enhanced-sampling can be effectively bridged with MD simulations. The MLCV code is versatile, applicable to all the CVs available in Colvars, and can be connected to any kind of dense neural networks. We believe that MLCV provides an effective, powerful, and user-friendly platform accessible to experts and nonexperts alike for machine-learning (ML)-guided CV discovery and enhanced-sampling simulations to unveil the molecular mechanisms underlying complex biochemical processes.

Importance sampling for Markovian tandem queues using subsolutions: exploring the possibilities

We consider importance sampling simulation for estimating the probability of reaching large total number of customers in an [Formula: see text] tandem queue, during a busy cycle of the system. Our main result is a procedure for obtaining a family of asymptotically efficient changes of measure based on subsolutions. We explicitly show these families for two-node tandem queues and we find that there exist more asymptotically efficient changes of measure based on subsolutions than currently available in literature.

Solving high-dimensional Hamilton\u2013Jacobi\u2013Bellman PDEs using neural networks: perspectives from the theory of controlled diffusions and measures on path space

Optimal control of diffusion processes is intimately connected to the problem of solving certain Hamilton–Jacobi–Bellman equations. Building on recent machine learning inspired approaches towards high-dimensional PDEs, we investigate the potential of iterative diffusion optimisation techniques, in particular considering applications in importance sampling and rare event simulation, and focusing on problems without diffusion control, with linearly controlled drift and running costs that depend quadratically on the control. More generally, our methods apply to nonlinear parabolic PDEs with a certain shift invariance. The choice of an appropriate loss function being a central element in the algorithmic design, we develop a principled framework based on divergences between path measures, encompassing various existing methods. Motivated by connections to forward-backward SDEs, we propose and study the novel log-variance divergence, showing favourable properties of corresponding Monte Carlo estimators. The promise of the developed approach is exemplified by a range of high-dimensional and metastable numerical examples.

Efficient procedure for failure probability function estimation in augmented space

An efficient procedure is proposed to estimate the failure probability function (FPF) with respect to design variables, which correspond to distribution parameters of basic structural random variables. The proposed procedure is based on the concept of an augmented reliability problem, which assumes the design variables as uncertain by assigning a prior distribution, transforming the FPF into an expression that includes the posterior distribution of those design variables. The novel contribution of this work consists of expressing this target posterior distribution as an integral, allowing it to be estimated by means of sampling, and no distribution fitting is needed, leading to an efficient estimation of FPF. The proposed procedure is implemented within three different simulation strategies: Monte Carlo simulation, importance sampling and subset simulation; for each of these cases, expressions for the coefficient of variation of the FPF estimate are derived. Numerical examples illustrate performance of the proposed approaches.

Sensitivity of Sample for Simulation-Based Reliability Analysis Methods

In structural reliability analysis, simulation methods are widely used. The statistical characteristics of failure probability estimate of these methods have been well investigated. In this study, the sensitivities of the failure probability estimate and its statistical characteristics with regard to sample, called ‘contribution indexes’, are proposed to measure the contribution of sample. The contribution indexes in four widely simulation methods, i.e., Monte Carlo simulation (MCS), importance sampling (IS), line sampling (LS) and subset simulation (SS) are derived and analyzed. The proposed contribution indexes of sample can provide valuable information understanding the methods deeply, and enlighten potential improvement of methods. It is found that the main differences between these investigated methods lie in the contribution indexes of the safety samples, which are the main factors to the efficiency of the methods. Moreover, numerical examples are used to validate these findings.

Multilevel Sequential Importance Sampling for Rare Event Estimation

The estimation of the probability of rare events is an important task in reliability and risk assessment. We consider failure events that are expressed in terms of a limit state function, which depends on the solution of a partial differential equation (PDE). Since numerical evaluations of PDEs are computationally expensive, estimating such probabilities of failure by Monte Carlo sampling is intractable. More efficient sampling methods from reliability analysis, such as Subset Simulation, are popular, but can still be impracticable if the PDE evaluations are very costly. In this article, we develop a novel, highly efficient estimator for probabilities of rare events. Our method is based on a Sequential Importance sampler using discretizations of PDE-based limit state functions with different accuracies. A twofold adaptive algorithm ensures that we obtain an estimate based on the desired discretization accuracy. In contrast to the Multilevel Subset Simulation estimator of [Ullmann, Papaioannou 2015; SIAM/ASA J. Uncertain. Quantif. 3(1):922-953], our estimator overcomes the nestedness problem. Of particular importance in Sequential Importance sampling algorithms is the correct choice of the MCMC kernel. Instead of the popular adaptive conditional sampling method, we propose a new algorithm that uses independent proposals from an adaptively constructed von Mises-Fischer-Nakagami distribution. The proposed algorithm is applied to test problems in 1D and 2D space, respectively, and is compared to the Multilevel Subset Simulation estimator and to single-level versions of Sequential Importance Sampling and Subset Simulation.

Value of Information Analysis via Active Learning and Knowledge Sharing in Error-Controlled Adaptive Kriging

Large uncertainties in many phenomena have challenged decision making. Collecting additional information to better characterize reducible uncertainties is among decision alternatives. Value of information (VoI) analysis is a mathematical decision framework that quantifies expected potential benefits of new data and assists with optimal allocation of resources for information collection. However, analysis of VoI is computational very costly because of the underlying Bayesian inference especially for equality-type information. This paper proposes the first surrogate-based framework for VoI analysis. Instead of modeling the limit state functions describing events of interest for decision making, which is commonly pursued in surrogate model-based reliability methods, the proposed framework models system responses. This approach affords sharing equality-type information from observations among surrogate models to update likelihoods of multiple events of interest. Moreover, two knowledge sharing schemes called model and training points sharing are proposed to most effectively take advantage of the knowledge offered by costly model evaluations. Both schemes are integrated with an error rate-based adaptive training approach to efficiently generate accurate Kriging surrogate models. The proposed VoI analysis framework is applied for an optimal decision-making problem involving load testing of a truss bridge. While state-of-the-art methods based on importance sampling and adaptive Kriging Monte Carlo simulation are unable to solve this problem, the proposed method is shown to offer accurate and robust estimates of VoI with a limited number of model evaluations. Therefore, the proposed method facilitates the application of VoI for complex decision problems.

Importance sampling for non-Markovian tandem queues using subsolutions

In this paper, we use importance sampling simulation to estimate the probability that the number of customers in a d-node GI|GI|1 tandem queue reaches some high level N in a busy cycle of the system. We present a state-dependent change of measure for a d-node GI|GI|1 tandem queue based on the subsolution approach, and we prove, under a mild conjecture, that this state-dependent change of measure gives an asymptotically efficient estimator for the probability of interest when all supports are bounded.

Global reliability sensitivity estimation based on failure samples

Global reliability sensitivity analysis (RSA) can help to assess the effects of input random variables X on the probability of failure Pr(F) of an engineering system. Conventionally, this requires repeated evaluations of the conditional failure probability Pr(F|Xi = xi) for multiple values of the input random variable Xi and for all Xi of interest. Such a solution is straightforward but computationally expensive. In this paper, we propose a new method to perform global RSA, which requires as an input only samples of X that fall in the failure domain. Such samples are a by-product of many sampling-based reliability analysis methods. The proposed method constructs the Pr(F|Xi = xi) by application of Bayes’ rule, based on the probability density function (PDF) of X conditioned on system failure F. This conditional PDF is approximated with a kernel density estimation from the failure samples. In this way, the reliability sensitivities of all the input random variables can be computed following a sampling-based reliability analysis with no additional computation cost. The approach is investigated on numerical examples in conjunction with crude Monte Carlo simulation, importance sampling and subset simulation. The results demonstrate the computational advantages over existing single-loop sampling methods for global RSA.

Exhaustive state-to-state cross sections for reactive molecular collisions from importance sampling simulation and a neural network representation

Exhaustive state-to-state cross sections for reactive molecular collisions from importance sampling simulation and a neural network representation

Exhaustive state-to-state cross sections for reactive molecular collisions from importance sampling simulation and a neural network representation.

High-temperature, reactive gas flow is inherently nonequilibrium in terms of energy and state population distributions. Modeling such conditions is challenging even for the smallest molecular systems due to the extremely large number of accessible states and transitions between them. Here, neural networks (NNs) trained on explicitly simulated data are constructed and shown to provide quantitatively realistic descriptions which can be used in mesoscale simulation approaches such as Direct Simulation Monte Carlo to model gas flow at the hypersonic regime. As an example, the state-to-state cross sections for N(4S) + NO(2Π) → O(3P) + N2(X1Σg +) are computed from quasiclassical trajectory (QCT) simulations. By training NNs on a sparsely sampled noisy set of state-to-state cross sections, it is demonstrated that independently generated reference data are predicted with high accuracy. State-specific and total reaction rates as a function of temperature from the NN are in quantitative agreement with explicit QCT simulations and confirm earlier simulations, and the final state distributions of the vibrational and rotational energies agree as well. Thus, NNs trained on physical reference data can provide a viable alternative to computationally demanding explicit evaluation of the microscopic information at run time. This will considerably advance the ability to realistically model nonequilibrium ensembles for network-based simulations.

Directional importance sampling simulation for structural reliability analysis using a procedure to update structural probability estimator through a repetitive simulation method

Directional importance sampling simulation for structural reliability analysis using a procedure to update structural probability estimator through a repetitive simulation method