Rare Event Probabilities Research Articles

A Comparison of Weighted Stochastic Simulation Methods for the Analysis of Genetic Circuits.

Rare events are of particular interest in synthetic biology because rare biochemical events may be catastrophic to a biological system by, for example, triggering irreversible events such as off-target drug delivery. To estimate the probability of rare events efficiently, several weighted stochastic simulation methods have been developed. Under optimal parameters and model conditions, these methods can greatly improve simulation efficiency in comparison to traditional stochastic simulation. Unfortunately, the optimal parameters and conditions cannot be deduced a priori. This paper presents a critical survey of weighted stochastic simulation methods. It shows that the methods considered here cannot consistently, efficiently, and exactly accomplish the task of rare event simulation without resorting to a computationally expensive calibration procedure, which undermines their overall efficiency. The results suggest that further development is needed before these methods can be deployed for general use in biological simulations.

Large Deviations for Additive Functionals of Reflected Jump-Diffusions

We consider a jump-diffusion process on a bounded domain with reflection at the boundary, and establish long-term results for a general additive process of its path. This includes the long-term behaviour of its occupation time and its local time on the boundaries. We derive a characterization of the large deviation rate function which quantifies the rate of exponential decay of probabilities of rare events for paths of the process. The characterization relies on a solution of a partial integro-differntial equation (PIDE) with boundary constraints. We develop a practical implementation of our results in terms of a numerical solution for the PIDE. We illustrate the method on a few standard examples (Brownian motion, birth-death processes) and on a particular jump-diffusion model arising from applications to biochemical reactions.

Inferring turbulent environments via machine learning.

The problem of classifying turbulent environments from partial observation is key for some theoretical and applied fields, from engineering to earth observation and astrophysics, e.g., to precondition searching of optimal control policies in different turbulent backgrounds, to predict the probability of rare events and/or to infer physical parameters labeling different turbulent setups. To achieve such goal one can use different tools depending on the system's knowledge and on the quality and quantity of the accessible data. In this context, we assume to work in a model-free setup completely blind to all dynamical laws, but with a large quantity of (good quality) data for training. As a prototype of complex flows with different attractors, and different multi-scale statistical properties we selected 10 turbulent 'ensembles' by changing the rotation frequency of the frame of reference of the 3d domain and we suppose to have access to a set of partial observations limited to the instantaneous kinetic energy distribution in a 2d plane, as it is often the case in geophysics and astrophysics. We compare results obtained by a machine learning (ML) approach consisting of a state-of-the-art deep convolutional neural network (DCNN) against Bayesian inference which exploits the information on velocity and entropy moments. First, we discuss the supremacy of the ML approach, presenting also results at changing the number of training data and of the hyper-parameters. Second, we present an ablation study on the input data aimed to perform a ranking on the importance of the flow features used by the DCNN, helping to identify the main physical contents used by the classifier. Finally, we discuss the main limitations of such data-driven methods and potential interesting applications.

Intercomparison of regional loss estimates from global synthetic tropical cyclone models

Tropical cyclones (TCs) cause devastating damage to life and property. Historical TC data is scarce, complicating adequate TC risk assessments. Synthetic TC models are specifically designed to overcome this scarcity. While these models have been evaluated on their ability to simulate TC activity, no study to date has focused on model performance and applicability in TC risk assessments. This study performs the intercomparison of four different global-scale synthetic TC datasets in the impact space, comparing impact return period curves, probability of rare events, and hazard intensity distribution over land. We find that the model choice influences the costliest events, particularly in basins with limited TC activity. Modelled direct economic damages in the North Indian Ocean, for instance, range from 40 to 246 billion USD for the 100-yr event over the four hazard sets. We furthermore provide guidelines for the suitability of the different synthetic models for various research purposes.

Reliability analysis of discrete-state performance functions via adaptive sequential sampling with detection of failure surfaces

The paper presents a new efficient and robust method for rare event probability estimation for computational models of an engineering product or a process returning categorical information only, for example, either success or failure. For such models, most of the methods designed for the estimation of failure probability, which use the numerical value of the outcome to compute gradients or to estimate the proximity to the failure surface, cannot be applied. Even if the performance function provides more than just binary output, the state of the system may be a non-smooth or even a discontinuous function defined in the domain of continuous input variables. This often happens because the mathematical model features non-smooth components or discontinuities (e.g., in the constitutive laws), bifurcations, or even domains in which no reasonable model response is obtained. In these cases, the classical gradient-based methods usually fail. We propose a simple yet efficient algorithm, which performs a sequential adaptive selection of points from the input domain of random variables to extend and refine a simple distance-based surrogate model. Two different tasks can be accomplished at any stage of sequential sampling: (i) estimation of the failure probability, and (ii) selection of the best possible candidate for the subsequent model evaluation if further improvement is necessary. The proposed criterion for selecting the next point for model evaluation maximizes the expected probability classified by using the candidate. Therefore, the perfect balance between global exploration and local exploitation is maintained automatically. If there are more rare events such as failure modes, the method can be generalized to estimate the probabilities of all these event types. Moreover, when the numerical value of model evaluation can be used to build a smooth surrogate, the algorithm can accommodate this information to increase the accuracy of the estimated probabilities. Lastly, we define a new simple yet general geometrical measure of the global sensitivity of the rare-event probability to individual variables, which is obtained as a by-product of the proposed refinement algorithm.

Adaptive sequential sampling for reliability estimation of binary functions

A novel method for estimation of rare event probability is proposed, which works also for computational models returning categorical information only: success or failure. It combines the robustness of simulation methods (counting failure events) with the strength of approximation methods which refine the boundary between the failure and safe sets. Two basic tasks are identified: (i) extension of the experimental design (ED) and (ii) estimation of probabilities. The new extension algorithm adds points for limit state evaluation to the ED by balancing the global exploration and local exploitation, and the estimation uses the pointwise information to build a simple surrogate and perform a novel optimized importance sampling. No connection is presumed between the limit function value at point and its proximity to the failure surface. A new global sensitivity measure of the failure probability to individual variables is proposed and obtained as a by-product of the proposed methods.

Numerical solution of the Fokker–Planck equation using physics-based mixture models

The Fokker–Planck equation governs the uncertainty propagation of dynamical systems driven by stochastic processes. The solution of the Fokker–Planck equation is a time-varying Probability Density Function (PDF) that is usually of high dimension with unbounded support. There are also properties that must be conserved in time for the joint and any marginal solution PDF (i.e., a time-varying PDF is a non-negative function that must integrate to unity at any given time.) Satisfying these properties poses a significant challenge to the numerical solution of the Fokker–Planck equation. Another challenge is to capture the tail behavior of the solution PDF with unbounded support, required to predict the probability of rare events like a system failure. Satisfying the conditions of a proper PDF with unbounded support limits the application of traditional grid-based numerical methods, like finite difference and finite element methods. This paper develops a novel numerical method based on physics-based mixture models for the transient and steady-state solutions of the Fokker–Planck equation. In the proposed numerical method, the trial function space consists of the convex combinations of some parametric PDFs (i.e., mixture models) that trivially satisfy the necessary conditions of the solution. Estimating the unknown parameters of the mixture model is via Bayesian inference while considering the constraints on model parameters. Bayesian inference facilitates integrating data on the responses of dynamical systems (e.g., from simulations) with the governing Fokker–Planck equation to estimate the unknown parameters. Since the solution PDF is not observable, combining it with observable data on system responses is far from trivial based on current numerical methods. The formulation of Bayesian inference also introduces a weighting function to reduce the estimation error in specific regions of interest like the tail of the solution PDF. To reduce the computational demand, the paper develops an importance sampling algorithm that generates a small set of collocation points at which the residual of the Fokker–Planck equation is evaluated. The performance of the proposed numerical method is demonstrated with several benchmark problems. The obtained results are compared with analytical solutions when available and otherwise with simulations.

A quadratic Wiener path integral approximation for stochastic response determination of multi-degree-of-freedom nonlinear systems

A Wiener path integral (WPI) technique is developed for determining the stochastic response of multi-degree-of-freedom (MDOF) nonlinear systems. Specifically, the nonlinear system response joint transition probability density function (PDF) is expressed as a WPI over the space of paths satisfying the initial and final conditions in time. Next, a functional series expansion is considered for the WPI and a quadratic approximation is employed. Further, relying on a variational principle yields a functional optimization problem to be solved for the most probable path, which is used for determining approximately the joint response transition PDF. It is shown that compared to the standard (semiclassical) WPI solution approach, which accounts only for the most probable path, the quadratic approximation developed herein exhibits enhanced accuracy. This is due to the fact that fluctuations around the most probable path are also accounted for by considering a localized state-dependent factor in the calculation of the WPI. Furthermore, the PDF normalization step of the most probable path approach is bypassed, and thus, probabilities of rare events (e.g., failures) can be determined in a direct manner without the need for obtaining the complete joint response PDF first. The herein developed technique can be construed as an extension of earlier efforts in the literature to account for MDOF systems. Several numerical examples are considered for demonstrating the accuracy of the technique. These pertain to various dynamical systems exhibiting diverse nonlinear behaviors. Comparisons with pertinent Monte Carlo simulation data are included as well.

Importance Sampling for a Simple Markovian Intensity Model Using Subsolutions

This article considers importance sampling for estimation of rare-event probabilities in a specific collection of Markovian jump processes used for, e.g., modeling of credit risk. Previous attempts at designing importance sampling algorithms have resulted in poor performance and the main contribution of the article is the design of efficient importance sampling algorithms using subsolutions. The dynamics of the jump processes cause the corresponding Hamilton-Jacobi equations to have an intricate state-dependence, which makes the design of efficient algorithms difficult. We provide theoretical results that quantify the performance of importance sampling algorithms in general and construct asymptotically optimal algorithms for some examples. The computational gain compared to standard Monte Carlo is illustrated by numerical examples.

Importance sampling for maxima on trees

We study the all-time supremum of the perturbed branching random walk, known to be the endogenous solution to the high-order Lindley equation: W=DmaxY,max1≤i≤N(Wi+Xi),where the {Wi} are independent copies of W, independent of the random vector (Y,N,{Xi}) taking values in R×N×R∞. Under Kesten assumptions, this solution satisfies P(W>t)∼He−αt,t→∞,where α>0 solves the Cramér–Lundberg equation E∑i=1NeαXi=1. This paper establishes the tail asymptotics of W by using the forward iterations of the map defining the fixed-point equation combined with a change of measure along a randomly chosen path. This new approach provides an explicit representation of the constant H and gives rise to unbiased and strongly efficient estimators for the rare event probabilities P(W>t).

Rare event estimation using stochastic spectral embedding

Estimating the probability of rare failure events is an essential step in the reliability assessment of engineering systems. Computing this failure probability for complex non-linear systems is challenging, and has recently spurred the development of active-learning reliability methods. These methods approximate the limit-state function (LSF) using surrogate models trained with a sequentially enriched set of model evaluations. A recently proposed method called stochastic spectral embedding (SSE) aims to improve the local approximation accuracy of global, spectral surrogate modelling techniques by sequentially embedding local residual expansions in subdomains of the input space. In this work we apply SSE to the LSF, giving rise to a stochastic spectral embedding-based reliability (SSER) method. The resulting partition of the input space decomposes the failure probability into a set of easy-to-compute conditional failure probabilities. We propose a set of modifications that tailor the algorithm to efficiently solve rare event estimation problems. These modifications include specialized refinement domain selection, partitioning and enrichment strategies. We showcase the algorithm performance on four benchmark problems of various dimensionality and complexity in the LSF.

Efficient subset simulation for rare-event integrating point-evolution kernel density and adaptive polynomial chaos kriging

Rare-event probability estimation has a wide range of applications, including the design and manufacture of precision equipment, aerospace systems, and critical industrial and civil structures. However, traditional simulation-based reliability calculation methods, such as brute Monte Carlo simulation (MCS) and subset simulation (SS), face challenges in efficiently evaluating small-failure probabilities due to the need for a large number of simulations, especially for non-linear and complex scenarios. Thus, to efficiently assess the probability of rare failure events in structural engineering, this paper develops a novel method for assessing the small-failure probability by integrating the point-evolution kernel density (PKDE), SS, and polynomial chaos kriging (PCK). The proposed PKDE-Adaptive PCK-based SS (PAPS) method aims to reduce the implementation of the original performance function by PCK and enrich the training set using an adaptive strategy. Moreover, the initial cumulative distribution function (CDF) of the performance function estimated by PKDE is modified gradually to facilitate the estimation of small-failure probability. Four numerical examples of small-failure probability estimation involving classical analytical cases, time-variant cases, and non-linear stochastic structures are used to illustrate the accuracy and efficiency of the proposed method. The computational results show that the proposed method can provide accurate computational results with a smaller computational burden than traditional methods (e.g., MCS, SS, LHS-PCK-SS).

Machine learning-based methods in structural reliability analysis: A review

Structural Reliability analysis (SRA) is one of the prominent fields in civil and mechanical engineering. However, an accurate SRA in most cases deals with complex and costly numerical problems. Machine learning-based (ML) techniques have been introduced to the SRA problems to deal with this huge computational cost and increase accuracy. This paper presents a review of the development and use of ML models in SRA. The review includes the most common types of ML methods used in SRA. More specifically, the application of artificial neural networks (ANN), support vector machines (SVM), Bayesian methods and Kriging estimation with active learning perspective in SRA are explained, and a state-of-the-art review of the prominent literature in these fields is presented. Aiming towards a fast and accurate SRA, the ML techniques adopted for the approximation of the limit state function with Monte Carlo simulation (MCS), first/second-order reliability methods (FORM/SORM) or MCS with importance sampling well as the methods for efficiently computing the probabilities of rare events in complex structural systems. In this regard, the focus of the current manuscript is on the different models’ structures and diverse applications of each ML method in different aspects of SRA. Moreover, imperative considerations on the management of samples in the Monte Carlo simulation for SRA purposes and the treatment of the SRA problem as pattern recognition or classification task are provided. This review helps the researchers in civil and mechanical engineering, especially those who are focused on reliability and structural analysis or dealing with product assurance problems.

Unbiased Simulation of Rare Events in Continuous Time

For rare events described in terms of Markov processes, truly unbiased estimation of the rare event probability generally requires the avoidance of numerical approximations of the Markov process. Recent work in the exact and varepsilon-strong simulation of diffusions, which can be used to almost surely constrain sample paths to a given tolerance, suggests one way to do this. We specify how such algorithms can be combined with the classical multilevel splitting method for rare event simulation. This provides unbiased estimations of the probability in question. We discuss the practical feasibility of the algorithm with reference to existing varepsilon-strong methods and provide proof-of-concept numerical examples.

Improvement of the cross-entropy method in high dimension for failure probability estimation through a one-dimensional projection without gradient estimation

Rare event probability estimation is an important topic in reliability analysis. Stochastic methods, such as importance sampling, have been developed to estimate such probabilities but they often fail in high dimension. In this paper, we propose a new cross-entropy-based importance sampling algorithm to improve rare event probability estimation in high dimension. We focus on the cross-entropy method with Gaussian auxiliary distributions and we suggest to update the Gaussian covariance matrix only in a one-dimensional subspace. For that purpose, the main idea is to consider the projection in the one-dimensional subspace spanned by the sample mean vector, which gives an influential direction for the variance estimation. This approach does not require any additional simulation budget compared to the basic cross-entropy algorithm and we show on different numerical test cases that it greatly improves its performance in high dimension.

Systems of small-noise stochastic reaction–diffusion equations satisfy a large deviations principle that is uniform over all initial data

Large deviations principles characterize the exponential decay rates of the probabilities of rare events. Cerrai and Röckner (2004) proved that systems of stochastic reaction–diffusion equations satisfy a large deviations principle that is uniform over bounded sets of initial data.This paper proves uniform large deviations results for a system of stochastic reaction–diffusion equations in a more general setting than Cerrai and Röckner. Furthermore, this paper identifies two common situations where the large deviations principle is uniform over unbounded sets of initial data, enabling the characterization of Freidlin–Wentzell exit time and exit shape asymptotics from unbounded sets.

2020 Hindsight: Should evolutionary virologists have expected the unexpected during a pandemic?

2020 Hindsight: Should evolutionary virologists have expected the unexpected during a pandemic?

Variance based sensitivity analysis for Monte Carlo and importance sampling reliability assessment with Gaussian processes

Running a reliability analysis on engineering problems involving complex numerical models can be computationally very expensive, requiring advanced simulation methods to reduce the overall numerical cost. Gaussian process based active learning methods for reliability analysis have emerged as a promising way for reducing this computational cost. In this paper, we propose a methodology to quantify the sensitivity of the failure probability estimator to uncertainties generated by the Gaussian process and the sampling strategy. This quantification also enables to control the whole error associated to the failure probability estimate and thus provides an accuracy criterion on the estimation. Thus, an active learning approach integrating this analysis to reduce the main source of error and stopping when the global variability is sufficiently low is introduced. The approach is proposed for both a Monte Carlo based method as well as an importance sampling based method, seeking to improve the estimation of rare event probabilities. Performance of the proposed strategy is then assessed on several examples.

Sequential stratified splitting for efficient Monte Carlo integration

The efficient evaluation of high-dimensional integrals is important from both theoretical and practical points of view. In particular, multidimensional integration plays a central role in Bayesian inference, statistical physics, data science, and machine learning. However, due to the curse of dimensionality, deterministic numerical methods are inefficient in the high-dimensional setting. Consequentially, for many practical problems one must resort to approximate estimation techniques such as Monte Carlo methods. In this article, we introduce a novel sequential Monte Carlo algorithm called stratified splitting. The method provides unbiased estimates and can handle various integrand types including indicator functions, which are important for rare-event probability estimation problems. We provide rigorous analysis of the efficiency of the proposed method and present a numerical demonstration of the algorithmic performance when applied to practical application domains. Our numerical experiments suggest that the stratified splitting method is capable of delivering accurate results for a variety of integration problems while requiring reasonable computational effort.

Gel’fand–Yaglom type equations for calculating fluctuations around instantons in stochastic systems

In recent years, instanton calculus has successfully been employed to estimate tail probabilities of rare events in various stochastic dynamical systems. Without further corrections, however, these estimates can only capture the exponential scaling. In this paper, we derive a general, closed form expression for the leading prefactor contribution of the fluctuations around the instanton trajectory for the computation of probability density functions of general observables. The key technique is applying the Gel’fand–Yaglom recursive evaluation method to the suitably discretized Gaussian path integral of the fluctuations, in order to obtain matrix evolution equations that yield the fluctuation determinant. We demonstrate agreement between these predictions and direct sampling for examples motivated from turbulence theory.