Stochastic Dimension Research Articles

A novel LSTM-integrated non-intrusive ROM for reliability analysis of hysteretic systems with large stochastic dimension

Failure probability estimation of hysteretic systems subjected to random process excitation appears in many practical engineering applications, such as structures subjected to earthquake and vibration control. This task is computationally challenging due to the need for repeated solutions of a high-resolution model. This cost grows significantly with the stochastic dimension — characterized by the number of random variables, in this case, arising from the discretization of the random excitation. To address this issue, a novel non-intrusive reduced order model (ROM) is proposed in this paper. Accordingly, the complexity due to spatial discretization is first reduced using proper orthogonal decomposition. However, interpolation among response time histories in the resulting reduced solution space is computationally infeasible due to large stochastic dimensionality. An existing ROM developed by the authors is now improved by adopting a long short-term memory (LSTM) network for this interpolation. Furthermore, this adoption required two additional modifications in the mathematical structure of the ROM related to data compression and nonlinear coupling. The proposed methodology can be seamlessly integrated with black-box nonlinear solvers. Through detailed numerical studies on a beam on Winkler foundation and a multi-storied building frame subjected to stationary and non-stationary excitations, the proposed ROM is found to be accurate, efficient, and robust with respect to the frequency band of interest. The ROM has shown reduction of two orders of magnitude in the computational time. Furthermore, the proposed ROM is inherently parallelizable and holds promise to be scaled to other types of nonlinear systems with large stochastic dimensionality.

Tackling the curse of dimensionality with physics-informed neural networks

The curse-of-dimensionality taxes computational resources heavily with exponentially increasing computational cost as the dimension increases. This poses great challenges in solving high-dimensional partial differential equations (PDEs), as Richard E. Bellman first pointed out over 60 years ago. While there has been some recent success in solving numerical PDEs in high dimensions, such computations are prohibitively expensive, and true scaling of general nonlinear PDEs to high dimensions has never been achieved. We develop a new method of scaling up physics-informed neural networks (PINNs) to solve arbitrary high-dimensional PDEs. The new method, called Stochastic Dimension Gradient Descent (SDGD), decomposes a gradient of PDEs’ and PINNs’ residual into pieces corresponding to different dimensions and randomly samples a subset of these dimensional pieces in each iteration of training PINNs. We prove theoretically the convergence and other desired properties of the proposed method. We demonstrate in various diverse tests that the proposed method can solve many notoriously hard high-dimensional PDEs, including the Hamilton–Jacobi-Bellman (HJB) and the Schrödinger equations in tens of thousands of dimensions very fast on a single GPU using the PINNs mesh-free approach. Notably, we solve nonlinear PDEs with nontrivial, anisotropic, and inseparable solutions in less than one hour for 1000 dimensions and in 12 h for 100,000 dimensions on a single GPU using SDGD with PINNs. Since SDGD is a general training methodology of PINNs, it can be applied to any current and future variants of PINNs to scale them up for arbitrary high-dimensional PDEs.

Optimized data driven fault detection and diagnosis in chemical processes

Fault detection and diagnosis (FDD) is crucial for ensuring process safety and product quality in the chemical industry. Despite the large amounts of process data recorded and stored in chemical plants, most of them are not well-labeled, and their conditions are not adequately specified. In this study, an optimized data-driven FDD model was developed for a chemical process based on automatic clustering algorithms. Due to data preprocessing importance, feature selection was performed by a non-dominated sorting genetic algorithm (NSGAII) based on k-means clustering. The optimal subset of features is selected by comparing clustering results for each subset. The performance of the proposed feature selection method was compared with the Fisher discriminant ratio (FDR), and XGBoost methods. The t-distributed stochastic neighbor embedding (t-SNE), Isomap, and KPCA dimension reduction methods were also employed for feature extraction. Finally, automatic clustering was performed based on metaheuristic algorithms for fault detection and diagnosis. Results were compared with non-automatic clustering methods. The performance of the proposed method was evaluated by examining the Tennessee Eastman and four water tank processes as case studies. The results showed that the proposed method is reliable and capable of online and offline chemical process fault detection and diagnosis. As a result, the findings of this study can be used to stabilize the operation of chemical processes.

Parabolic PDE-constrained optimal control under uncertainty with entropic risk measure using quasi-Monte Carlo integration

We study the application of a tailored quasi-Monte Carlo (QMC) method to a class of optimal control problems subject to parabolic partial differential equation (PDE) constraints under uncertainty: the state in our setting is the solution of a parabolic PDE with a random thermal diffusion coefficient, steered by a control function. To account for the presence of uncertainty in the optimal control problem, the objective function is composed with a risk measure. We focus on two risk measures, both involving high-dimensional integrals over the stochastic variables: the expected value and the (nonlinear) entropic risk measure. The high-dimensional integrals are computed numerically using specially designed QMC methods and, under moderate assumptions on the input random field, the error rate is shown to be essentially linear, independently of the stochastic dimension of the problem—and thereby superior to ordinary Monte Carlo methods. Numerical results demonstrate the effectiveness of our method.

Efficient stochastic modal decomposition methods for structural stochastic static and dynamic analyses

AbstractThis article presents unified and efficient stochastic modal decomposition methods to solve stochastic structural static and dynamic problems. We extend the idea of deterministic modal decomposition method for structural dynamic analysis to stochastic cases. Standard/generalized stochastic eigenvalue equations are adopted to calculate the stochastic subspaces for stochastic static/dynamic problems and they are solved by an efficient reduced‐order method. The stochastic solutions of both static and dynamic equations are approximated by stochastic bases of the stochastic subspaces. Original stochastic static/dynamic equations are then transformed into a set of single‐degree‐of‐freedom (SDoF) stochastic static/dynamic equations, which are efficiently solved by the proposed non‐intrusive methods. Specifically, a non‐intrusive stochastic Newmark method is developed for the solution of SDoF stochastic dynamic equations, and the element‐wise division of sample vectors is used to solve the SDoF stochastic static equations. All of these methods have low computational effort and are weakly sensitive to the stochastic dimension, thus the proposed methods avoid the curse of dimensionality successfully. Two numerical examples, including two‐ and three‐dimensional spatial problems with low and high stochastic dimensions, are given to show the efficiency and accuracy of the proposed methods.

Hutchinson Trace Estimation for high-dimensional and high-order Physics-Informed Neural Networks

Physics-Informed Neural Networks (PINNs) have proven effective in solving partial differential equations (PDEs), especially when some data are available by seamlessly blending data and physics. However, extending PINNs to high-dimensional and even high-order PDEs encounters significant challenges due to the computational cost associated with automatic differentiation in the residual loss function calculation. Herein, we address the limitations of PINNs in handling high-dimensional and high-order PDEs by introducing the Hutchinson Trace Estimation (HTE) method. Starting with the second-order high-dimensional PDEs, which are ubiquitous in scientific computing, HTE is applied to transform the calculation of the entire Hessian matrix into a Hessian vector product (HVP). This approach not only alleviates the computational bottleneck via Taylor-mode automatic differentiation but also significantly reduces memory consumption from the Hessian matrix to an HVP’s scalar output. We further showcase HTE’s convergence to the original PINN loss and its unbiased behavior under specific conditions. Comparisons with the Stochastic Dimension Gradient Descent (SDGD) highlight the distinct advantages of HTE, particularly in scenarios with significant variability and variance among dimensions. We further extend the application of HTE to higher-order and higher-dimensional PDEs, specifically addressing the biharmonic equation. By employing tensor-vector products (TVP), HTE efficiently computes the colossal tensor associated with the fourth-order high-dimensional biharmonic equation, saving memory and enabling rapid computation. The effectiveness of HTE is illustrated through experimental setups, demonstrating comparable convergence rates with SDGD under memory and speed constraints. Additionally, HTE proves valuable in accelerating the Gradient-Enhanced PINN (gPINN) version as well as the Biharmonic equation. Overall, HTE opens up a new capability in scientific machine learning for tackling high-order and high-dimensional PDEs.

Uncertainty quantification for random domains using periodic random variables

We consider uncertainty quantification for the Poisson problem subject to domain uncertainty. For the stochastic parameterization of the random domain, we use the model recently introduced by Kaarnioja et al. (SIAM J. Numer. Anal., 2020) in which a countably infinite number of independent random variables enter the random field as periodic functions. We develop lattice quasi-Monte Carlo (QMC) cubature rules for computing the expected value of the solution to the Poisson problem subject to domain uncertainty. These QMC rules can be shown to exhibit higher order cubature convergence rates permitted by the periodic setting independently of the stochastic dimension of the problem. In addition, we present a complete error analysis for the problem by taking into account the approximation errors incurred by truncating the input random field to a finite number of terms and discretizing the spatial domain using finite elements. The paper concludes with numerical experiments demonstrating the theoretical error estimates.

A novel non-intrusive ROM for randomly excited linear dynamical systems with high stochastic dimension using ANN

Analyzing large stochastic dynamical systems is computationally very expensive. A statistical simulation framework requires invoking the solver multiple times — ranging from thousands to millions. A non-intrusive reduced order model (ROM) serves as a computationally efficient alternative in this framework. Uncertainties in dynamical systems originate from two sources: system parameters and excitation. However, all the existing ROMs have been developed for uncertainty only in the system parameters. To the best of the authors’ knowledge, no ROM exists for dynamical systems under random excitation. A discretization of the input process increases the stochastic dimensionality, typically by a few hundred or thousand. This leads to a high dimensional interpolation or regression in the ROM, which is practically infeasible. This issue is addressed in this work by proposing a novel non-intrusive ROM that bypasses the need for such discretization. Accordingly, a regression is carried out directly on the random excitation using a neural network-based surrogate model. A principal component-based data compression is used in tandem to reduce the stochastic dimensionality of excitations. Detailed numerical studies are conducted to study the accuracy and efficiency of the proposed ROM using two examples: a mistuned bladed disk problem and a soil–structure interaction. The numerical results show that the proposed ROM is accurate and gains a significant speed-up of more than sixty for both examples. Using the proposed ROM, the cost of uncertainty quantification can be reduced significantly within the framework of Monte Carlo simulation.

Sparse polynomial chaos expansion for high-dimensional nonlinear damage mechanics

Finite Element Simulations in solid mechanics are nowadays common practice in engineering. However, considering uncertainties based on this powerful method remains a challenging task, especially when nonlinearities and high stochastic dimensions have to be taken into account. Although Monte Carlo Simulation (MCS) is a robust method, the computational burden is high, especially when a nonlinear finite element analysis has to be performed behind each sample. To overcome this burden, several “model-order reduction” techniques have been discussed in the literature. Often, these studies are limited to quite smooth responses (linear or smooth nonlinear models and moderate stochastic dimensions).This paper presents systematic studies of the promising Sparse Polynomial Chaos Expansion (SPCE) method to investigate the capabilities and limitations of this approach using MCS as a benchmark. A nonlinear damage mechanics problem serves as a reference, which involves random fields of material properties. By this, a clear limitation of SPCE with respect to the stochastic dimensionality could be shown, where, as expected, the advantage over MCS disappears.As part of these investigations, options to optimise SPCE have been studied, such as different error measures and optimisation algorithms. Furthermore, we have found that combining SPCEs with sensitivity analysis to reduce the stochastic dimension improves accuracy in many cases at low computational cost.

A stochastic LATIN method for stochastic and parameterized elastoplastic analysis

The LATIN method has been developed and successfully applied to a variety of deterministic problems, but few work has been developed for nonlinear stochastic problems. This paper presents a stochastic LATIN method to solve stochastic and/or parameterized elastoplastic problems. To this end, the stochastic solution is decoupled into spatial, temporal and stochastic spaces, and approximated by the sum of a set of products of triplets of spatial functions, temporal functions and random variables. Each triplet is then calculated in a greedy way using a stochastic LATIN iteration. The high efficiency of the proposed method relies on two aspects: The nonlinearity is efficiently handled by inheriting advantages of the classical LATIN method, and the randomness and/or parameters are effectively treated by a sample-based approximation of stochastic spaces. Further, the proposed method is not sensitive to the stochastic and/or parametric dimensions of inputs due to the sample description of stochastic spaces. It can thus be applied to high-dimensional stochastic and parameterized problems. Five numerical examples demonstrate the promising performance of the proposed stochastic LATIN method.

Uncertainty Quantification for Thermodynamic Simulations with High-Dimensional Input Spaces Using Sparse Polynomial Chaos Expansion: Retrofit of a Large Thermal Power Plant

The assessment of the future thermodynamics performance of a retrofitted heat and power production unit is prone to many uncertainties due to the large number of parameters involved in the modeling of all its components. To carry out uncertainty quantification analysis, alternatives to the traditional Monte Carlo method must be used due to the large stochastic dimension of the problem. In this paper, sparse polynomial chaos expansion (SPCE) is applied to the retrofit of a large coal-fired power plant into a biomass-fired combined heat and power unit to quantify the main drivers and the overall uncertainty on the plant’s performance. The thermodynamic model encompasses over 180 components and 1500 parameters. A methodology combining the use of SPCE and expert judgment is proposed to narrow down the sources of uncertainty and deliver reliable probability distributions for the main key performance indicators (KPIs). The impact of the uncertainties on each input parameter vary with the considered KPI and its assessment through the computation of Sobol’ indices. For both coal and biomass operations, the most impactful input parameters are the composition of the fuel and its heating value. The uncertainty on the performance and steam quality parameters is not much affected by the retrofit. Key furnace parameters exhibit a skewed probability distribution with large uncertainties, which is a strong attention point in terms of boiler operation and maintenance.

Arbitrage theory in a market of stochastic dimension

AbstractThis paper studies an equity market of stochastic dimension, where the number of assets fluctuates over time. In such a market, we develop the fundamental theorem of asset pricing, which provides the equivalence of the following statements: (i) there exists a supermartingale numéraire portfolio; (ii) each dissected market, which is of a fixed dimension between dimensional jumps, has locally finite growth; (iii) there is no arbitrage of the first kind; (iv) there exists a local martingale deflator; (v) the market is viable. We also present the optional decomposition theorem, which characterizes a given nonnegative process as the wealth process of some investment‐consumption strategy. Furthermore, similar results still hold in an open market embedded in the entire market of stochastic dimension, where investors can only invest in a fixed number of large capitalization stocks. These results are developed in an equity market model where the price process is given by a piecewise continuous semimartingale of stochastic dimension. Without the continuity assumption on the price process, we present similar results but without explicit characterization of the numéraire portfolio.

Karhunen-Loève expansion based on an analytical solution over a bounding box domain

This paper explores the accuracy and the efficiency of analytical solution of Fredholm integral equation to represent a random field on complex geometry. Because no analytical solution is available for arbitrary domains, it is proposed to use the analytical solution on simple bounding domains that enclose complex two- or three-dimensional geometries. It is a simple, accurate and robust approach for discretising a random field. The effect of the size of the bounding box on the resulting random field variance is investigated carefully and compared with the numerical solution given by the finite element method. The error variance is particularly localised near to the support domain boundary. Therefore, it is suggested to expand the bounding domain. This paper proposes a calibration of the correlation length to the ratio of the domain to maintain the convergence rate and the variance accuracy of the KLE without enlarging the stochastic dimension.

Stochastic virtual element methods for uncertainty propagation of stochastic linear elasticity

This paper presents stochastic virtual element methods for propagating uncertainty in linear elastic stochastic problems. We first derive stochastic virtual element equations for 2D and 3D linear elastic problems that may involve uncertainties in material properties, external forces, boundary conditions, etc. A stochastic virtual element space that couples the deterministic virtual element space and the stochastic space is constructed for this purpose and used to approximate the unknown stochastic solution. Two numerical frameworks are then developed to solve the derived stochastic virtual element equations, including a Polynomial Chaos approximation based approach and a weakly intrusive approximation based approach. In the Polynomial Chaos based framework, the stochastic solution is approximated using the Polynomial Chaos basis and solved via an augmented deterministic virtual element equation that is generated by applying the stochastic Galerkin procedure to the original stochastic virtual element equation. In the weakly intrusive approximation based framework, the stochastic solution is approximated by a summation of a set of products of random variables and deterministic vectors, where the deterministic vectors are solved via converting the original stochastic problem to deterministic virtual element equations by the stochastic Galerkin approach, and the random variables are solved via converting the original stochastic problem to one-dimensional stochastic algebraic equations by the classical Galerkin procedure. This method avoids the curse of dimensionality in high-dimensional stochastic problems successfully since all random inputs are embedded into one-dimensional stochastic algebraic equations whose computational effort weakly depends on the stochastic dimension. Numerical results on 2D and 3D problems with low- and high-dimensional random inputs demonstrate the good performance of the proposed methods.

HBWO-JS: jellyfish search boosted hybrid beluga whale optimization algorithm for engineering applications

Abstract Beluga whale optimization (BWO) algorithm is a recently proposed population intelligence algorithm. Inspired by the swimming, foraging, and whale falling behaviors of beluga whale populations, it shows good competitive performance compared to other state-of-the-art algorithms. However, the original BWO faces the challenges of unbalanced exploration and exploitation, premature stagnation of iterations, and low convergence accuracy in high-dimensional complex applications. Aiming at these challenges, a hybrid BWO based on the jellyfish search optimizer (HBWO-JS), which combines the vertical crossover operator and Gaussian variation strategy with a fusion of jellyfish search (JS) optimizer, is developed for solving global optimization in this paper. First, the BWO algorithm is fused with the JS optimizer to improve the problem that BWO tends to fall into the best local solution and low convergence accuracy in the exploitation stage through multi-stage exploration and collaborative exploitation. Then, the introduced vertical cross operator solves the problem of unbalanced exploration and exploitation processes by normalizing the upper and lower bounds of two stochastic dimensions of the search agent, thus further improving the overall optimization capability. In addition, the introduced Gaussian variation strategy forces the agent to explore the minimum neighborhood, extending the entire iterative search process and thus alleviating the problem of premature stagnation of the algorithm. Finally, the superiority of the proposed HBWO-JS is verified in detail by comparing it with basic BWO and eight state-of-the-art algorithms on the CEC2019 and CEC2020 test suites, respectively. Also, the scalability of HBWO-JS is evaluated in three dimensions (10D, 30D, 50D), and the results show the stable performance of the proposed algorithm in terms of dimensional scalability. In addition, three practical engineering designs and two Truss topology optimization problems demonstrate the practicality of HBWO-JS. The optimization results show that HBWO-JS has a strong competitive ability and broad application prospects.

Monte Carlo estimates of extremes of stationary/nonstationary Gaussian processes

Abstract Finite-dimensional (FD) models X d ⁢ ( t ) X_{d}(t) , i.e., deterministic functions of time and finite sets of 𝑑 random variables, are constructed for stationary and nonstationary Gaussian processes X ⁢ ( t ) X(t) with continuous samples defined on a bounded time interval [ 0 , τ ] [0,\tau] . The basis functions of these FD models are finite sets of eigenfunctions of the correlation functions of X ⁢ ( t ) X(t) and of trigonometric functions. Numerical illustrations are presented for a stationary Gaussian process X ⁢ ( t ) X(t) with exponential correlation function and a nonstationary version of this process obtained by time distortion. It was found that the FD models are consistent with the theoretical results in the sense that their samples approach the target samples as the stochastic dimension is increased.

Semi-reduced order stochastic finite element methods for solving contact problems with uncertainties

This paper develops two-step methods for solving contact problems with uncertainties. In the first step, we propose stochastic Lagrangian multiplier/penalty methods to compute a set of reduced basis. In the stochastic Lagrangian multiplier method, the stochastic solution is represented as a sum of products of a set of random variables and deterministic vectors. In the stochastic penalty method, the problem is divided into the solutions of non-contact and possible contact nodes, which are represented as sums of the products of two different sets of random variables and deterministic vectors, respectively. The original problems are then transformed into deterministic finite element equations and one-dimensional (corresponding to stochastic Lagrangian multiplier method)/two-dimensional (corresponding to stochastic penalty method) stochastic algebraic equations. The deterministic finite element equations are solved by existing numerical techniques, and the one-/two-dimensional stochastic algebraic equations are solved by a sampling method. Since the computational cost for solving stochastic algebraic equations does not increase dramatically as the stochastic dimension increases, the proposed methods avoid the curse of dimensionality in high-dimensional problems. Based on the reduced basis, we propose semi-reduced order Lagrangian multiplier/penalty equations with two components in the second step. One component is a reduced order equation obtained by smooth solutions of the reduced basis and the other is the full order equation for the nonsmooth solutions. A significant amount of computational cost is saved since the sizes of the semi-reduced order equations are usually small. Numerical examples of up to 100 dimensions demonstrate the good performance of the proposed methods.

Sparse polynomial chaos expansion for nonlinear finite element simulations with random material properties

AbstractThis contribution deals with the uncertainty quantification for applied nonlinear structural engineering problems, including high stochastic dimensions. A finite element problem with different material models is investigated. The efficiency, accuracy and convergence of sparse PCE are studied numerically and compared with Monte‐Carlo Simulation (MCS) for non‐linear structural analysis including elasto‐plastic and damage models. In both models, the Young's modulus is considered as random fields discretised by Karhunen Loeve Expansion (KLE). In the provided studies, sparse PCE converges fast and is highly efficient for linear elastic and elasto‐plastic material models. However, sparse PCE loses its effectiveness and exhibits lower accuracy for the damage material model.

A Reduced-Order Wiener Path Integral Formalism for Determining the Stochastic Response of Nonlinear Systems With Fractional Derivative Elements

Abstract A technique based on the Wiener path integral (WPI) is developed for determining the stochastic response of diverse nonlinear systems with fractional derivative elements. Specifically, a reduced-order WPI formulation is proposed, which can be construed as an approximation-free dimension reduction approach that renders the associated computational cost independent of the total number of stochastic dimensions of the problem. In fact, the herein developed technique can determine, directly, any lower-dimensional joint response probability density function corresponding to a subset only of the response vector components. This is done by utilizing an appropriate combination of fixed and free boundary conditions in the related variational, functional minimization, problem. Notably, the reduced-order WPI formulation is particularly advantageous for problems where the interest lies in few only specific degrees-of-freedom whose stochastic response is critical for the design and optimization of the overall system. An indicative numerical example is considered pertaining to a stochastically excited tuned mass-damper-inerter nonlinear system with a fractional derivative element. Comparisons with relevant Monte Carlo simulation data demonstrate the accuracy and computational efficiency of the technique.

A Wiener path integral technique for determining the stochastic response of nonlinear oscillators with fractional derivative elements: A constrained variational formulation with free boundaries

A technique based on the concept of Wiener path integral (WPI) is developed for determining approximately the joint response probability density function (PDF) of nonlinear oscillators endowed with fractional derivative elements. Specifically, first, the dependence of the state of the system on its history due to the fractional derivative terms is accounted for, alternatively, by augmenting the response vector and by considering additional auxiliary state variables and equations. In this regard, the original single-degree-of-freedom (SDOF) nonlinear system with fractional derivative terms is cast, equivalently, into a multi-degree-of-freedom (MDOF) nonlinear system involving integer-order derivatives only. From a mathematics perspective, the equations of motion referring to the latter can be interpreted as constrained. Second, to circumvent the challenge of increased dimensionality of the problem due to the augmentation of the response vector, a WPI formulation with mixed fixed/free boundary conditions is developed for determining directly any lower-dimensional joint PDF corresponding to a subset only of the response vector components. This can be construed as an approximation-free dimension reduction approach that renders the associated computational cost independent of the total number of stochastic dimensions of the problem. Thus, the original SDOF oscillator joint PDF corresponding to the response displacement and velocity is determined efficiently, while circumventing the computationally challenging task of treating directly equations of motion involving fractional derivatives. Two illustrative numerical examples are considered for demonstrating the reliability of the developed technique. These pertain to a nonlinear Duffing and a nonlinear vibro-impact oscillators with fractional derivative elements subjected to combined stochastic and deterministic periodic loading. Note that alternative standard approximate techniques, such as statistical linearization, need to be significantly modified and extended to account for such cases of combined loading. Remarkably, it is shown herein that the WPI technique exhibits the additional advantage of treating such types of excitation in a straightforward manner without the need for any ad hoc modifications. Comparisons with pertinent Monte Carlo simulation data are included as well.