Low-dimensional Problems Research Articles

Graph-accelerated non-intrusive polynomial chaos expansion using partially tensor-structured quadrature rules for uncertainty quantification

Recently, the graph-accelerated non-intrusive polynomial chaos (NIPC) method has been proposed for solving uncertainty quantification (UQ) problems. This method leverages the full-grid integration-based NIPC method to address UQ problems while employing the computational graph transformation approach, AMTC, to accelerate the tensor-grid evaluations. This method exhibits remarkable efficacy on a broad range of low-dimensional (three dimensions or less) UQ problems featuring multidisciplinary models. However, it often does not scale well with problem dimensions due to the exponential increase in the number of quadrature points when using the full-grid quadrature rule. To expand the applicability of this method to a broader range of UQ problems, this paper introduces a new framework for generating a tailored, partially tensor-structured quadrature rule to use with the graph-accelerated NIPC method. This quadrature rule, generated through the designed quadrature approach, possesses a tensor structure that is tailored for the computational model. The selection of the tensor structure is guided by an analysis of the computational graph, ensuring that the quadrature rule effectively capitalizes on the sparsity within the computational graph when paired with the AMTC method. This method has been tested on one 4D and one 6D UQ problem, both originating from aircraft design scenarios and featuring multidisciplinary models. Numerical results show that, when using with graph-accelerated NIPC method, our approach generates a partially tensor-structured quadrature rule that outperforms the full-grid Gauss quadrature and the designed quadrature methods (more than 40% reduction in computational costs) in both of the test problems.

Stiefel manifold interpolation for non-intrusive model reduction of parameterized fluid flow problems

Many engineering problems are parameterized. In order to minimize the computational cost necessary to evaluate a new operating point, the interpolation of singular matrices representing the data seems natural. Unfortunately, interpolating such data by conventional methods usually leads to unphysical solutions, as the data live on manifolds and not vector spaces. An alternative is to perform the interpolation in the tangent space to the Grassmann manifold to obtain interpolated spatial modes. Temporal modes are afterwards determined via the Galerkin projection of the high-fidelity model onto the interpolated spatial basis. This method, which is known for some fifteen years, is intrusive. Recently, Oulghelou and Allery (JCP, 2021) have proposed a non-intrusive approach (equation-free), but requiring the resolution of two low-dimensional optimization problems after interpolation. In this paper, a non-intrusive alternative based on Interpolation on the Tangent Space of the Stiefel Manifold (ITSSM) is presented. This approach has the advantage of not requiring a calibration phase after interpolation. To assess the method, we compare our results with those obtained using global POD on the one hand, and two methods based on Grassmann interpolation on the other. These comparisons are performed for two classical configurations encountered in fluid dynamics. The first corresponds to the one-dimensional non-linear Burgers' equation. The second example is the two-dimensional cylinder wake flow. We show that the proposed strategy can accurately reconstruct the physical quantities associated with a new operating point. Moreover, the estimation is fast enough to allow real-time computation.

Expected coordinate improvement for high-dimensional Bayesian optimization

Bayesian optimization (BO) algorithm is very popular for solving low-dimensional expensive optimization problems. Extending Bayesian optimization to high dimension is a meaningful but challenging task. One of the major challenges is that it is difficult to find good infill solutions as the acquisition functions are also high-dimensional. In this work, we propose the expected coordinate improvement (ECI) criterion for high-dimensional Bayesian optimization. The proposed ECI criterion measures the potential improvement we can get by moving the current best solution along one coordinate. The proposed approach selects the coordinate with the highest ECI value to refine in each iteration and covers all the coordinates gradually by iterating over the coordinates. The greatest advantage of the proposed ECI-BO (expected coordinate improvement based Bayesian optimization) algorithm over the standard BO algorithm is that the infill selection problem of the proposed algorithm is always a one-dimensional problem thus can be easily solved. Numerical experiments show that the proposed algorithm can achieve significantly better results than the standard BO algorithm and competitive results when compared with five high-dimensional BOs and six surrogate-assisted evolutionary algorithms. This work provides a simple but efficient approach for high-dimensional Bayesian optimization. A Matlab implementation of our ECI-BO is available at https://github.com/zhandawei/Expected_Coordinate_Improvement.

A dimensionality reduction assisted evolutionary algorithm for high-dimensional expensive multi/many-objective optimization

Surrogate-assisted multi/many-objective evolutionary algorithms (SA-MOEAs) have shown significant progress in tackling expensive optimization problems. However, existing research primarily focuses on low-dimensional optimization problems. The main reason lies in the fact that some surrogate techniques used in SA-MOEAs, such as the Kriging model, are not applicable for exploring high-dimensional decision space. This paper introduces a surrogate-assisted multi-objective evolutionary algorithm with dimensionality reduction to address high-dimensional expensive optimization problems. The proposed algorithm includes two key insights. Firstly, we propose a dimensionality reduction framework containing three different feature extraction algorithms and a feature drift strategy to map the high-dimensional decision space into a low-dimensional decision space; this strategy helps to improve the robustness of surrogates. Secondly, we propose a sub-region search strategy to define a series of promising sub-regions in the high-dimensional decision space; this strategy helps to improve the exploration ability of the proposed SA-MOEA. Experimental results demonstrate the effectiveness of our proposed algorithm in comparison to several state-of-the-art algorithms.

Extreme value analysis of multivariate Gaussian processes with wave-passage effects

Extreme value analysis is a central aspect of random vibration applications. Most studies focus on a univariate process. System reliability necessitates the extreme value across multiple correlated processes, but analytical methods are scarce and confined to low-dimensional problems. Recently, the authors proposed an analytical method for the extreme analysis of multivariate Gaussian processes. The exact upcrossing rate is derived for the maximum process representing the instantaneous maxima over all processes, and the extreme value distribution is obtained from the Poisson approximation. Nevertheless, for applications involving the wave-passage effect that is commonplace in random vibration, the upcrossings manifest in clumps, rendering the Poisson approximation conservative. The clumping from wave-passage is a complex novel phenomenon, differing from the clumping in narrowband processes. This paper extends the prior work by developing an analytical method for predicting the clump size, thereby providing an accurate prediction of the multivariate extreme value while accounting for the wave-passage effect. The method is powerful as it is fast and amenable to high-dimensional problems. Two examples include the propagation of ocean waves and a multi-span bridge subjected to propagating ground motions. The proposed method is shown to accurately predict the clumping factor and the probability of failure, compared to numerical simulations. In contrast, the Poisson approximation using the exact upcrossing rate noticeably overestimates the failure probability.

Replication in random translation designs

Replication is a commonly recommended feature of experimental designs. However, its impact in model-robust design is relatively under-explored; indeed, replication is impossible within the current formulation of random translation designs, which were introduced recently for model-robust prediction. Here we extend the framework of random translation designs to allow replication, and quantify the resulting performance impact. The extension permits a simplification of our earlier heuristic for constructing random translation strategies from a traditional V-optimal design. Namely, in the previous formulation any replicates of the V-optimal design first had to be split up before a random translation can be applied to the design points. With the new framework we can instead preserve the replicates instead if we so wish. Surprisingly, we find that in low-dimensional problems it is often substantially more efficient to continue to split replicates, while in high-dimensional problems it can be substantially better to retain replicates.

Random Contrastive Interaction for Particle Swarm Optimization in High-Dimensional Environment

In high dimensional environment, the interaction among particles significantly affects their movements in searching the vast solution space and thus plays a vital role in assisting particle swarm optimization (PSO) to attain good performance. To this end, this paper designs a random contrastive interaction (RCI) strategy for PSO, resulting in RCI-PSO, to tackle large-scale optimization problems (LSOPs) effectively and efficiently. Unlike existing interaction mechanisms for low-dimensional problems, RCI randomly chooses several different peers from the current swarm to construct a random interaction topology for each particle. Then, it lets the particle interact with the selected peers based on their current evolutionary information instead of their historical evolutionary information. Within the topology, RCI only propagates the evolutionary information of two contrastive dominators with the largest difference in fitness to direct the evolution of the particle. Therefore, particles with no more than two dominators in their topologies are not updated. Furthermore, a dynamic topology size adjustment scheme is devised to gradually enlarge the interaction topology. In this way, the swarm gradually switches from exploring the immense search space dispersedly to exploiting the found optimal regions intensively as the evolution continues. With these two strategies, RCI-PSO expectedly compromises search diversity and search convergence well at the swarm level and the particle level. At last, extensive experiments executed on two public LSOP suites verify that RCI-PSO performs competitively with or even much better than totally 40 state-of-theart large-scale approaches and preserves a good capability and scalability in tackling complex LSOPs.

Mechanism analysis and application of multi-dimensional single potential well stochastic resonance system

Currently, the focus of stochastic resonance (SR) research is primarily on bistable systems, and classical bistable SR systems have the problems of low dimension, inconvenient parameter adjustment, and high threshold for SR effects. In this paper, based on the classic bistable system, a class of multi-dimensional single potential well SR systems without transition threshold is proposed and applied to signal processing. Firstly, the mechanism of double potential well SR and single potential well SR is studied. On this basis, a kind of multi-dimensional single potential well SR system is defined and its theoretical conditions are analyzed. Then a specific four-dimensional single potential well SR system is constructed, and the dynamics of the system is analyzed. The gain range and the approximate relationship between the system input and the four-dimensional output are derived. Finally, the four-dimensional single potential well SR system is applied to the processing of various signals. The experimental results show that the constructed system has good noise reduction and feature amplification effects on noisy signals through the advantages of multi-dimensional output, and can be used to highlight the fault feature frequency in bearing fault signals.

A naïve Bayes regularized logistic regression estimator for low-dimensional classification

To reduce the estimator's variance and prevent overfitting, regularization techniques have attracted great interest from the statistics and machine learning communities. Most existing regularized methods rely on the sparsity assumption that a model with fewer parameters predicts better than one with many parameters. This assumption works particularly well in high-dimensional problems. However, the sparsity assumption may not be necessary when the number of predictors is relatively small compared to the number of training instances. This paper argues that shrinking the coefficients towards a low-variance data-driven estimate could be a better regularization strategy for such situations. For low-dimensional classification problems, we propose a naïve Bayes regularized logistic regression (NBRLR) that shrinks the logistic regression coefficients toward the naïve Bayes estimate to provide a reduction in variance. Our approach is primarily motivated by the fact that naïve Bayes is functionally equivalent to logistic regression if naïve Bayes' conditional independence assumption holds. Under standard conditions, we prove the consistency of the NBRLR estimator. Extensive simulation and empirical experimental results show that NBRLR is a competitive alternative to various state-of-the-art classifiers, especially on low-dimensional datasets.

VIP - Variational Inversion Package with example implementations of Bayesian tomographic imaging

Bayesian inference has become an important methodology to solve inverse problems and to quantify uncertainties in their solutions. Variational inference is a method that provides probabilistic, Bayesian solutions efficiently by using optimisation. In this study we present a Python Variational Inversion Package (VIP), to solve inverse problems using variational inference methods. The package includes automatic differential variational inference (ADVI), Stein variational gradient descent (SVGD) and stochastic SVGD (sSVGD), and provides implementations of 2D travel time tomography and 2D full waveform inversion including test examples and solutions. Users can solve their own problems by supplying an appropriate forward function and a gradient calculation code. In addition, the package provides a scalable implementation which can be deployed easily on a desktop machine or using modern high performance computational facilities. The examples demonstrate that VIP is an efficient, scalable, extensible and user-friendly package, and can be used to solve a wide range of low or high dimensional inverse problems in practice.

Reprint of: The likelihood ratio test for structural changes in factor models

A factor model with a break in its factor loadings is observationally equivalent to a model without changes in the loadings but with a change in the variance of its factors. This approach effectively transforms a high-dimensional structural change problem into a low-dimensional problem. This paper considers the likelihood ratio (LR) test for a variance change in the estimated factors. The LR test implicitly explores a special feature of the estimated factors: the pre-break and post-break variances can be a singular matrix under the alternative hypothesis, making the LR test diverging faster and thus more powerful than Wald-type tests. The better power property of the LR test is also confirmed by simulations. We also consider mean changes and multiple breaks. We apply this procedure to the factor modeling of the US employment and study the structural change problem using monthly industry-level data.

Local physical gradient-based coupling information interpolate method and application on double-wall turbine blade multidisciplinary analysis

A novel generalization interpolate modification method based on local gradient is presented. The main idea was to remove the sample points located in the gradient direction away from the unknown points compared to the sample points located in the perpendicular direction. This method first uses the Taylor expansion to construct a set of equations and solves the local physical gradient. Subsequently, a scale vector is derived to modify the interpolate space. Three classical interpolation methods were modified: Inverse Distance Weighted (IDW), Radial Basis Function (RBF), and Kriging method. The results of the test functions prove that the proposed method can effectively decrease the interpolate error regardless of low or high dimensional interpolate problem and irrespective of the sample point size. Surface and volume interpolate validations demonstrate that the local gradient generates a large interpolate error, and the modified method can essentially diminish this error with a maximum reduction of 9.94 %. Subsequently, this method is applied to implement a multidisciplinary analysis of a practical double-wall cooling blade with a large physical gradient. The results showed that the pressure and temperature interpolate errors could be decreased by 5.18 % and 14.5 %, respectively, with only an additional 115 s, at the fillet and film hole area where the local gradient was high. When the strength analysis was implemented, the maximum stress error reduction was 25.04 MPa at the stress concentration region.

Multi-surrogate-assisted stochastic fractal search based on scale-free network for high-dimensional expensive optimization

Surrogate-assisted meta-heuristic algorithms (SAMAs) have been increasingly popular in recent years for solving challenging optimization problems. However, the majority of recent studies concentrate on low-dimensional problems. In this paper, a scale-free network based multi-surrogate-assisted stochastic fractal search (SF-MSASFS) algorithm is proposed. Specifically, based on the stochastic fractal search (SFS) algorithm, multiple surrogate models, namely RBF and Kriging models, are used to enhance the robustness of the algorithm. The scale-free network is used to build the topology structure of the SFS algorithm, and the offspring particles are generated by means of the connection relationship between the parent particles. In addition, to further enhance adaptability, an adaptive mechanism is implemented, tailoring three distinct update mechanisms based on their corresponding reward values. Finally, the performance of the proposed algorithm is demonstrated by comparing the proposed algorithm with a number of state-of-the-art SAMAs on several well-known benchmark functions, in particular in solving high-dimensional expensive problems (HEOPs). The results underscore the SF-MSASFS algorithm’s commendable optimization performance. (The MATLAB code can be found at the authors github: https://github.com/xiaodi-Cheng/SF-MSASFS)

CLUSTERING-BASED OBJECTS STATE DIAGNOSTICS IN CONDITIONS OF FUZZY SOURCE DATA

У The problem of distribution of a set of objects, the state of which is determined by a set of controlled parameters, into a set of subsets of objects maximally homogeneous in their properties is considered. Relevance of the problem and important advantage of the clustering procedure: when its implementing it is possible to reduce the initial difficult problem of high dimensionality objects analysing to the solution of a number of simpler problems of lower dimensionality. This circumstance acquires additional attractiveness and importance if the initial data of the problem contain uncertainty, for example, are vaguely defined. Research object is the procedure of partitioning a set of objects into clusters under conditions of uncertainty. In this regard, the purpose of the study is to develop a method for solving the problem of clustering in conditions where the initial data on objects controlled parametersthe values contain uncertainty. The method of solving the problem is based on clustering procedure mathematical model development, containing analytical expressions for the criterion of its effectiveness, written in the form of a twice fractionally quadratic function. The impossibility of mathematical programming problem direct solution initiated the development of a heuristic algorithm for its solution. As a result, an iterative method was obtained and applied to solve the clustering problem under conditions of fuzzy initial data. The developed computational procedure is based on a reasonable system of rules for performing operations on fuzzy numbers. The situations when the belonging functions of problem fuzzy parameters are defined on infinite or compact media are considered. The developed system of rules allows to correctly perform operations in the metric of fuzzy defined states between clustering objects. The proposed method is easily generalised to the case when the uncertainty of the initial data is hierarchical.

The likelihood ratio test for structural changes in factor models

A factor model with a break in its factor loadings is observationally equivalent to a model without changes in the loadings but with a change in the variance of its factors. This approach effectively transforms a high-dimensional structural change problem into a low-dimensional problem. This paper considers the likelihood ratio (LR) test for a variance change in the estimated factors. The LR test implicitly explores a special feature of the estimated factors: the pre-break and post-break variances can be a singular matrix under the alternative hypothesis, making the LR test diverging faster and thus more powerful than Wald-type tests. The better power property of the LR test is also confirmed by simulations. We also consider mean changes and multiple breaks. We apply this procedure to the factor modeling of the US employment and study the structural change problem using monthly industry-level data.

High-Dimensional Multi-Objective Bayesian Optimization With Block Coordinate Updates: Case Studies in Intelligent Transportation System

Many transportation system problems can be formulated as high-dimensional expensive multi-objective problems. They are challenging for Gaussian process-based Bayesian optimization methods to find the Pareto fronts due to the curse of dimensionality and the boundary issue in the acquisition function optimization. This paper presents a multi-objective Bayesian optimization method with block coordinate updates, Block-MOBO, to solve high-dimensional expensive multi-objective problems. Block-MOBO first partitions the decision variable space into different blocks, each of which includes a low-dimensional multi-objective problem. At each iteration, one block is considered and the decision variables not in this block are approximated by context-vector generation embedded with the Pareto prior knowledge thus promoting convergence. To tackle the boundary issue, we present <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\epsilon$</tex-math> </inline-formula> -greedy acquisition function in a Bayesian and multi-objective fashion, which recommends candidates either from the exploitation-exploration trade-off perspective or with probability <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\epsilon$</tex-math> </inline-formula> from the Pareto dominance relationship perspective. We verify the effectiveness of Block-MOBO by comparing it with other multi-objective Bayesian methods on two real-world optimization problems in transportation system and three multi-objective synthetic test suites. The experimental results show that Block-MOBO can find more evenly distributed and non-dominated solutions in the whole search space with lower complexity compared with other state-of-the-art approaches. Our analyses illustrate that block coordinate updates and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\epsilon$</tex-math> </inline-formula> -greedy acquisition function contribute to computational complexity reduction and convergence-diversity trade-offs, respectively.

Using Gaussian Processes for Metamodeling in Robust Optimization Problems

&#x0D; This article proposes an approach based on Gaussian Processes for building metamodels for robust optimization problems that seek to reduce the computational effort required to quantify uncertainties. The approach is applied to two cases: a low-dimensional benchmark problem and a high-dimensional structural design, which consists of minimizing the mass of a structure formed by bars of different materials and diameters, subjected to point loads in different locations. The cases are modeled as robust optimization problems, where the objective function is estimated by a Gaussian Process and the optimization procedure uses a population meta-heuristic. The results indicate that the proposed approach is effective in reducing the number of objective function evaluations required to obtain a robust solution, with no significant statistical differences in the quality of solutions achieved.&#x0D;

Accelerating model evaluations in uncertainty propagation on tensor grids using computational graph transformations

Methods such as non-intrusive polynomial chaos (NIPC), and stochastic collocation are frequently used for uncertainty propagation problems. Particularly for low-dimensional problems, these methods often use a tensor-product grid for sampling the space of uncertain inputs. A limitation of this approach is that it encounters a significant challenge: the number of sample points grows exponentially with the increase of uncertain inputs. Current strategies to mitigate computational costs abandon the tensor structure of sampling points, with the aim of reducing their overall count. Contrastingly, our investigation reveals that preserving the tensor structure of sample points can offer distinct advantages in specific scenarios. Notably, by manipulating the computational graph of the targeted model, it is feasible to avoid redundant evaluations at the operation level to significantly reduce the model evaluation cost on tensor-grid inputs. This paper presents a pioneering method: Accelerated Model Evaluations on Tensor grids using Computational graph transformations (AMTC). The core premise of AMTC lies in the strategic modification of the computational graph of the target model to algorithmically remove the repeated evaluations on the operation level. We implemented the AMTC method within the compiler of a new modeling language called the Computational System Design Language (CSDL). We demonstrate the effectiveness of AMTC by using it with the full-grid NIPC method to solve four low-dimensional UQ problems involving an analytical piston model, a multidisciplinary unmanned aerial vehicle design model, a multi-point air taxi mission analysis model, and a single-disciplinary rotor model, respectively. For three of the four test problems, AMTC reduces the model evaluation cost by between 50% and 90%, making the full-grid NIPC the most efficacious method to use among the UQ methods implemented.

A Simulated Annealing Algorithm for Quay Crane Scheduling and Assignment Problem Under Uncertain Conditions

In this paper, a mixed integer programming (MIP) mathematical model was formulated for the quay crane scheduling and assignment problem (QCSAP) in container terminals with the aim of minimizing the total cost of activities. With the assumption that due to the uncertain nature of the ship’s activities and operations, we have considered most of the parameters of the problem to be indeterminate in order to bring the optimal solution closer to the real world. In this paper, the uncertain nature of the parameters was investigated by using uncertainty theory and in the form of indeterminate quantities. These uncertainties might be subject to the conditions, such as the failure of cranes, container terminal transportation means, congestion in the dockyard, adverse weather conditions, etc. Further, considering the large dimensions of such real problems and the complexity of solving them in terms of processing time, the simulated annealing (SA) meta-heuristic optimization algorithm was utilized to solve the above model. The proposed algorithm has been coded by MATLAB software and its efficiency was put on test by comparing its results for a low-dimensional problem with the accurate solution from GAMS software is compared in terms of computational times for small sizes. The proposed algorithm exhibited promising potentials for quick approximation of usable solutions to high-dimensional QCSAPs. The calculations show that, nondeterministic model with the help of the above proposed algorithm can be one of the basic factors in increasing the productivity of container terminals and the main purpose and advantage of this research is compared to the basic model presented by Tavakkoli-Moghaddam et al. in 2009 which has been analyzed and investigated with definitive data and optimized with the help of genetic algorithm. In addition, some restrictions were added to the basic model in order to make the above crane assignment scheduling problem more practical.

A time independent least squares algorithm for parameter identification of Turing patterns in reaction-diffusion systems.

Turing patterns arising from reaction-diffusion systems such as epidemic, ecology or chemical reaction models are an important dynamic property. Parameter identification of Turing patterns in spatial continuous and networked reaction-diffusion systems is an interesting and challenging inverse problem. The existing algorithms require huge account operations and resources. These drawbacks are amplified when apply them to reaction-diffusion systems on large-scale complex networks. To overcome these shortcomings, we present a new least squares algorithm which is rooted in the fact that Turing patterns are the stationary solutions of reaction-diffusion systems. The new algorithm is time independent, it translates the parameter identification problem into a low dimensional optimization problem even a low order linear algebra equations. The numerical simulations demonstrate that our algorithm has good effectiveness, robustness as well as performance.