Low-dimensional Problems Research Articles

GOZDE: A novel metaheuristic algorithm for global optimization

This study proposes a new metaheuristic algorithm, called “Geometric Octal Zones Distance Estimation” (GOZDE) algorithm to solve global optimization problems. The presented GOZDE employs a search scheme with the information sharing between the zones considering the distance of the zones utilizing median values. The whole population represents the eight zones that are the combination of different search strategies to guide knowledge dissemination from one zone to others in the search space. To demonstrate the effectiveness of the proposed optimizer, it is compared with two classes of metaheuristics, which are (1) GA, PSO, DE, CS and HS as the classical metaheuristics and (2) BWO, SSA, MVO, HHO, ChOA, AOA and EBOwithCMAR as the up-to-date metaheuristics. The search capability of the proposed algorithm is tested on two different numerical benchmark sets including low and high dimensional problems. The developed algorithm is also adapted to ten real-world applications to handle constraint optimization problems. In addition, to further analyse the results of the proposed algorithm, three well-known statistical metrics, Friedman, Wilcoxon rank-sum and Whisker-Box statistical tests are conducted. The experimental results statistically show that GOZDE is significantly better than, or at least comparable to the twelve metaheuristic algorithms with outstanding performance in solving numerical functions and real-world optimization problems.

Homogenization of the time-dependent heat equation on planar one-dimensional periodic structures

ABSTRACT In this paper we consider the homogenization of a time-dependent heat conduction problem on a planar one-dimensional periodic structure. On the edges of a graph the one-dimensional heat equation is posed, while the Kirchhoff junction condition is applied at all (inner) vertices. Using the two-scale convergence adapted to homogenization of lower-dimensional problems we obtain the limit homogenized problem defined on a two-dimensional domain that is occupied by the mesh when the mesh period δ tends to 0. The homogenized model is given by the classical heat equation with the conductivity tensor depending on the unit cell graph only through the topology of the graph and lengthes of its edges. We show the well-posedness of the limit problem and give a purely algebraic formula for the computation of the homogenized conductivity tensor. The analysis is completed by numerical experiments showing a convergence to the limit problem where the convergence order in δ depends on the unit cell pattern.

A scalability study of the Ice-sheet and Sea-level System Model (ISSM, version 4.18)

Abstract. Accurately modelling the contribution of Greenland and Antarctica to sea level rise requires solving partial differential equations at a high spatial resolution. In this paper, we discuss the scaling of the Ice-sheet and Sea-level System Model (ISSM) applied to the Greenland Ice Sheet with horizontal grid resolutions varying between 10 and 0.25 km. The model setup used as benchmark problem comprises a variety of modules with different levels of complexity and computational demands. The core builds the so-called stress balance module, which uses the higher-order approximation (or Blatter–Pattyn) of the Stokes equations, including free surface and ice-front evolution as well as thermodynamics in form of an enthalpy balance, and a mesh of linear prismatic finite elements, to compute the ice flow. We develop a detailed user-oriented, yet low-overhead, performance instrumentation tailored to the requirements of Earth system models and run scaling tests up to 6144 Message Passing Interface (MPI) processes. The results show that the computation of the Greenland model scales overall well up to 3072 MPI processes but is eventually slowed down by matrix assembly, the output handling and lower-dimensional problems that employ lower numbers of unknowns per MPI process. We also discuss improvements of the scaling and identify further improvements needed for climate research. The instrumented version of ISSM thus not only identifies potential performance bottlenecks that were not present at lower core counts but also provides the capability to continually monitor the performance of ISSM code basis. This is of long-term significance as the overall performance of ISSM model depends on the subtle interplay between algorithms, their implementation, underlying libraries, compilers, runtime systems and hardware characteristics, all of which are in a constant state of flux. We believe that future large-scale high-performance computing (HPC) systems will continue to employ the MPI-based programming paradigm on the road to exascale. Our scaling study pertains to a particular modelling setup available within ISSM and does not address accelerator techniques such as the use of vector units or GPUs. However, with 6144 MPI processes, we identified issues that need to be addressed in order to improve the ability of the ISSM code base to take advantage of upcoming systems that will require scaling to even higher numbers of MPI processes.

High-Dimensional Reliability Analysis with Error-Guided Active-Learning Probabilistic Support Vector Machine: Application to Wind-Reliability Analysis of Transmission Towers

Adaptive reliability analysis methods based on surrogate models, especially kriging, have been successfully implemented in many problems. However, the application of kriging is limited to low-dimensional problems with noncategorical performance data. Support vector machine (SVM), by contrast, addresses these limitations, but its application in reliability analysis faces several challenges with regard to robustness, accuracy, and efficiency. This study proposed a new adaptive approach based on probabilistic support vector machine for reliability analysis (PSVM-RA). Different from existing methods that only select training points in the margin of the SVM, the proposed method adopts a new learning function that considers the wrong classification probability for each realization and maximizes the potential for new information offered by a candidate sample for the training set. Moreover, the upper bound of the error that is introduced by the SVM in estimating the failure probability is derived based on a Poisson binomial distribution model considering the likelihood of wrong classification for all the points in the margin of the SVM. This upper bound of error was used in the proposed framework as a stopping criterion to guarantee the desired accuracy. Three numerical examples and an engineering application regarding the wind-reliability analysis of transmission towers were investigated to demonstrate the performance of the proposed method. It was demonstrated that PSVM-RA can provide robust estimates of failure probability when other state-of-the-art methods fail. Moreover, it offers a balance between efficiency and accuracy.

On complexity and convergence of high-order coordinate descent algorithms for smooth nonconvex box-constrained minimization

Coordinate descent methods have considerable impact in global optimization because global (or, at least, almost global) minimization is affordable for low-dimensional problems. Coordinate descent methods with high-order regularized models for smooth nonconvex box-constrained minimization are introduced in this work. High-order stationarity asymptotic convergence and first-order stationarity worst-case evaluation complexity bounds are established. The computer work that is necessary for obtaining first-order $\varepsilon$-stationarity with respect to the variables of each coordinate-descent block is $O(\varepsilon^{-(p+1)/p})$ whereas the computer work for getting first-order $\varepsilon$-stationarity with respect to all the variables simultaneously is $O(\varepsilon^{-(p+1)})$. Numerical examples involving multidimensional scaling problems are presented. The numerical performance of the methods is enhanced by means of coordinate-descent strategies for choosing initial points.

Sedimentological data-driven bottom friction parameter estimation in modelling Bristol Channel tidal dynamics

Accurately representing the bottom friction effect is a significant challenge in numerical tidal models. Bottom friction effects are commonly defined via parameter estimation techniques. However, the bottom friction coefficient (BFC) can be related to the roughness of the sea bed. Therefore, sedimentological data can be beneficial in estimating BFCs. Taking the Bristol Channel and Severn Estuary as a case study, we perform a number of BFC parameter estimation experiments, utilising sedimentological data in a variety of ways. Model performance is explored through the results of each parameter estimation experiment, including applications to tidal range and tidal stream resource assessment. We find that theoretically derived sediment-based BFCs are in most cases detrimental to model performance. However, good performance is obtained by retaining the spatial information provided by the sedimentological data in the formulation of the parameter estimation experiment; the spatially varying BFC can be represented as a piecewise-constant field following the spatial distribution of the observed sediment types. By solving the resulting low-dimensional parameter estimation problem, we obtain good model performance as measured against tide gauge data. This approach appears well suited to modelling tidal range energy resource, which is of particular interest in the case study region. However, the applicability of this approach for tidal stream resource assessment is limited, since modelled tidal currents exhibit a strong localised response to the BFC; the use of piecewise-constant (and therefore discontinuous) BFCs is found to be detrimental to model performance for tidal currents.

Transit search: An optimization algorithm based on exoplanet exploration

In this article, a novel astrophysics-inspired meta-heuristic optimization algorithm, namely Transit Search (TS) is proposed based on a famous exoplanet exploration method. More than 3800 planets have been detected using transit technique by the database of the space telescopes. Transit is a method that has shown more potential than the second well-known successful method (radial velocity) with 915 discovered planets until 2022 March. It is difficult to detect the planets because of their small dimension in the cosmos scale. Due to the high efficiency of the transit method in astrophysics and its capabilities, it has been used to formulate an optimization technique for this research. In the transit algorithm, by studying the light received from the stars at certain intervals, the changes in luminosity are examined and if a decrease in the amount of the received light is observed, it indicates that a planet passes from the star front. In order to evaluate the capability of the proposed algorithm, 73 constrained and unconstrained problems are considered and the results have been compared with 13 well-known optimization algorithms. This set of examples includes a wide range of types of problems including mathematical functions (28 high-dimensional and 15 low-dimensional problems), CEC functions (10 problems), constrained mathematical benchmark problems (G01–G13), as well as 7 constrained engineering problems. The results indicated that the overall average error for the proposed algorithm is the lowest amount for the benchmark problems in comparison with the other efficient algorithms

Isolation Forests and Deep Autoencoders for Industrial Screw Tightening Anomaly Detection

Within the context of Industry 4.0, quality assessment procedures using data-driven techniques are becoming more critical due to the generation of massive amounts of production data. In this paper, we address the detection of abnormal screw tightening processes, which is a key industrial task. Since labeling is costly, requiring a manual effort, we focus on unsupervised detection approaches. In particular, we assume a computationally light low-dimensional problem formulation based on angle–torque pairs. Our work is focused on two unsupervised machine learning (ML) algorithms: isolation forest (IForest) and a deep learning autoencoder (AE). Several computational experiments were held by assuming distinct datasets and a realistic rolling window evaluation procedure. First, we compared the two ML algorithms with two other methods, a local outlier factor method and a supervised Random Forest, on older data related with two production days collected in November 2020. Since competitive results were obtained, during a second stage, we further compared the AE and IForest methods by adopting a more recent and larger dataset (from February to March 2021, totaling 26.9 million observations and related to three distinct assembled products). Both anomaly detection methods obtained an excellent quality class discrimination (higher than 90%) under a realistic rolling window with several training and testing updates. Turning to the computational effort, the AE is much lighter than the IForest for training (around 2.7 times faster) and inference (requiring 3.0 times less computation). This AE property is valuable within this industrial domain since it tends to generate big data. Finally, using the anomaly detection estimates, we developed an interactive visualization tool that provides explainable artificial intelligence (XAI) knowledge for the human operators, helping them to better identify the angle–torque regions associated with screw tightening failures.

Multiobjective Project Scheduling with Multiple Compression Capabilities of the Multistate Activities with Project Reliability Approach using the Metaheuristic Algorithm

We examine the time-cost-reliability-project balance of the project by considering the limitations of renewable and nonrenewable resources and performing multistate group activities in the same conditions and with the possibility of multiple compression activities. This study aimed to select the best mode for conducting actions in each subset and find the best execution method by determining the number of compression time units of actions to maximize project reliability and minimize project completion risk, time, and cost. Drilling projects will also be implemented for evaluation in the Azar oil field. The exact Epsilon Limitation method was used to solve this problem in low dimensions. The combined genetic metaheuristic algorithm and particle swarming were used to solve high dimensions problems. The low-dimensional Epsilon solution method could clearly balance the objective functions. Also, the Taguchi experimental design method was used to adjust the parameters of the problem algorithm. The correct value of the algorithm parameters is also specified. Based on the results, the hybrid metaheuristic algorithm has less solution time than the exact method. In high dimensions, as it was observed, GAMS software could not solve the problem. Still, the hybrid metaheuristic algorithm solved the problem well in high dimensions.

A Kernel-Expanded Stochastic Neural Network

Abstract The deep neural network suffers from many fundamental issues in machine learning. For example, it often gets trapped into a local minimum in training, and its prediction uncertainty is hard to be assessed. To address these issues, we propose the so-called kernel-expanded stochastic neural network (K-StoNet) model, which incorporates support vector regression as the first hidden layer and reformulates the neural network as a latent variable model. The former maps the input vector into an infinite dimensional feature space via a radial basis function kernel, ensuring the absence of local minima on its training loss surface. The latter breaks the high-dimensional non-convex neural network training problem into a series of low-dimensional convex optimization problems, and enables its prediction uncertainty easily assessed. The K-StoNet can be easily trained using the imputation-regularized optimization algorithm. Compared to traditional deep neural networks, K-StoNet possesses a theoretical guarantee to asymptotically converge to the global optimum and enables the prediction uncertainty easily assessed. The performances of the new model in training, prediction and uncertainty quantification are illustrated by simulated and real data examples.

An Improved Meerkat Clan Algorithm for Solving 0-1 Knapsack Problem

Meerkat Clan Algorithm (MCA) is a nature-based metaheuristic algorithm which imitates the intelligent behavior of the meerkat animal. This paper presents an improvement on the MCA based on a chaotic map and crossover strategy (MCA-CC). These two strategies increase the diversification and intensification of the proposed algorithm and boost the searching ability to find more quality solutions. The 0-1 knapsack problem was solved by the basic MCA and the improved version of this algorithm (MCA-CC). The performance of these algorithms was tested on low and high dimensional problems. The experimental results demonstrate that the proposed algorithm had overcome the basic algorithm in terms of solution quality, speed and gained optimality with low dimensional problems. Furthermore, in high dimensional problems, it has competitive results in comparison with the other algorithms.

Deep learning based optimization under uncertainty for surfactant-enhanced DNAPL remediation in highly heterogeneous aquifers

The optimization for surfactant-enhanced aquifer remediation (SEAR) of dense non-aqueous phase liquid (DNAPL)-contaminated aquifers is usually accompanied by uncertainties, which may arise from the characterization of complex aquifer heterogeneity and DNAPL source zone architecture (SZA) due to measurement sparsity. Optimization under uncertainty is computationally expensive as it involves an enormous number of model runs. The huge computational burden can be alleviated by utilizing a surrogate model for repeated model evaluations. However, most of the developed surrogates are often limited to low-dimensional optimization problems that only consider simplified aquifer heterogeneity. In this study, we developed a multi-objective simulation-optimization framework to optimize the SEAR schemes considering the characterization uncertainties from both highly heterogeneous aquifer permeability and complex SZA. A fast-to-run convolutional neural network (CNN)-based surrogate model was developed to approximate the high-dimensional and highly complex input-output mapping of the DNAPL multiphase flow simulation model. We first used the rejection sampling strategy to generate random realizations of permeability and SZA conditioning on their limited measurements and then formulated a multi-objective optimization under uncertainty problem based on these realizations. The developed 3-D CNN was trained and used as the surrogate for repeated model runs in optimization to identify the optimal SEAR schemes under uncertainty. A 3-D numerical experiment was used to test the performance of the CNN-based simulation-optimization framework. Comprehensive analysis on the obtained Pareto fronts demonstrates that the proposed framework can efficiently identify reliable Pareto-optimal solutions with a 99.8% speedup compared to the traditional optimization coupled with the forward model. Moreover, the optimization considering multiple realizations enables us to perform the risk assessment to locate the risk zone where the NAPL phase possibly exists after remediation, which provides useful information for decision-making.

Failure of the simultaneous block diagonalization technique applied to complete and cluster synchronization of random networks.

We discuss here the application of the simultaneous block diagonalization (SBD) of matrices to the study of the stability of both complete and cluster synchronization in random (generic) networks. For both problems, we define indices that measure success (or failure) of application of the SBD technique in decoupling the stability problem into problems of lower dimensionality. We then see that in the case of random networks the extent of the dimensionality reduction achievable is the same as that produced by application of a trivial transformation.

Exploring space separation techniques for 3D elastic waves simulations

This paper explores numerical methods dedicated to 3D elastic waves simulations in spatially separable domains such as plates. The objective is to reduce the computation time and the memory requirements associated to these large simulations involving fine space and time discretizations. The 3D problem is decomposed into a sequence of lower dimensional problems with the Proper Generalized Decomposition. The spatial discretization is performed with the Spectral Element Method to provide more compact separated representations compared to the ones obtained with a finite element discretization. Following previous works on space separation in elastodynamics, we explore hybrid explicit/implicit time marching schemes to improve the solution through one direction as needed, without decreasing the time step due to stability constraints. Large 3D numerical problems with several millions of degrees of freedom are efficiently solved with memory requirements characteristic of 2D problems.

Stable and Efficient Petrov--Galerkin Methods for a Kinetic Fokker--Planck Equation

We propose a stable Petrov-Galerkin discretization of a kinetic Fokker-Planck equation constructed in such a way that uniform inf-sup stability can be inferred directly from the variational formulation. Inspired by well-posedness results for parabolic equations, we derive a lower bound for the dual inf-sup constant of the Fokker-Planck bilinear form by means of stable pairs of trial and test functions. The trial function of such a pair is constructed by applying the kinetic transport operator and the inverse velocity Laplace-Beltrami operator to a given test function. For the Petrov-Galerkin projection we choose an arbitrary discrete test space and then define the discrete trial space using the same application of transport and inverse Laplace-Beltrami operator. As a result, the spaces replicate the stable pairs of the continuous level and we obtain a well-posed numerical method with a discrete inf-sup constant identical to the inf-sup constant of the continuous problem independently of the mesh size. We show how the specific basis functions can be efficiently computed by low-dimensional elliptic problems, and confirm the practicability and performance of the method for a numerical example.

Comparison of the performance and reliability between improved sampling strategies for polynomial chaos expansion.

As uncertainty and sensitivity analysis of complex models grows ever more important, the difficulty of their timely realizations highlights a need for more efficient numerical operations. Non-intrusive Polynomial Chaos methods are highly efficient and accurate methods of mapping input-output relationships to investigate complex models. There is substantial potential to increase the efficacy of the method regarding the selected sampling scheme. We examine state-of-the-art sampling schemes categorized in space-filling-optimal designs such as Latin Hypercube sampling and L1-optimal sampling and compare their empirical performance against standard random sampling. The analysis was performed in the context of L1 minimization using the least-angle regression algorithm to fit the GPCE regression models. Due to the random nature of the sampling schemes, we compared different sampling approaches using statistical stability measures and evaluated the success rates to construct a surrogate model with relative errors of <0.1%, <1%, and <10%, respectively. The sampling schemes are thoroughly investigated by evaluating the y of surrogate models constructed for various distinct test cases, which represent different problem classes covering low, medium and high dimensional problems. Finally, the sampling schemes are tested on an application example to estimate the sensitivity of the self-impedance of a probe that is used to measure the impedance of biological tissues at different frequencies. We observed strong differences in the convergence properties of the methods between the analyzed test functions.

Binary social group optimization algorithm for solving 0-1 knapsack problem

In this paper, we propose the binary version of the Social Group Optimization (BSGO) algorithm for solving the 0-1 knapsack problem. The standard Social Group Optimization (SGO) is used for continuous optimization problems. So a transformation function is used to convert the continuous values generated from SGO into binary ones. The experiments are carried out using both low-dimensional and high-dimensional knapsack problems. The results obtained by the BSGO algorithm are compared with other binary optimization algorithms. Experimental results reveal the superiority of the BSGO algorithm in achieving a high quality of solutions over different algorithms and prove that it is one of the best finding algorithms especially in high-dimensional cases.

Dynamic H∞ Consensus of Higher-Order Nonlinear Multi-Agent Systems With General Directed Topology

This paper investigates the leaderless <inline-formula> <tex-math notation="LaTeX">$H_{\infty }$ </tex-math></inline-formula> consensus problem of multi-agent systems with higher-order Lipschitz dynamics and external disturbance. The topology is assumed to be directed. Distributed controllers using only relative outputs information of the adjacent agents are constructed. In order to deal with the non-symmetric property of the associated Laplacian matrix of directed topology, some properties of Laplacian matrix are introduced. By introducing an appropriate state transformation, the <inline-formula> <tex-math notation="LaTeX">$H_{\infty }$ </tex-math></inline-formula> consensus problem is converted to a low-dimensional system stability problem which is solved using Lyapunov stability analysis method. The effectiveness of the proposed control design is demonstrated through simulation examples.

Inverse Design of Two-Dimensional Airfoils Using Conditional Generative Models and Surrogate Log-Likelihoods

Abstract This paper shows how to use conditional generative models in two-dimensional (2D) airfoil optimization to probabilistically predict good initialization points within the vicinity of the optima given the input boundary conditions, thus warm starting and accelerating further optimization. We accommodate the possibility of multiple optimal designs corresponding to the same input boundary condition and take this inversion ambiguity into account when designing our prediction framework. To this end, we first employ the conditional formulation of our previous work BézierGAN–Conditional BézierGAN (CBGAN)—as a baseline, then introduce its sibling conditional entropic BézierGAN (CEBGAN), which is based on optimal transport regularized with entropy. Compared with CBGAN, CEBGAN overcomes mode collapse plaguing conventional GANs, improves the average lift-drag (Cl/Cd) efficiency of airfoil predictions from 80.8% of the optimal value to 95.8%, and meanwhile accelerates the training process by 30.7%. Furthermore, we investigate the unique ability of CEBGAN to produce a log-likelihood lower bound that may help select generated samples of higher performance (e.g., aerodynamic performance). In addition, we provide insights into the performance differences between these two models with low-dimensional toy problems and visualizations. These results and the probabilistic formulation of this inverse problem justify the extension of our GAN-based inverse design paradigm to other inverse design problems or broader inverse problems.

A SWARA-CoCoSo-Based Approach for Spray Painting Robot Selection

In order to avoid working in a constrained hazardous environment, manual spray painting operation is gradually being replaced by automated robotic systems in many manufacturing industries. Application of spray painting robots ensures defect-free painting of dissimilar components with higher repeatability, flexibility, productivity, reduced cycle time and minimum wastage of paint. Due to availability of a large number of viable options in the market, selection of a spray painting robot suitable for a given application poses a great problem. Thus, this paper proposes the integrated application of step-wise weight assessment ratio analysis (SWARA) and combined compromise solution (CoCoSo) methods to identify the most apposite spray painting robot for an automobile industry based on seven evaluation criteria (payload, mass, speed, repeatability, reach, cost and power consumption). The SWARA method identifies cost as the most significant criterion based on a set preference order, whereas, Fanuc P-350iA/45 is selected as the best spray painting robot by CoCoSo method. The derived ranking results are also contrasted with other popular multi-criteria decision making (MCDM) techniques (TOPSIS, VIKOR, COPRAS, PROMETHEE and MOORA) and subjective criteria weighting methods (AHP, PIPRECIA, BWM and FUCOM). High degrees of similarity in the ranking patterns between the adopted approach and other MCDM techniques prove its effectiveness in solving complex industrial robot selection problems. This integrated approach is proved to be quite robust being almost unaffected by the changing values of the corresponding tuning parameter in low-dimensional MCDM problems.