Benchmark Test Problems Research Articles

Revolutionizing population sparsity assessment: machine learning–powered solutions for multi-objective evolutionary algorithms

Decomposed multi-objective evolutionary algorithms have recently gained attention in research, with population sparsity often evaluated through Euclidean distance. However, individuals with high sparsity tend to be located along the edges of the Pareto front, whereas those with low sparsity cluster towards the centre. This article introduces a more accurate method for assessing individual sparsity in the objective space using the density-based spatial clustering of applications with noise (DBSCAN) algorithm. The contributions of this work include (1) identifying the limitations of using Euclidean distance for sparsity measurement, (2) mitigating DBSCAN sensitivity to input parameters through adaptive adjustment via a genetic algorithm and (3) integrating DBSCAN with multi-objective algorithms to address the limitations of Euclidean distance by leveraging core and boundary points. Experimental results on three benchmark test problems and an engineering application involving mechanical bearings highlight the proposed algorithm's strong performance and competitiveness.

Solving Expensive Optimization Problems in Dynamic Environments With Meta-Learning.

Dynamic environments pose great challenges for expensive optimization problems, as the objective functions of these problems change over time and thus require remarkable computational resources to track the optimal solutions. Although data-driven evolutionary optimization and Bayesian optimization (BO) approaches have shown promise in solving expensive optimization problems in static environments, the attempts to develop such approaches in dynamic environments remain rarely explored. In this article, we propose a simple yet effective meta-learning-based optimization framework for solving the expensive dynamic optimization problems. This framework is flexible, allowing any off-the-shelf continuously differentiable surrogate model to be used in a plug-in manner, either in data-driven evolutionary optimization or BO approaches. In particular, the framework consists of two unique components: 1) the meta-learning component, in which a gradient-based meta-learning approach is adopted to learn experience (effective model parameters) across different dynamics along the optimization process and 2) the adaptation component, where the learned experience (model parameters) is used as the initial parameters for fast adaptation in the dynamic environment based on few shot samples. By doing so, the optimization process is able to quickly initiate the search in a new environment within a strictly restricted computational budget. Experiments demonstrate the effectiveness of the proposed algorithm framework compared to several state-of-the-art algorithms on common benchmark test problems under different dynamic characteristics.

Enhanced Projection Method for the Solution of the System of Nonlinear Equations Under a More General Assumption than Pseudo-Monotonicity and Lipschitz Continuity

In this manuscript, we propose an efficient algorithm for solving a class of nonlinear operator equations. The algorithm is an improved version of previously established method. The algorithm’s features are as follows: (i) the search direction is bounded and satisfies the sufficient descent condition; (ii) the global convergence is achieved when the operator is continuous and satisfies a condition weaker than pseudo-monotonicity. Moreover, by comparing it with previously established method the algorithm’s efficiency was shown. The comparison was based on the iteration number required for each algorithm to solve a particular problem and the time taken. Some benchmark test problems, which included monotone and pseudo-monotone problems, were considered for the experiments. Lastly, the algorithm was utilized to solve the logistic regression (prediction) model.

Ranking and selection with two-stage decision

Ranking & selection (R&S) is concerned with the selection of the best decision from a finite set of alternative decisions when the outcome of the decision has to be estimated using stochastic simulation. In this paper, we extend the R&S problem to a two-stage setting where after a first-stage decision has been made, some information may be observed and a second-stage decision then needs to be made based on the observed information to achieve the best outcome. We then extend two popular single-stage R&S algorithms, expected value of information (EVI) and optimal computing budget allocation (OCBA), to efficiently solve the new two-stage R&S problem. We prove the consistency of the new two-stage EVI (2S-EVI) and OCBA (2S-OCBA) algorithms. Experiment results on benchmark test problems and a two-stage multi-product assortment problem show that both algorithms outperform applying single-stage EVI and OCBA in the two-stage setting. Between 2S-EVI and 2S-OCBA, numerical results suggest that 2S-EVI tends to perform better with smaller number of decisions at first and second stage while 2S-OCBA has better performance for larger problems.

Dynamic Multiobjective Optimization Based on Multi-Environment Knowledge Selection and Transfer

Background: Dynamic multiobjective optimization problems (DMOPs) involve multiple conflicting and time-varying objectives, and dynamic multiobjective algorithms (DMOAs) aim to find Pareto optima that are closer to the real one in the new environment as soon as possible. In particular, the introduction of transfer learning in DMOAs has led to good results in solving DMOPs. However, the selection of valuable historical knowledge and the mitigation of negative transfer remain important problems in existing transfer learning-based DMOAs. Method: A DMOA based on multi-environment knowledge selection and transfer (MST-DMOA) is proposed in this article. First, by clustering historical Pareto optima, some representative solutions that can reflect the main evolutionary information are selected as knowledge of the environment. Second, the similarity between the historical and current environments is evaluated, and then the knowledge of multiple similar environments is selected as valuable historical knowledge to construct the source domain. Third, solutions with high quality in the new environment are obtained to form the target domain, which can better help historical knowledge to adapt to the current environment, thus effectively alleviating negative transfer. Conclusions: We compare the proposed MST-DMOA with five state-of-the-art DMOAs on fourteen benchmark test problems, and the experimental results verify the excellent performance of MST-DMOA in solving DMOPs.

A Hybrid Conjugate Gradient Algorithm for Nonlinear System of Equations through Conjugacy Condition

For the purpose of solving a large-scale system of nonlinear equations, a hybrid conjugate gradient algorithm is introduced in this paper, based on the convex combination of βFR k and βPRP k parameters. It is made possible by incorporating the conjugacy condition together with the proposed conjugate gradient search direction. Furthermore, a significant property of the method is that through a non-monotone type line search it gives a descent search direction. Under appropriate conditions, the algorithm establishes its global convergence. Finally, results from numerical tests on a set of benchmark test problems indicate that the method is more effective and robust compared to some existing methods.

Adaptive Constraint Relaxation-Based Evolutionary Algorithm for Constrained Multi-Objective Optimization

The key problem to solving constrained multi-objective optimization problems (CMOPs) is how to achieve a balance between objectives and constraints. Unfortunately, most existing methods for CMOPs still cannot achieve the above balance. To this end, this paper proposes an adaptive constraint relaxation-based evolutionary algorithm (ACREA) for CMOPs. ACREA adaptively relaxes the constraints according to the iteration information of population, whose purpose is to induce infeasible solutions to transform into feasible ones and thus improve the ability to explore the unknown regions. Completely ignoring constraints can cause the population to waste significant resources searching for infeasible solutions, while excessively satisfying constraints can trap the population in local optima. Therefore, balancing constraints and objectives is a crucial approach to improving algorithm performance. By appropriately relaxing the constraints, it induces infeasible solutions to be transformed into feasible ones, thus obtaining more information from infeasible solutions. At the same time, it also establishes an archive for the storage and update of solutions. In the archive update process, a diversity-based ranking is proposed to improve the convergence speed of the algorithm. In the selection process of the mating pool, common density selection metrics are incorporated to enable the algorithm to obtain higher-quality solutions. The experimental results show that the proposed ACREA algorithm not only achieved the best Inverse Generation Distance (IGD) value in 54.6% of the 44 benchmark test problems and the best Hyper Volume (HV) value in 50% of them, but also obtained the best results in seven out of nine real-world problems. Clearly, CP-TSEA outperforms its competitors.

ThunderBoltz: an open-source direct simulation Monte Carlo Boltzmann solver for plasma transport, chemical kinetics, and 0D modeling

Abstract Plasma-neutral interactions, including reactive kinetics, are often either studied in 0D using ODE-based descriptions, or in multi-dimensional fluid or particle-based plasma codes. The latter case involves a complex assembly of procedures that are not always necessary to test effects of underlying physical models and mechanisms for particle-based descriptions. Here we present ThunderBoltz, a lightweight, publicly available 0D direct simulation Monte Carlo code designed to accommodate a generalized combination of species and arbitrary cross sections without the overhead of expensive field solves. It can produce electron, ion, and neutral velocity distributions in applied AC/DC E-field and/or static B-field scenarios. The code is built in the C++ standard library and includes a convenient Python interface that allows for input file generation (including compatibility with cross section data from the LXCat database), electron transport and reaction rate constant post-processing, input parameter constraint satisfaction, calculation scheduling, and diagnostic plotting. These codes can be accessed at the repository: https://github.com/lanl/ThunderBoltz. In this work we compare ThunderBoltz transport calculations against Bolsig+ calculations, benchmark test problems, and swarm experiment data, finding good agreement with all three in the appropriate field regimes. In addition, we present example use cases where the electron, ion, and background neutral particle species are self-consistently evolved to model the background kinetics, a feature that is absent in fixed-background Monte Carlo and n-term Boltzmann solvers. The latter functionality allows for the possibility of particle-based chemical kinetics simulations of the plasma and neutral species as a new alternative to ODE-based approaches.

Deep reinforcement learning as multiobjective optimization benchmarks: Problem formulation and performance assessment

The successful deployment of Deep learning in several challenging tasks has been translated into complex control problems from different domains through Deep Reinforcement Learning (DRL). Although DRL has been extensively formulated and solved as single-objective problems, nearly all real-world RL problems often feature two or more conflicting objectives, where the goal is to obtain a high-quality and diverse set of optimal policies for different objective preferences. Consequently, the development of Multi-Objective Deep Reinforcement Learning (MODRL) algorithms has gained a lot of traction in the literature. Generally, Evolutionary Algorithms (EAs) have been demonstrated to be scalable alternatives to the classical DRL paradigms when formulated as an optimization problem. Hence it is reasonable to employ Multi-objective Evolutionary Algorithms (MOEAs) to handle MODRL tasks. However, there are several factors constraining the progress of research along this line: first, there is a lack of a general problem formulation of MODRL tasks from an optimization perspective; second, there exist several challenges in performing benchmark assessments of MOEAs for MODRL problems. To overcome these limitations: (i) we present a formulation of MODRL tasks as general multi-objective optimization problems and analyze their complex characteristics from an optimization perspective; (ii) we present an end-to-end framework, termed DRLXBench, to generate MODRL benchmark test problems for seamless running of MOEAs (iii) we propose a test suite comprising of 12 MODRL problems with different characteristics such as many-objectives, degenerated Pareto fronts, concave and convex optimization problems, etc. (iv) Finally, we present and discuss baseline results on the proposed test problems using seven representative MOEAs.

A New Variant of the Conjugate Descent Method for Solving Unconstrained Optimization Problems and Applications

Unconstrained optimization problems have a long history in computational mathematics and have been identified as being among the crucial problems in the fields of applied sciences, engineering, and management sciences. In this paper, a new variant of the conjugate descent method for solving unconstrained optimization problems is introduced. The proposed algorithm can be seen as a modification of the popular conjugate descent (CD) algorithm of Fletcher. The algorithm of the proposed method is well-defined, and the sequence of the directions of search is shown to be sufficiently descending. The convergence result of the proposed method is discussed under the common standard conditions. The proposed algorithm together with some existing ones in the literature is implemented to solve a collection of benchmark test problems. Numerical experiments conducted show the performance of the proposed method is very encouraging. Furthermore, an additional efficiency evaluation is carried out on problems arising from signal processing and it works well.

Local spherical collapsing box in Athena++ : Numerical implementation and benchmark tests

We implement a local model for a spherical collapsing or expanding gas cloud in the Athena++ magnetohydrodynamic code. This local model consists of a Cartesian periodic box with time-dependent geometry. We present a series of benchmark test problems, including nonlinear solutions and linear perturbations of the local model, confirming the code's desired performance. During a spherical collapse, a horizontal shear flow is amplified, corresponding to angular momentum conservation of zonal flows in the global problem; wave speed and the amplitude of sound waves increase in the local frame, due to the reduction in the characteristic length scale of the box, which can lead to an anisotropic effective sound speed in the local box. Our code conserves both mass and momentum-to-machine precision. This numerical implementation of the local model has potential applications to the study of local physics and hydrodynamic instabilities during protostellar collapse, providing a powerful framework for better understanding the earliest stages of star and planet formation.

Solving dynamic multimodal optimization problems via a niching-based brain storm optimization with two archives algorithm

Dynamic and multimodal properties are simultaneously possessed in the dynamic multimodal optimization problems (DMMOPs), which aim to find multiple optimal solutions in a dynamic environment. However, more work still needs to be devoted to solving DMMOPs, which still require significant attention. A niching-based brain storm optimization with two archives (NBSO2A) algorithm is proposed to solve DMMOPs. The two niching methods, i.e., neighborhood-based speciation (NS), and nearest-better clustering (NBC), are incorporated into a BSO algorithm to generate new solutions. The two archives preserve the optimal solutions that meet the requirements and practical, inferior solutions discarded during the generation. Improved taboo area (ITA) removes highly similar individuals from the population. An evolution strategy with covariance matrix adaptation (CMA-ES) is adopted to enhance the local search ability and improve the quality of the solutions. The NBSO2A algorithm and four other algorithms were tested on 12 benchmark problems to validate the performance of the NBSO2A algorithm on DMMOPs. The experimental results show that the NBSO2A algorithm outperforms the other compared algorithms on most tested benchmark problems.

An improved gray wolf optimization algorithm solving to functional optimization and engineering design problems

As a newly proposed optimization algorithm based on the social hierarchy and hunting behavior of gray wolves, grey wolf algorithm (GWO) has gradually become a popular method for solving the optimization problems in various engineering fields. In order to further improve the convergence speed, solution accuracy, and local minima escaping ability of the traditional GWO algorithm, this work proposes a multi-strategy fusion improved gray wolf optimization (IGWO) algorithm. First, the initial population is optimized using the lens imaging reverse learning algorithm for laying the foundation for global search. Second, a nonlinear control parameter convergence strategy based on cosine variation is proposed to coordinate the global exploration and local exploitation ability of the algorithm. Finally, inspired by the tunicate swarm algorithm (TSA) and the particle swarm algorithm (PSO), a nonlinear tuning strategy for the parameters, and a correction based on the individual historical optimal positions and the global optimal positions are added in the position update equations to speed up the convergence of the algorithm. The proposed algorithm is assessed using 23 benchmark test problems, 15 CEC2014 test problems, and 2 well-known constraint engineering problems. The results show that the proposed IGWO has a balanced E&P capability in coping with global optimization as analyzed by the Wilcoxon rank sum and Friedman tests, and has a clear advantage over other state-of-the-art algorithms.

A hyperbolic–elliptic PDE model and conservative numerical method for gravity-dominated variably-saturated groundwater flow

Richards equation is often used to represent two-phase fluid flow in an unsaturated porous medium when one phase is much heavier and more viscous than the other. However, it cannot describe the fully saturated flow for some capillary functions without specialized treatment due to degeneracy in the capillary pressure term. Mathematically, gravity-dominated variably saturated flows are interesting because their governing partial differential equation switches from hyperbolic in the unsaturated region to elliptic in the saturated region. Moreover, the presence of wetting fronts introduces strong spatial gradients often leading to numerical instability. In this work, we develop a robust, multidimensional mathematical model and implement a well-known efficient and conservative numerical method for such variably saturated flow in the limit of negligible capillary forces. The elliptic problem in saturated regions is integrated efficiently into our framework by solving a reduced system corresponding only to the saturated cells using fixed head boundary conditions in the unsaturated cells. In summary, this coupled hyperbolic–elliptic PDE framework provides an efficient, physics-based extension of the hyperbolic Richards equation to simulate fully saturated regions. Finally, we provide a suite of easy-to-implement yet challenging benchmark test problems involving saturated flows in one and two dimensions. These simple problems, accompanied by their corresponding analytical solutions, can prove to be pivotal for the code verification, model validation (V&V) and performance comparison of simulators for variably saturated flow. Our numerical solutions show an excellent comparison with the analytical results for the proposed problems. The last test problem on two-dimensional infiltration in a stratified, heterogeneous soil shows the formation and evolution of multiple disconnected saturated regions.

Evolutionary Bilevel Optimization via Multiobjective Transformation-Based Lower-Level Search

Nested evolutionary algorithms (EAs) have been regarded as very promising tools for bi-level optimization. Due to the nested structure, the upper level population evaluation requires a set of complete lower level optimizations, thereby reducing the efficiency and practicability of EA methods. In this paper, a multi-objective transformation-based evolutionary algorithm (MOTEA) is proposed to perform multiple lower level optimizations in a parallel and collaborative manner. Specifically, the corresponding multiple lower level optimizations for each generation of the upper level population evaluation are transformed into locating a set of Pareto optimal solutions of a constructed multi-objective optimization problem. By utilizing the built-in implicit parallelism of evolutionary multi-objective optimization, multiple lower level problems can thus be optimized in parallel. Within one multi-objective search population, the collaboration among the parallel lower level optimization can be realized by exploiting and utilizing the implicit similarities among them for better efficiency. The effectiveness and efficiency of the proposed MOTEA are verified by comparing it with four state-of-the-art evolutionary bi-level optimization algorithms on two sets of popular bi-level optimization benchmark test problems and three application problems.

Neural Architecture Search as Multiobjective Optimization Benchmarks: Problem Formulation and Performance Assessment

The ongoing advancements in network architecture design have led to remarkable achievements in deep learning across various challenging computer vision tasks. Meanwhile, the development of neural architecture search (NAS) has provided promising approaches to automating the design of network architectures for lower prediction error. Recently, the emerging application scenarios of deep learning (e.g., autonomous driving) have raised higher demands for network architectures considering multiple design criteria: number of parameters/weights, number of floating-point operations, inference latency, among others. From an optimization point of view, the NAS tasks involving multiple design criteria are intrinsically multiobjective optimization problems; hence, it is reasonable to adopt evolutionary multiobjective optimization (EMO) algorithms for tackling them. Nonetheless, there is still a clear gap confining the related research along this pathway: on the one hand, there is a lack of a general problem formulation of NAS tasks from an optimization point of view; on the other hand, there are challenges in conducting benchmark assessments of EMO algorithms on NAS tasks. To bridge the gap: (i) we formulate NAS tasks into general multi-objective optimization problems and analyze the complex characteristics from an optimization point of view; (ii) we present an end-to-end pipeline, dubbed EvoXBench, to generate benchmark test problems for EMO algorithms to run efficiently -without the requirement of GPUs or Pytorch/Tensorflow; (iii) we instantiate two test suites comprehensively covering two datasets, seven search spaces, and three hardware devices, involving up to eight objectives. Based on the above, we validate the proposed test suites using six representative EMO algorithms and provide some empirical analyses. The code of EvoXBench is available at https://github.com/EMI-Group/EvoXBench.

Determination of Timewise-Source Coefficient in Time-Fractional Reaction-Diffusion Equation from First Order Heat Moment

This article aims to determine the time-dependent heat coefficient together with the temperature solution for a type of semi-linear time-fractional inverse source problem by applying a method based on the finite difference scheme and Tikhonov regularization. An unconditionally stable implicit finite difference scheme is used as a direct (forward) solver. While by the MATLAB routine lsqnonlin from the optimization toolbox, the inverse problem is reformulated as nonlinear least square minimization and solved efficiently. Since the problem is generally incorrect or ill-posed that means any error inclusion in the input data will produce a large error in the output data. Therefore, the Tikhonov regularization technique is applied to obtain stable and accurate results. Finally, to demonstrate the accuracy and effectiveness of our scheme, two benchmark test problems have been considered, and its good working with different noise levels.

Performance prediction and multi-objective optimization for the Atkinson cycle engine using eXtreme Gradient Boosting

In this study, substantial efforts have been done to enhance both the economy and nitrogen dioxide (NOx) emission characteristics of an Atkinson cycle engine (ACE). The integrated simulation model was meticulously formulated and calibrated utilizing GT-Power software based on test data, and four machine learning (ML) models were developed to predict the performance of the ACE. On this basic, the improved multi-objective hybrid particle swarm optimization (IMHPSO) algorithm was proposed to conduct comprehensive optimization of ACE performance. The results highlight the superior prediction performance and generalization ability of the eXtreme Gradient Boosting (XGBoost) model, attaining an average coefficient of determination (R2) of 0.9978 and 0.9966 on the train and test sets, respectively. The IMHPSO algorithm exhibited efficient convergence, uniform distribution and rich diversity across five benchmark test problems, effectively mitigating issues related to local optima and premature convergence. The Optimal Brake Specific Fuel Consumption (BSFC) solution is chosen as the preferred optimized scheme and realizes significant reductions of 8.5 % for BSFC and 96.1 % for NOx emissions, which experiences a remarkable increase of over 1300 times in computational efficiency compared to the GT-Power model. These findings provide theoretical basis, data support and novel insights for multi-objective optimization of ACE performance.

Direct integral pseudospectral and integral spectral methods for solving a class of infinite horizon optimal output feedback control problems using rational and exponential Gegenbauer polynomials

This study is concerned with the numerical solution of a class of infinite-horizon linear regulation problems with state equality constraints and output feedback control. We propose two numerical methods to convert the optimal control problem into nonlinear programming problems (NLPs) using collocations in a semi-infinite domain based on rational Gegenbauer (RG) and exponential Gegenbauer (EG) basis functions. We introduce new properties of these basis functions and derive their quadratures and associated truncation errors. A rigorous stability analysis of the RG and EG interpolations is also presented. The effects of various parameters on the accuracy and efficiency of the proposed methods are investigated. The performance of the developed integral spectral method is demonstrated using two benchmark test problems related to a simple model of a divert control system and the lateral dynamics of an F-16 aircraft. Comparisons of the results of the current study with available numerical solutions show that the developed numerical scheme is efficient and exhibits faster convergence rates and higher accuracy.

High-Order ADER Discontinuous Galerkin Schemes for a Symmetric Hyperbolic Model of Compressible Barotropic Two-Fluid Flows

This paper presents a high-order discontinuous Galerkin (DG) finite-element method to solve the barotropic version of the conservative symmetric hyperbolic and thermodynamically compatible (SHTC) model of compressible two-phase flow, introduced by Romenski et al. in [59, 62], in multiple space dimensions. In the absence of algebraic source terms, the model is endowed with a curl constraint on the relative velocity field. In this paper, the hyperbolicity of the system is studied for the first time in the multidimensional case, showing that the original model is only weakly hyperbolic in multiple space dimensions. To restore the strong hyperbolicity, two different methodologies are used: (i) the explicit symmetrization of the system, which can be achieved by adding terms that contain linear combinations of the curl involution, similar to the Godunov-Powell terms in the MHD equations; (ii) the use of the hyperbolic generalized Lagrangian multiplier (GLM) curl-cleaning approach forwarded. The PDE system is solved using a high-order ADER-DG method with a posteriori subcell finite-volume limiter to deal with shock waves and the steep gradients in the volume fraction commonly appearing in the solutions of this type of model. To illustrate the performance of the method, several different test cases and benchmark problems have been run, showing the high order of the scheme and the good agreement when compared to reference solutions computed with other well-known methods.