Expensive Problems Research Articles

A Stage-Specific Gene Expression Framework for Promoting Genetic Algorithm

Genetic algorithm (GA) has become very popular for tackling computationally expensive numerical optimization problems. To optimize their performance and convergence rate, we propose and demonstrate an embedded stage-specific gene expression framework (SGEF) with a two-tier coding scheme and a filter operator based on variational auto-encoder. For the first time, the structure of variational auto-encoder is used to map the parent population of a genetic algorithm from the uniformly distributed high dimensions to the Gaussian distributed low dimensions. The design of the filter operator which can maintain the trade-off balance between exploitation and exploration is also analyzed in detail. The experimental results demonstrated the good performance of the proposed SGEF in promoting both single-objective optimization genetic algorithms and multiple-objective optimization genetic algorithms.

Enumeration PCRLB-Based Power Allocation for Multitarget Tracking With Colocated MIMO Radar Systems in Clutter

An effective resource allocation strategy can maximize the remote sensing performance of radar systems, such as target detection and tracking. In this paper, two typical power allocation (PA) strategies are developed for the multi-target tracking (MTT) task in colocated MIMO (C-MIMO) radar systems with the consideration of the clutter. The multi-beam concept and the posterior PDF fusion are adopted by the C-MIMO radar system to obtain the global posterior distribution. Specifically, each radar generates multiple simultaneous beams with controllable power during each interval. To ensure that the limited system resources can be utilized effectively, the online PA scheme is implemented according to the prior knowledge predicted from the tracking cyclic recursive feedback results. The posterior Cramér-Rao lower bound (PCRLB) is derived by enumerating all possible target detection and false alarm occurrence cases, and is utilized as the tracking performance metric since it provides a more accurate lower bound on the target state estimation in clutter. Besides, to solve the computationally expensive problem of this PCRLB caused by enumeration operation, we propose a two-step approximate approach. Then, combined with the system resource configuration, two different types of resource optimization problems are designed, namely, performance maximization for a fixed power budget and direct resource minimization. These formulated PA problems are shown to be non-convex and non-linear. Therefore, we further propose a modified particle swarm optimization (MPSO) algorithm to solve these problems efficiently. Simulation results verify the superiority and effectiveness of the proposed PA strategies in terms of tracking performance in clutter.

Reference Vector Assisted Candidate Search with Aggregated Surrogate for Computationally Expensive Many Objective Optimization Problems

Pareto-optimal sets of multiobjective optimization problems with black-box and computationally expensive objective functions are generally hard to locate within a limited computational budget, and this situation gets even worse when more than three objectives are involved. To this end, we present a novel surrogate-assisted many-objective optimization algorithm RECAS. Unlike most prior studies, the proposed algorithm is a non–evolutionary-based method, and it iteratively determines new points for expensive evaluation via a series of independent reference vector assisted candidate searches. Furthermore, to make the number of surrogates to be maintained independent of the number of objectives, in each candidate search, RECAS constructs a surrogate model in an aggregated manner to approximate the quality assessment indicator of each point rather than a certain objective function. Under some mild assumptions, this study proves that RECAS converges almost surely to the Pareto-optimal front. In the numerical experiments, the effectiveness and reliability of RECAS are examined on both DTLZ and WFG test suites with the number of objectives varying from 2 to 10. Compared with six state-of-the-art many-objective optimization algorithms, RECAS generally performs better in maintaining convergent and well-spread approximation of the Pareto-optimal front. Finally, the good performance of RECAS on two watershed simulation model calibration problems indicates its great potential in handling real-world applications. History: Accepted by Antonio Frangioni, Area Editor for Design and Analysis of Algorithms–Continuous. Funding: This work was supported by the National University of Singapore start-up grant to C.A. Shoemaker [Grant R-266-000-109-133]. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplementary Information [ https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2022.1260 ] or is available from the IJOC GitHub software repository ( https://github.com/INFORMSJoC ) at [ http://dx.doi.org/10.5281/zenodo.7243971 ].

Data-driven Harris Hawks constrained optimization for computationally expensive constrained problems

Aiming at the constrained optimization problem where function evaluation is time-consuming, this paper proposed a novel algorithm called data-driven Harris Hawks constrained optimization (DHHCO). In DHHCO, Kriging models are utilized to prospect potentially optimal areas by leveraging computationally expensive historical data during optimization. Three powerful strategies are, respectively, embedded into different phases of conventional Harris Hawks optimization (HHO) to generate diverse candidate sample data for exploiting around the existing sample data and exploring uncharted region. Moreover, a Kriging-based data-driven strategy composed of data-driven population construction and individual selection strategy is presented, which fully mines and utilizes the potential available information in the existing sample data. DHHCO inherits and develops HHO's offspring updating mechanism, and meanwhile exerts the prediction ability of Kriging, reduces the number of expensive function evaluations, and provides new ideas for data-driven constraint optimization. Comprehensive experiments have been conducted on 13 benchmark functions and a real-world expensive optimization problem. The experimental results suggest that the proposed DHHCO can achieve quite competitive performance compared with six representative algorithms and can find the near global optimum with 200 function evaluations for most examples. Moreover, DHHCO is applied to the structural optimization of the internal components of the real underwater vehicle, and the final satisfactory weight reduction effect is more than 18%.

Surrogate-assisted hybrid evolutionary algorithm with local estimation of distribution for expensive mixed-variable optimization problems

Some real-world design optimization problems can be formulated as expensive mixed-variable optimization problems (EMVOPs), which involve both continuous and discrete decision variables and expensive function evaluations. The main challenges for solving EMVOPs are the handling of mixed variables, limited number of function evaluations and multiple disconnected regions in the search space. In this work, we propose a novel algorithm with global and local search strategies for improving the search ability on disconnected regions, and it only consumes hundreds of function evaluations. The global module employs hybrid evolutionary operators and a Gower distance based surrogate model for handling mixed variables. The local estimation of distributions in different local regions performs in a competitive switching way to combine their advantages, and local surrogate models trained with selected samples improve the accuracy of approximated evaluations for the locally generated solutions. In the late stage, a local continuous search module is executed for refining the continuous decision variables. Verification results on the artificial benchmarks demonstrate that our proposed algorithm is competitive and works effectively. To verify the practicability of the algorithm, it is applied on a convolutional neural network hyperparameter optimization problems and obtains satisfactory results.

An Analysis of the RBF Hyperparameter Impact on Surrogate-Assisted Evolutionary Optimization

Computationally expensive optimization problems are often solved using surrogates and a common variant is the radial basis functions (RBF) model. It aggregates several basis functions which all depend on a hyperparameter affecting their individual outputs and consequentially the overall surrogate prediction. However, the optimal value of the hyperparameter is typically unknown and should therefore be calibrated. This raises the question how does the hyperparameter affect the overall optimization search effectiveness (and not just the stand-alone surrogate accuracy) and to what extent is such a calibration beneficial, which is an important consideration both for end-users and algorithm researchers alike. To rigorously address this issue this paper presents an analysis based on an extensive set of numerical experiments with an RBF surrogate-assisted evolutionary algorithm. It follows that the hyperparameter strongly affected performance and that the extent of its impact varied depending on the basis function, objective function modality, and problem dimension. Overall, calibration of the hyperparameter was typically highly beneficial to the search performance while dynamically optimizing the hyperparameter during the search yielded additional performance gains.

PRETTY: A parallel transgenerational learning-assisted evolutionary algorithm for computationally expensive multi-objective optimization

Many real-world optimization problems, called computationally expensive optimization problems, often require a time-consuming fitness evaluation through computer simulation or neural network training. A persistent challenge is to obtain reasonable solutions to these problems quickly. In this study, we design a parallel transgenerational learning-assisted evolutionary algorithm, called PRETTY, to accelerate the search process using three novel mechanisms. First, we fuse the transgenerational learning mechanism with metaheuristic algorithms to reduce unnecessary fitness evaluations. Then, a Hypervolume-Cauchy (HV-Cauchy) algorithm is proposed to balance population convergence and diversity adaptively. Finally, a parallel-computing framework is proposed to fuse the transgenerational learning mechanism and the HV-Cauchy algorithm into a multipopulation evolutionary algorithm. To verify the performance of the proposed algorithm, empirical experiments are conducted on nine benchmarks and four real-world problems, including feature selection, critical node detection, flocking, and neural network training. In terms of execution time, the experimental results demonstrate that PRETTY has significant advantages over several expensive multi-objective algorithms. Furthermore, PRETTY can maintain population diversity and fast convergence while accelerating the computation. In particular, in tackling real-world problems that require expensive fitness calculations, PRETTY can reduce the execution time by more than five times while ensuring high-quality solutions.

Implementing Jamaica’s Blue Economy Future: Understanding Stakeholder Perceptions, Risks and Opportunities, Along With Associated Implications for Marine Ecosystems, Resource Management

Globally and locally, whilst the blue economy may be the latest or last investment frontier in the absence of pursing space or mass scale resource recycling or the circular economy; understanding the mechanisms, core risks, opportunities, priorities and insights of stakeholders remains far less understood. In contrast to myriad studies on conventional economy, green economy and individual sectors comprising the blue economy; most holistic attempts focus on definitions rather than specific case studies. In a world professing the need for greater transparency, many experiences remain opaque, the subject of consultancies, aid donors, investors and policymakers or business supply chains, often working with a segmented, silo mentality rather than holistic vision. Yet, this can lead to expensive problems of delays, unimplemented or duplicated projects, failures to effective coordinate and prioritise risks or opportunities, often squandering scarce resources, especially compounding the issues of smaller islands/developing nation states. This despite increasing significance and attention committed to the blue economy, increasing the challenges and adjustment expense, maximising disruption risks. Through desktop research and interviews/surveys with 89 stakeholders, this research based on a January to September 2022, World Bank consultancy, aims to provide a pioneering insight of a coordinated effort. In 2020 Jamaica’s blue economy contributed 19.5% of 2020 GDP or US $2,544 billion despite the challenges of the COVID-19 pandemic. Blue economy wages reached 18.7% of 2020 GNI. Total direct employment was calculated at 514,000 or 36.8% of the employed labour force. Despite initially fragmented legislation, finance and stakeholders, 25% stakeholders primarily recommended investing in additional human resource capacity (25%), marine pollution, and 15% each in IUU fisheries, offering specific leadership and support politically and legally/from a policy perspective. 41% recommended investing in growing mariculture. 35%: valued Marine Protected Areas (MPAs) and fish sanctuaries, blue bonds, blue carbon and biodiversity offsets. 30%: advise cultural, heritage, adventure, eco and nautical/yachting tourism opportunities. floating boardwalk / Caribbean smaller ports, luxury-high end tourism markets and cultural events. 30% favoured marine renewable energy and energy /eco-efficiency. 20% proposed alternative fuels, biofuels and decarbonisation of ports, vessels. 18% expressed hope in plastics/other forms of recycling and circular economy to address marine pollution. This provides the basis for improving decision-making, marine resource management, ocean governance, reduced conflicts and international cooperation for a more effective blue economy implementation for Jamaica and the Caribbean.

Context and mitigation of lost circulation during geothermal drilling in diverse geologic settings

Lost circulation is one of the most common and expensive problems facing geothermal energy development, representing up to 30% of drilling costs. We examined drilling records from four geothermal fields—McGinness Hills in central Nevada, Don A. Campbell and Steamboat Hills in western Nevada, and Puna Geothermal Venture on the Big Island of Hawai'i— to identify geologies most prone to lost circulation, as well as common mitigation strategies. Depths of lost circulation events varied, but their frequency often increased in the production interval. Lost circulation commonly occurred near fault intersections, and heavily faulted fields like McGinness Hills and Don A. Campbell showed secondary mineralization within approximately 100 m (328 ft) or less of where circulation was lost. Lost circulation mitigation strategies included using locally available materials (e.g., cotton seed hulls) as well as more expensive proprietary lost circulation materials, cement plugs above the reservoir, and drilling blind with aerated, polymer-based mud in the production zone. Addressing lost circulation using a well thought out decision-making approach and materials above the reservoir will save time and cost, and provide needed well integrity. Mitigation often requires a series of steps, typically applied from perceived least expensive to most, and are dependent on the severity and location in the well where circulation was lost and availability of materials. Placing cement plugs can cure lost circulation events, however these plugs are often expensive, time-consuming, and may not be successful.

Pharmaceutical industry and academia linkage in Pakistan: a prevalent challenge

The pharmaceutical industry and academia together play a pivotal role in addressing drug-associated issues, formulation, and processing errors. Therefore, the waste of expensive therapeutic moieties and problems such as batch failure during pharmaceutical dosage form production processes can easily be overcome. This partnership can also reduce the economic burden at the patient’s end. Understanding that trust is a prime factor in establishing strong linkages between the pharmaceutical industry and academia is pertinent. However, before making such commitments, organizations must revisit their capacities. Academia should focus on updating and improving the existing curriculum by including subjects with practical significance, such as biotechnology, rheology, particle processing, formulation technologies, and material sciences, to promote problem-based learning among students.

Practical initialization of the Nelder–Mead method for computationally expensive optimization problems

Black-box optimization (BBO) algorithms are widely employed by practitioners to address computationally expensive real-world problems such as automatic tuning of machine learning models and evacuation route planning. The Nelder–Mead (NM) method is a well-known local search heuristic for BBO that has been applied to solve many real-world problems from way back because of its promising performance. However, this method has a strong dependence on initialization due to its local search tendency. Nevertheless, a discussion on the proper initialization of the NM method is limited to the recent study by Wessing (Optim Lett 13(4):847–856, 2019), which is solely based on an analysis using the simple sphere function. In this study, we take a further step to improve Wessing’s result by massively investigating how the initialization affects the search performance in views of the initial simplex size and shape and a constraint handling method that is employed on 24 BBO benchmarking problems. Based on the numerical results, we present the empirical best practice for the initialization of the NM method for cases involving a limited evaluation budget.

A surrogate-assisted differential evolution for expensive constrained optimization problems involving mixed-integer variables

Many Surrogate-Assisted Evolutionary Algorithms (SAEAs) have been developed for expensive constrained optimization problems (ECOPs) with continuous variables. However, there exist some ECOPs that contain mixed-integer variables in real world engineering applications. The mixed-integer variables make the feasible regions of ECOPs exhibit complex disjoint characteristics, which brings two challenges for the existing methods, i.e., the classical surrogate-assisted pre-screening may easily fall into a local optimum and the population diversity tends to reduce with the influence of integer variables. For tackling these problems, a surrogate-assisted differential evolution for solving ECOPs involving mixed-integer variables (SADE-MI) is proposed. Firstly, the mutation strategy DE/best/2 is used to balance the local and global characteristics of the candidate solutions. Secondly, an adaptive pre-screening operation where the information of the population is effectively employed to construct the distance constraints is designed to avoid excessive exploitation in local regions. Finally, by selecting diversely-located individuals and limiting the number of individuals with the same integer variables, a population diversity maintenance operation is introduced to strengthen global exploration. Experimental studies on some benchmark problems and engineering optimization cases demonstrate that SADE-MI outperforms other classical algorithms for solving ECOPs with mixed-integer variables.

Surrogate-assisted hierarchical learning water cycle algorithm for high-dimensional expensive optimization

Excessive function evaluations are the main obstacle preventing the application of metaheuristic algorithms to expensive real-world problems. Although many surrogate-assisted metaheuristic algorithms have been proposed to tackle this challenge, most of them still suffer from low prediction accuracy on high-dimensional problems. This article presents a surrogate-assisted hierarchical learning water cycle algorithm (SA-HLWCA) for high-dimensional expensive optimization problems. SA-HLWCA utilizes two searching modes, namely, global search and local search, to cooperatively search for the optimal solution. A global search is conducted by a surrogate using a global diverse subset and aims to locate promising optimal areas. A local search is conducted by a surrogate built in the neighboring region of the current best and aims to refine the optimal solution. To validate the performance, comprehensive studies of the impacts of major components of SA-HLWCA are conducted. The proposed algorithm is then compared with three state-of-the-art algorithms on a series of tests on problems that range from 20 dimensions to 100 dimensions, and the results show that SA-HLWCA performs better in terms of both effectiveness and robustness. In addition, SA-HLWCA is applied to the shape optimization of a blended-wing-body underwater glider (BWBUG), and the lift-drag ratio of the optimized shape is improved by 7.66% compared with that of the initial shape.

A data augmentation based Kriging-assisted reference vector guided evolutionary algorithm for expensive dynamic multi-objective optimization

Expensive dynamic multi-objective optimization problems (EXDMOPs) involve multiple objective functions changing over time steps. In this kind of problems, only a small number of function evaluations can be allowed in each time step. The challenge of EXDMOP is how to quickly and accurately track the changing optimal solutions with only a small number of function evaluations. Although EXDMOP can be solved as a series of independent expensive multi-objective optimization problems or as an ordinary dynamic multi-objective optimization problems, these two methods either ignore the connection between different time steps or degrade the performance significantly due to the requirement for a large number of function evaluations. In this paper, we incorporate two proposed strategies named training data augmentation and sampling pool expansion to the Kriging-assisted reference vector guided evolutionary algorithm to construct a novel algorithm for solving EXDMOPs. The training data augmentation strategy tries to look for the previous similar time step, and adds the solutions evaluated by the objective functions at that time step to current time step. The accuracies of the Kriging models are improved in this way. The sampling pool expansion strategy sends the predicted optimal solutions to the competition of the function evaluations assignment at each time step, where the predicted optimal solutions are generated by the improved population prediction strategy. We set the ablation experiment for the proposed algorithm and compare it with four dynamic multi-objective evolutionary algorithms on various of problems. The experiment results validate the effectiveness of the proposed two strategies. It also indicates that the proposed algorithm is promising for solving EXDMOPs.

An approach for computationally expensive multi-objective optimization problems with independently evaluable objectives

Multi-objective optimization problems involve simultaneous optimization of two or more objectives in conflict. For example, in automotive design, one might be interested in simultaneously minimizing the aerodynamic drag and maximizing the torsional rigidity/collision strength of the vehicle. For a number of problems encountered in engineering design, the objectives can be independently evaluated and such evaluations are often computationally expensive. While surrogate-assisted optimization (SAO) methods are typically used to deal with such problems, they evaluate all objectives for the chosen infill solution(s). If however the objectives can be independently evaluated, there is an opportunity to improve the computational efficiency by evaluating the selected objective(s) only. In this study, we introduce a SAO approach capable of selectively evaluating the objective(s) of the infill solution(s). The approach exploits principles of non-dominance and sparse subset selection to facilitate decomposition and identifies the infill solutions through maximization of probabilistic dominance measure. Thereafter, for each of these infill solutions, one or more objectives are evaluated, taking into account the evaluation status of its closest neighbor and the probability of improvement along each objective. The performance of the approach is benchmarked against state-of-the-art methods on a range of mathematical problems to highlight the efficacy of the approach. Thereafter, we present the performance on two engineering design problems namely a vehicle crashworthiness design problem and an airfoil design problem. We hope that this study would motivate further algorithmic developments to cater to such classes of problems.

Adaptive surrogate-assisted multi-objective evolutionary algorithm using an efficient infill technique

Surrogate-assisted multi-objective evolutionary algorithms have become increasingly popular for solving computationally expensive problems, profiting from surrogate modeling and infill approaches to reduce the time cost of optimization. Most existing algorithms have specified the type of surrogate model before a run and keep the type static during the optimization process. However, a sole surrogate model may not consistently perform well for all problems without any prior knowledge. In this context, this paper proposes an adaptive technique for surrogate models with multiple radial basis functions (RBFs), as the technique can dynamically establish the most promising RBF for each objective, thereby enhancing the reliability of surrogate prediction. Moreover, multi-objective evolutionary algorithms (MOEAs) that are employed as optimizers for infilling criteria can highly affect the search behavior of a surrogate-assisted evolutionary algorithm. The proposed infill technique develops a crowding distance-based prescreening operator to embed various MOEAs. Two techniques collaboratively promote the convergence, coverage, and diversity of the predicted Pareto front. Representative benchmark problems and a structural optimization problem are given to show the effectiveness of the algorithm that employs these techniques. Empirical experiments demonstrate that the proposed algorithm significantly outperforms other state-of-the-art algorithms in most cases.

A fast sampling based evolutionary algorithm for million-dimensional multiobjective optimization

With their complexity and vast search space, large-scale multiobjective optimization problems (LSMOPs) challenge existing multiobjective evolutionary algorithms (MOEAs). Recently, several large-scale multiobjective evolutionary algorithms have been developed to tackle LSMOPs. Unlike conventional MOEAs that concentrate on selection operations in the objective space, large-scale MOEAs emphasize operations in the decision space, such as offspring generation, to tackle the large number of decision variables. Nevertheless, most present large-scale MOEAs experience difficulty effectively and efficiently solving LSMOPs with tens of thousands or more decision variables or exhibit poor versatility in solving different LSMOPs. We propose a fast large-scale MOEA framework with reference-guided offspring generation, named FLEA, aiming at these issues. Generally, FLEA constructs several reference vectors in the decision space to steer the sampling of promising solutions during offspring generation. A parameter is used to allocate computation resources between the convergence and diversity of the offspring population adaptively. Without computationally expensive problem reformulation or decision variable analysis techniques, the proposed method can significantly accelerate the search speed of conventional MOEAs in solving LSMOPs. FLEA is examined on various LSMOPs with up to 1.6 million decision variables, demonstrating its superior effectiveness, efficiency, and versatility in large-scale multiobjective optimization.

Multiple Classifiers-Assisted Evolutionary Algorithm Based on Decomposition for High-Dimensional Multiobjective Problems

Surrogate-assisted multiobjective evolutionary algorithms (MOEAs) have advanced the field of computationally expensive optimization, but their progress is often restricted to low-dimensional problems. This manuscript presents a multiple classifiers-assisted evolutionary algorithm based on decomposition, which is adapted for high-dimensional expensive problems in terms of the following two insights. Compared to approximation-based surrogates, the accuracy of classification-based surrogates is robust for few high-dimensional training samples. Furthermore, multiple local classifiers can hedge the risk of overfitting issues. Accordingly, the proposed algorithm builds multiple classifiers with support vector machines (SVMs) on a decomposition-based multiobjective algorithm, wherein each local classifier is trained for a corresponding scalarization function. Experimental results confirm that the proposed algorithm is competitive to the state-of-the-art algorithms and computationally efficient as well.

Variable surrogate model-based particle swarm optimization for high-dimensional expensive problems

Many industrial applications require time-consuming and resource-intensive evaluations of suitable solutions within very limited time frames. Therefore, many surrogate-assisted evaluation algorithms (SAEAs) have been widely used to optimize expensive problems. However, due to the curse of dimensionality and its implications, scaling SAEAs to high-dimensional expensive problems is still challenging. This paper proposes a variable surrogate model-based particle swarm optimization (called VSMPSO) to meet this challenge and extends it to solve 200-dimensional problems. Specifically, a single surrogate model constructed by simple random sampling is taken to explore different promising areas in different iterations. Moreover, a variable model management strategy is used to better utilize the current global model and accelerate the convergence rate of the optimizer. In addition, the strategy can be applied to any SAEA irrespective of the surrogate model used. To control the trade-off between optimization results and optimization time consumption of SAEAs, we consider fitness value and running time as a bi-objective problem. Applying the proposed approach to a benchmark test suite of dimensions ranging from 30 to 200 and comparisons with four state-of-the-art algorithms show that the proposed VSMPSO achieves high-quality solutions and computational efficiency for high-dimensional problems.

Multiple Surrogate-Model-Based Optimization Method Using the Multimodal Expected Improvement Criterion for Expensive Problems

In this article, a multiple surrogate-model-based optimization method using the multimodal expected improvement criterion (MSMEIC) is proposed. In MSMEIC, an important region is first identified and used alternately with the whole space. Then, in each iteration, three common surrogate models, kriging, radial basis function (RBF), and quadratic response surface (QRS), are constructed, and a multipoint expected improvement (EI) criterion that selects the highest peak and other peaks of EI is proposed to obtain several potential candidates. Furthermore, the optimal predictions of the three surrogate models are regarded as potential candidates. After deleting redundant candidates, the remaining points are saved as the new sampling points. Finally, several well-known benchmark functions and an engineering application are employed to assess the performance of MSMEIC. The testing results demonstrate that, compared with four recent counterparts, the proposed method can obtain more precise solutions more efficiently and with strong robustness.