Mixed-variable Optimization Problems Research Articles

Metaheuristics for variable-size mixed optimization problems: A unified taxonomy and survey

Many real world optimization problems are formulated as mixed-variable optimization problems (MVOPs) which involve both continuous and discrete variables. MVOPs including dimensional variables are characterized by a variable-size search space. Depending on the values of dimensional variables, the number and type of the variables of the problem can vary dynamically. MVOPs and variable-size MVOPs (VMVOPs) are difficult to solve and raise a number of scientific challenges in the design of metaheuristics. Standard metaheuristics have been first designed to address continuous or discrete optimization problems, and are not able to tackle VMVOPs in an efficient way. The development of metaheuristics for solving such problems has attracted the attention of many researchers and is increasingly popular. However, to our knowledge there is no well established taxonomy or comprehensive survey for handling this important family of optimization problems. This paper presents an unified taxonomy for metaheuristic solutions for solving VMVOPs in an attempt to provide a common terminology and classification mechanisms. It provides a general mathematical formulation and concepts of VMVOPs, and identifies the various solving methodologies than can be applied in metaheuristics. The advantages, the weaknesses and the limitations of the presented methodologies are discussed. The proposed taxonomy also allows to identify some open research issues which needs further in-depth investigations.

A coevolutionary estimation of distribution algorithm based on dynamic differential grouping for mixed-variable optimization problems

The mixed variable optimization problems (MVOPs), which involves both continuous and discrete decision variables, are difficult to be solved due to the complex search space. Recently, many EA-based algorithms have been designed to address MVOPs. However, due to the mixed variables with different evolutionary operators and complex search space, it is difficult to handle the mixed variables effectively and the search efficiency cannot be guaranteed. How to solve MVOPs efficiently has been a challengeable issue. In this paper, we propose a mixed-variable optimization algorithm called coevolutionary estimation of distribution algorithm (CoEDAmv). First, a dynamic differential grouping (DDG) method is employed to improve the search efficiency of CoEDAmv, in which both the interaction of variables and search performance on the current search region are considered simultaneously. Second, two probabilistic models, i.e. fitness rank based continuous histogram (FRCH) and fitness rank based discrete histogram (FRDH), are proposed to handle continuous and discrete variables respectively, which can benefit from elite individuals obtained during the fitness ranking strategy and enhance the convergence performance with the elite neighborhood-based updating probability strategy. Compared with eight state-of-the-art algorithms, the experimental results on 28 artificial MVOPs show that CoEDAmv is effective and efficient.

Optical design algorithm utilizing continuous-discrete variables grounded on stochastic processes

This work proposes an optimization algorithm in optical design based on the concepts of ergodic and stochastic processes in statistical mechanics. In mixed-variable optimization problems, pseudo-random number and discrete-to-continuous variable conversion dramatically increase the speed at which the system solves for the optimal solution. Pseudo-random numbers are mainly applied in two important steps in the optimization algorithm: determining the combination of glasses involved and the order in which the successive glass parameters are replaced by real glasses. After two series of stochastic processes, the merit function value decreases rapidly along the steepest descent path, and thus the optical system approaches the optimal solution within a very short duration of time. By using the method proposed in this paper, a plan apochromatic objective with a long working distance was optimized, and finally, a high-quality optical system was obtained.

Multi-strategy multi-objective differential evolutionary algorithm with reinforcement learning

Multiobjective evolutionary algorithms (MOEAs) have gained much attention due to their high effectiveness and efficiency in solving multiobjective optimization problems (MOPs). However, when solving MOPs, it is important but difficult to maintain a good balance of exploration and exploitation. In addition, some reference point based MOEAs with fixed reference points perform poorly on MOPs with irregular frontiers. Therefore, this paper proposes a new multistrategy multiobjective differential evolutionary (DE) algorithm, named RLMMDE. In RLMMDE, a multistrategy and multicrossover DE optimizer is utilized to alleviate the exploration and exploitation dilemma. An adaptive reference point activation mechanism based on RL is proposed to activate the adaptive adjustment of reference points. Moreover, a reference point adaptation method is proposed to improve the performance of RLMMDE on irregular frontier problems. Experimental results of RLMMDE tested on some benchmark test suites (i.e., ZDT, DTLZ, UF, WFG, and LSMOP) and two practical mixed-variable optimization problems show that the algorithm outperforms some advanced MOEAs.

A General Mathematical Framework for Constrained Mixed-variable Blackbox Optimization Problems with Meta and Categorical Variables

A mathematical framework for modelling constrained mixed-variable optimization problems is presented in a blackbox optimization context. The framework introduces a new notation and allows solution strategies. The notation framework allows meta and categorical variables to be explicitly and efficiently modelled, which facilitates the solution of such problems. The new term meta variables is used to describe variables that influence which variables are included or excluded: meta variables may affect the number of variables and constraints. The flexibility of the solution strategies supports the main blackbox mixed-variable optimization approaches: direct search methods and surrogate-based methods (Bayesian optimization). The notation system and solution strategies are illustrated through an example of a hyperparameter optimization problem from the machine learning community.

Surrogate-assisted hybrid evolutionary algorithm with local estimation of distribution for expensive mixed-variable optimization problems

Some real-world design optimization problems can be formulated as expensive mixed-variable optimization problems (EMVOPs), which involve both continuous and discrete decision variables and expensive function evaluations. The main challenges for solving EMVOPs are the handling of mixed variables, limited number of function evaluations and multiple disconnected regions in the search space. In this work, we propose a novel algorithm with global and local search strategies for improving the search ability on disconnected regions, and it only consumes hundreds of function evaluations. The global module employs hybrid evolutionary operators and a Gower distance based surrogate model for handling mixed variables. The local estimation of distributions in different local regions performs in a competitive switching way to combine their advantages, and local surrogate models trained with selected samples improve the accuracy of approximated evaluations for the locally generated solutions. In the late stage, a local continuous search module is executed for refining the continuous decision variables. Verification results on the artificial benchmarks demonstrate that our proposed algorithm is competitive and works effectively. To verify the practicability of the algorithm, it is applied on a convolutional neural network hyperparameter optimization problems and obtains satisfactory results.

Surrogate-Assisted Hybrid Evolutionary Algorithm with Local Estimation of Distribution for Expensive Mixed-Variable Optimization Problems

Surrogate-Assisted Hybrid Evolutionary Algorithm with Local Estimation of Distribution for Expensive Mixed-Variable Optimization Problems

Multi-strategy co-evolutionary differential evolution for mixed-variable optimization

In real world, engineering design problems have been regarded as common and complex problems which always include non-linear standard functions, many complex constraints and decision variables containing both continuous and discrete variables. Problems with these features also can be named as mixed-variable optimization problems (MVOPs). However, because the decision variables of MVOPs present different spatial distribution features, how to design an effective algorithm to solve MVOPs is still a challenge. In this paper, a new variation of differential evolution (DE), named multi-strategy co-evolutionary differential evolution (MCDEmv) is proposed to solve MVOPs. A mixed-variable co-evolutionary scheme simultaneously considering continuous and discrete variables in MVOPs is utilized. Based on this scheme, a multi-strategy co-evolutionary approach considering a dynamic adaptive selection mechanism and the combination of different characteristic mutation strategies and crossover operators is proposed for adapting to the all-inclusive situations in MVOPs. Furthermore, a statistics-based local search (SBA) specially designed for the optimization of discrete variables in MVOPs is proposed for enhancing the efficiency and flexibility of the MCDEmv. 28 artificial benchmark functions are adopted to verify the effectiveness and efficiency of the proposed algorithm. The experimental results show that our proposed algorithm is more competitive and efficient than other compared similar algorithms, i.e., AEDA, ACOMV, DEMV and PSOmv. Comparing with other similar algorithms in coping with two mixed-variable engineering design problems, MCDEmv also shows higher efficiency and performance.

A particle swarm optimization algorithm for mixed-variable optimization problems

Abstract Many optimization problems in reality involve both continuous and discrete decision variables, and these problems are called mixed-variable optimization problems (MVOPs). The mixed decision variables of MVOPs increase the complexity of search space and make them difficult to be solved. The Particle Swarm Optimization (PSO) algorithm is easy to implement due to its simple framework and high speed of convergence, and has been successfully applied to many difficult optimization problems. Many existing PSO variants have been proposed to solve continuous or discrete optimization problems, which make it feasible and promising for solving MVOPs. In this paper, a new PSO algorithm for solving MVOPs is proposed, namely P S O m v , which can deal with both continuous and discrete decision variables simultaneously. To efficiently handle mixed variables, the P S O m v employs a mixed-variable encoding scheme. Based on the mixed-variable encoding scheme, two reproduction methods respectively for continuous variables and discrete variables are proposed. Furthermore, an adaptive parameter tuning strategy is employed and a constraints handling method is utilized to improve the overall efficiency of the P S O m v .The experimental results on 28 artificial MVOPs and two practical MVOPs demonstrate that the proposed P S O m v is a competitive algorithm for MVOPs.

Bayesian optimization of variable-size design space problems

Within the framework of complex system design, it is often necessary to solve mixed variable optimization problems, in which the objective and constraint functions can depend simultaneously on continuous and discrete variables. Additionally, complex system design problems occasionally present a variable-size design space. This results in an optimization problem for which the search space varies dynamically (with respect to both number and type of variables) along the optimization process as a function of the values of specific discrete decision variables. Similarly, the number and type of constraints can vary as well. In this paper, two alternative Bayesian optimization-based approaches are proposed in order to solve this type of optimization problems. The first one consists of a budget allocation strategy allowing to focus the computational budget on the most promising design sub-spaces. The second approach, instead, is based on the definition of a kernel function allowing to compute the covariance between samples characterized by partially different sets of variables. The results obtained on analytical and engineering related test-cases show a faster and more consistent convergence of both proposed methods with respect to the standard approaches.

Obtaining More Accurate Thermal Boundary Conditions of Machine Tool Spindle Using Response Surface Model Hybrid Artificial Bee Colony Algorithm

With the application of finite element method on structure design and engineering analysis more and more widely, this paper presents a response surface model hybrid artificial bee colony method to optimize the thermal boundary conditions in finite element thermal analysis of a machine tool spindle to improve its finite element simulation precision. Initially, the thermal experiment and finite element thermal analysis of the machine tool spindle with initials that were calculated by empirical formulas were conducted, respectively. Additionally, focusing on thermal boundary conditions, a response surface model is designed to establish the explicit expression between thermal boundary conditions and the simulation errors; then, an artificial bee colony algorithm is used to solve the mixed-variable optimization problems of a response surface model. Finally, the optimized thermal boundary conditions are brought into the finite element method of a machine tool system, and the simulation accuracy has been greatly improved.

A Modified Jaya Algorithm for Mixed-Variable Optimization Problems

Abstract Mixed-variable optimization problems consist of the continuous, integer, and discrete variables generally used in various engineering optimization problems. These variables increase the computational cost and complexity of optimization problems due to the handling of variables. Moreover, there are few optimization algorithms that give a globally optimal solution for non-differential and non-convex objective functions. Initially, the Jaya algorithm has been developed for continuous variable optimization problems. In this paper, the Jaya algorithm is further extended for solving mixed-variable optimization problems. In the proposed algorithm, continuous variables remain in the continuous domain while continuous domains of discrete and integer variables are converted into discrete and integer domains applying bound constraint of the middle point of corresponding two consecutive values of discrete and integer variables. The effectiveness of the proposed algorithm is evaluated through examples of mixed-variable optimization problems taken from previous research works, and optimum solutions are validated with other mixed-variable optimization algorithms. The proposed algorithm is also applied to two-plane balancing of the unbalanced rigid threshing rotor, using the number of balance masses on plane 1 and plane 2. It is found that the proposed algorithm is computationally more efficient and easier to use than other mixed optimization techniques.

A hybrid differential evolution algorithm for mixed-variable optimization problems

Mixed-variable optimization problems (MVOPs) that involve continuous and discrete decision variables widely exist in industrial and scientific domains. However, how to solve MVOPs efficiently remains an open issue because the fact that continuous and discrete variables present different spatial distribution features posts a great challenge to algorithmic design. In this paper, a hybrid differential evolution (DE) framework is proposed for MVOPs. The proposed framework, namely DEMV, hybridizes the original DE and the set-based DE for evolving continuous and discrete variables, respectively. The two DEs are selected for hybridization because algorithmic analysis and experimental studies show that they share the same search mechanism. The compatibility and consistency of the two DEs is the key for enabling DEMV to coevolve different types of decision variables efficiently. Experiments are conducted on a set of artificial MVOPs converted from continuous benchmark functions and real-world engineering problems with mixed variables. Experimental results and comparisons with other representative algorithms show that DEMV is effective and efficient.

A Hybrid-coded Human Learning Optimization for mixed-variable optimization problems

Human Learning Optimization (HLO) is an emerging meta-heuristic with promising potential, which is inspired by human learning mechanisms. Although binary algorithms like HLO can be directly applied to mixed-variable problems that contains both continuous values and discrete or Boolean values, the search efficiency and the performance of those algorithms may be significantly spoiled due to “the curse of dimensionality” caused by the binary coding strategy especially when the continuous parameters of problems require high accuracy. Therefore, this paper extends HLO and proposes a novel hybrid-coded HLO (HcHLO) framework to tackle mix-coded problems more efficiently and effectively, in which real-coded parameters are optimized by a new continuous HLO (CHLO) based on the linear learning mechanism of humans and the other variables are handled by the binary learning operators of HLO. Finally, HcHLO is adopted to solve 14 benchmark problems and its performance is compared with that of recent meta-heuristic algorithms. The experimental results show that the proposed HcHLO achieves the best-known overall performance so far on the test problems, which demonstrates the validity and superiority of HcHLO.

Efficient Ant Colony Optimization (EACO) Algorithm for Deterministic Optimization

In this paper, an efficient ant colony optimization (EACO) algorithm is proposed based on efficient sampling method for solving combinatorial, continuous and mixed-variable optimization problems. In EACO algorithm, Hammersley Sequence Sampling (HSS) is introduced to initialize the solution archive and to generate multidimensional random numbers. The capabilities of the proposed algorithm are illustrated through 9 benchmark problems. The results of the benchmark problems from EACO algorithm and the conventional ACO algorithm are compared. More than 99% of the results from the EACO show efficiency improvement and the computational efficiency improvement range from 3% to 71%. Thus, this new algorithm can be a useful tool for large-scale and wide range of optimization problems. Moreover, the performance of the EACO is also tested using the five variants of ant algorithms for combinatorial problems.

Ant Colony Optimization for Mixed-Variable Optimization Problems

In this paper, we introduce ACO MV : an ant colony optimization (ACO) algorithm that extends the ACO R algorithm for continuous optimization to tackle mixed-variable optimization problems. In ACO MV , the decision variables of an optimization problem can be explicitly declared as continuous, ordinal, or categorical, which allows the algorithm to treat them adequately. ACO MV includes three solution generation mechanisms: a continuous optimization mechanism (ACO R ), a continuous relaxation mechanism (ACO MV -o) for ordinal variables, and a categorical optimization mechanism (ACO MV -c) for categorical variables. Together, these mechanisms allow ACO MV to tackle mixed-variable optimization problems. We also define a novel procedure to generate artificial, mixed-variable benchmark functions, and we use it to automatically tune ACO MV 's parameters. The tuned ACO MV is tested on various real-world continuous and mixed-variable engineering optimization problems. Comparisons with results from the literature demonstrate the effectiveness and robustness of ACO MV on mixed-variable optimization problems.

Multiple solutions of mixed variable optimization by multistart Hooke and Jeeves filter method

In this study, we propose a multistart method based on an extended version of the Hooke and Jeeves (HJ) algorithm for computing multiple solutions of mixed variable optimization problems. The inequality and equality constraints of the problem are handled by a lter set methodology. The basic ideas present in the HJ algorithm, namely the exploratory and pattern moves, are extended to consider two objective functions and to handle continuous and integer variables simultaneously. This proposal is integrated into a multistart method as a local search procedure that is repeatedly invoked to converge to dierent global and non-global optimal solutions starting from randomly generated points. To avoid repeated convergence to previously computed solutions, the concept of region of attraction of an optimizer is implemented. The performance of the new method is tested on benchmark problems. Its

The Structure Optimization of Main Beam for Bridge Crane Based on An Improved PSO

The structure optimization of main beam is a nonlinear constrained optimization problem, which is important for bridge crane to save manufacturing cost on quality assurance. The modified particle swarm optimization (MPSO) with feasibility-based rules (1), which was advanced to solve mixed-variable optimization problems, is proposed to optimize the structure of main beam in order to find the optimal parameters so as to make minimize the deadweight of main beam. The comparison results with the enumeration algorithm illustrated that MPSO can get best optimal solutions in much less calculation numbers.

Comprehensive Learning Particle Swarm Optimizer for Constrained Mixed-Variable Optimization Problems

This paper presents an improved particle swarm optimizer (PSO) for solving multimodal optimization problems with problem-specific constraints and mixed variables. The standard PSO is extended by employing a comprehensive learning strategy, different particle updating approaches, and a feasibility-based rule method. The experiment results show the algorithm located the global optima in all tested problems, and even found a better solution than those previously reported in the literature. In some cases, it outperforms other methods in terms of both solution accuracy and computational cost.

Comprehensive Learning Particle Swarm Optimizer for Constrained Mixed-Variable Optimization Problems

This paper presents an improved particle swarm optimizer (PSO) for solving multimodal optimization problems with problem-specific constraints and mixed variables. The standard PSO is extended by employing a comprehensive learning strategy, different particle updating approaches, and a feasibility-based rule method. The experiment results show the algorithm located the global optima in all tested problems, and even found a better solution than those previously reported in the literature. In some cases, it outperforms other methods in terms of both solution accuracy and computational cost.