Unconstrained Global Optimization Research Articles

A piecewise smooth version of the Griewank function

The Griewank test function for global unconstrained optimization has multiple local minima clustered around the global minimum at the origin. A new version of this test function is proposed that has a similar structure, but whose behavior at the local minima and maxima is non-smooth. This piecewise smooth version of the Griewank function represents an abs-factorable test case of objective functions for global non-smooth optimization as, for example, observed in the training of neural networks.

A New Class of Filled Functions with Two Parameters for Solving Unconstrained Global Optimization Problems

A New Class of Filled Functions with Two Parameters for Solving Unconstrained Global Optimization Problems

Two Algorithms for Solving Unconstrained Global Optimization by Auxiliary Function Method

Two Algorithms for Solving Unconstrained Global Optimization by Auxiliary Function Method

A new one-parameter filled function method for unconstrained global optimization problems

A new one-parameter filled function method for unconstrained global optimization problems

A new filled function method based on global search for solving unconstrained optimization problems

<abstract><p>The filled function method is a deterministic algorithm for finding a global minimizer of global optimization problems, and its effectiveness is closely related to the form of the constructed filled function. Currently, the filled functions mainly have three drawbacks in form, namely, parameter adjustment and control (if any), inclusion of exponential or logarithmic functions, and properties that are discontinuous and non-differentiable. In order to overcome these limitations, this paper proposed a parameter-free filled function that does not include exponential or logarithmic functions and is continuous and differentiable. Based on the new filled function, a filled function method for solving unconstrained global optimization problems was designed. The algorithm selected points in the feasible domain that were far from the global minimum point as initial points, and improved the setting of the step size in the stage of minimizing the filled function to enhance the algorithm's global optimization capability. In addition, tests were conducted on 14 benchmark functions and compared with existing filled function algorithms. The numerical experimental results showed that the new algorithm proposed in this paper was feasible and effective.</p></abstract>

A New Integral Function Algorithm for Global Optimization and Its Application to the Data Clustering Problem

The filled function method is an approach to finding global minimum points of multidimensional unconstrained global optimization problems. The conventional parametric filled functions have computational weaknesses when they are employed in some benchmark optimization functions. This paper proposes a new integral function algorithm based on the auxiliary function approach. The proposed method can successfully be used to find the global minimum point of a function of several variables. Some testing global optimization problems have been used to show the ability of this recommended method. The integral function algorithm is then implemented to solve the center-based data clustering problem. The results show that the proposed algorithm can solve the problem successfully.

A Non-Parameter Filled Function Method for Unconstrained Global Optimization Problems

In the paper, we give a new non-parameter filled function method for finding global minimizer of global optimization programming problems, the filled function consists of a inverse cosine function and a logarithm function, and without parameter. Its theoretical residences are proved. A new filled function algorithm is given based on the proposed new parameterless filled function, The results of numerical with ten experiments verify the efficient and reliability for the algorithm.

An Eikonal Equation with Vanishing Lagrangian Arising in Global Optimization

We show a connection between global unconstrained optimization of a continuous function f and weak KAM theory for an eikonal-type equation arising also in ergodic control. A solution v of the critical Hamilton–Jacobi equation is built by a small discount approximation as well as the long time limit of an associated evolutive equation. Then v is represented as the value function of a control problem with target, whose optimal trajectories are driven by a differential inclusion describing the gradient descent of v. Such trajectories are proved to converge to the set of minima of f, using tools in control theory and occupational measures. We prove also that in some cases the set of minima is reached in finite time.

A physics-based machine-learning approach for modeling the temperature-dependent yield strengths of medium- or high-entropy alloys

Machine learning is becoming a powerful tool to accurately predict temperature-dependent yield strengths (YS) of structural materials, particularly for multi-principal-element systems. However, successful machine-learning predictions depend on the use of reasonable machine-learning models. Here, we present a comprehensive and up-to-date overview of a bilinear log model for predicting temperature-dependent YS of medium-entropy or high-entropy alloys (MEAs or HEAs). In this model, a break temperature, Tbreak, is introduced, which can guide the design of MEAs or HEAs with attractive high-temperature properties. Unlike assuming black-box structures, our model is based on the underlying physics, incorporated in the form of a priori information. A technique for the unconstrained global optimization is employed to enable the concurrent optimization of model parameters over low- and high-temperature regimes, showing that the break temperature is consistent across the YS and ultimate strength for a variety of HEA compositions. A high-level comparison between YS of MEAs/HEAs and those of Nickel-based superalloys reveals superior strength properties of selected refractory HEAs. For reliable operations, the temperature of a structural component, such as a turbine blade, made from refractory alloys, may need to stay below Tbreak. Once above Tbreak, phase transformations may start taking place, and the alloy may begin losing structural integrity.

A New Parameterless Filled Function Method for Global Optimization

The filled function method is an effective way to solve global optimization problems. However, its effectiveness is greatly affected by the selection of parameters, and the non-continuous or non-differentiable properties of the constructed filled function. To overcome the above-mentioned drawbacks, in this paper, a new parameterless filled function is proposed that is continuous and differentiable. Theoretical proofs have been made to show the properties of the proposed filled function. Based on the new filled function, a filled function algorithm is proposed to solve unconstrained global optimization problems. Experiments are carried out on widely used test problems and an application of supply chain problems with equality and inequality constraints. The numerical results show that the proposed filled function is effective.

A Class of Parameter-Free Filled Functions for Unconstrained Global Optimization

The deficiencies of existing filled functions have been extensively reviewed in the literature. Their use of an exponent function, the existence of adjustable parameters, their discontinuities, and their difficulty in determining an initial minimization point are significant issues that must be addressed. Although a new generation of filled functions, i.e., parameter-free filled functions, has been proposed, these filled functions still include an exponential term or some discontinuous points in their feasible domains. In this paper, these two adverse effects were diminished by providing a more general form, a new parameter free, and nonexponential filled function, which is continuously differentiable. The results of the analysis demonstrated that, under a specific condition, the properties of the filled functions presented in this paper meet the definition of a filled function. A comparison of numerical results confirmed that the proposed filled function performs better compared to some other parametric and nonparametric filled functions.

Modified Particle Swarm Optimization with Chaotic Initialization Scheme for Unconstrained Optimization Problems

Different variants of particle swarm optimization (PSO) algorithms were introduced in recent years with various improvements to tackle different types of optimization problems more robustly. However, the conventional initialization scheme tends to generate an initial population with relatively inferior solution due to the random guess mechanism. In this paper, a PSO variant known as modified PSO with chaotic initialization scheme is introduced to solve unconstrained global optimization problems more effectively, by generating a more promising initial population. Experimental studies are conducted to assess and compare the optimization performance of the proposed algorithm with four existing well-establised PSO variants using seven test functions. The proposed algorithm is observed to outperform its competitors in solving the selected test problems.

A new filled function for global minimization and system of nonlinear equations

Filled function methods have been considered as an effective approach for solving global optimization problems. However, most filled functions have the drawbacks of discontinuity, non-differentiability, and they could be sensitive to parameters and have exponential or logarithmic terms which may reduce their efficiency. In this paper, we propose a continuously differentiable filled function without parameters and exponential/logarithmic terms to overcome these problems. The continuous differentiability of the presented filled function makes its minimization an easy process and allows using efficient indirect local search methods. The proposed filled function has no parameters to adjust. Adjustment of the parameters is not an easy task, since the parameters may take different values for different problems. Moreover, the new filled function is numerically stable, since there are no exponential or logarithmic terms. Theoretical features of the considered filled function are investigated and a new algorithm for unconstrained global optimization problems is designed. The numerical results show that this method can successfully be used to solve global optimization problems, with a large number of variables. Furthermore, we extend the proposed filled function method to solve systems of nonlinear equations. Finally, some test problems for systems of nonlinear equations are reported, with satisfactory numerical results.

A dimensionality reduction technique for unconstrained global optimization of functions with low effective dimensionality

Abstract We investigate the unconstrained global optimization of functions with low effective dimensionality, which are constant along certain (unknown) linear subspaces. Extending the technique of random subspace embeddings in Wang et al. (2016, J. Artificial Intelligence Res., 55, 361–387), we study a generic Random Embeddings for Global Optimization (REGO) framework that is compatible with any global minimization algorithm. Instead of the original, potentially large-scale optimization problem, within REGO, a Gaussian random, low-dimensional problem with bound constraints is formulated and solved in a reduced space. We provide novel probabilistic bounds for the success of REGO in solving the original, low effective-dimensionality problem, which show its independence of the (potentially large) ambient dimension and its precise dependence on the dimensions of the effective and embedding subspaces. These results significantly improve existing theoretical analyses by providing the exact distribution of a reduced minimizer and its Euclidean norm and by the general assumptions required on the problem. We validate our theoretical findings by extensive numerical testing of REGO with three types of global optimization solvers, illustrating the improved scalability of REGO compared with the full-dimensional application of the respective solvers.

Dense conjugate initialization for deterministic PSO in applications: ORTHOinit+

This paper describes a class of novel initializations in Deterministic Particle Swarm Optimization (DPSO) for approximately solving costly unconstrained global optimization problems. The initializations are based on choosing specific dense initial positions and velocities for particles. These choices tend to induce in some sense orthogonality of particles’ trajectories, in the early iterations, in order to better explore the search space. Our proposal is inspired by both a theoretical analysis on a reformulation of PSO iteration, and by possible limits of the proposals reported in Campana et al. (2010); Campana et al. (2013). We explicitly show that, in comparison with other initializations from the literature, our initializations tend to scatter PSO particles, at least in the first iterations. The latter goal is obtained by imposing that the initial choice of particles’ position/velocity satisfies specific conjugacy conditions, with respect to a matrix depending on the parameters of PSO. In particular, by an appropriate condition on particles’ velocities, our initializations also resemble and partially extend a general paradigm in the literature of exact methods for derivative-free optimization. Moreover, we propose dense initializations for DPSO, so that the final approximate global solution obtained is possibly not too sparse, which might cause troubles in some applications. Numerical results, on both Portfolio Selection and Computational Fluid Dynamics problems, validate our theory and prove the effectiveness of our proposal, which applies also in case different neighborhood topologies are adopted in DPSO.

Non parameter-filled function for global optimization

It has been generally recognized that most of the existing parametric filled function methods used for finding the global mininizer of unconstrained global optimization problems have computational weaknesses. In this paper, a new non parameter-filled function is proposed. This type of auxiliary function behaves as a bridge that can deliver one minimizer to another local minimizer, if one exists. To prove that the proposed filled function satisfies the filling properties required by the filled function definition, the analytical properties are explored. Through several test examples, the capability of this proposed method is demonstrated and the computational weaknesses of the parametric filled function are proved to be surmounted by this new non parameter-filled function.

A new parameter free filled function for solving unconstrained global optimization problems

The filled function method is an efficient approach for finding a global minimizer of global optimization problems. This paper introduces a new filled function which overcomes the drawbacks of sensitivity to parameters, containing exponential or logarithmic terms, discontinuity and non-differentiability for some previous filled functions. It proposes a filled function without any parameters to be adjusted. This filled function has no exponential or logarithmic terms which make the filled function numerically unstable. Also it is continuously differentiable, so gradient information is available in order to use effective local minimization algorithms. Theories of the proposed filled function are investigated and an algorithm for unconstrained global optimization is presented. Numerical results on many test problems with large number of variables are reported. A comparison with some existing algorithms shows that this algorithm is efficient and reliable.

Filled Fonksiyonunda Yeni Bir Stokastik Arama Yöntemi

In this study, a new stochastic search approach is presented as a faster and more efficient alternative to classic filled function search strategy. An unconstrained global optimization method based on clustering and parabolic approximation (GOBC-PA) has been used as a stochastic method for accelerating the L type filled function as a deterministic method. Searching the basin regions of the filled function is performed by GOBC-PA. The methods used in this study are preferred due to their popularity, speed and robustness. The objective function of the stochastic method is the epsilon value of the gradient that gives the location of basin region. Therefore, the whole purpose of the stochastic method is not to find the global optimum but to find the basin region. The role of finding the global minimum has been left to the deterministic method. The developed method has been tested against classical filled function using 11 benchmark functions and process repeated 10 times. When the obtained results are examined, it is seen that the stochastic search approach has superiority over the mean error, standard deviation and elapsed time values according to the classical approach. These results show that the combination of deterministic and stochastic methods can be more successful in finding the global minimum against the classic deterministic method.

A pattern representation of stock time series based on DTW

Time series analysis based on pattern discovery has received a lot of interests in the fields of economic physics and machine learning due to its simplicity and ability to reveal complex nonlinear behavior in stock market. Dynamic Time Warping (DTW) is a useful tool to extract morphological characteristics of time series for its capacity to cope with time shifts and warpings. In this paper, we propose a new time series representation method for stock time series based on dynamic time warping (DTW) called PR-DTW. A combinatorial optimization model with strict constraints is built to get the pattern representation of stock time series. To simplify the calculation, we construct another unconstrained global optimization problem whose optimal solution includes the optimal solution of the original combinatorial optimization problem based on a theorem proved in this paper. Particle Swarm Optimization algorithm is used to solve the global optimization problem, then the results can be converted into the optimal solution of the combinatorial optimization problem through a few simple formulas given in the theorem. The results of three classifiers (1NN, Decision Tree, Multi-layer Perceptron) implemented on 15 sectors in Chinese A-share market unanimously demonstrate that PR-DTW has the capability of extracting time series short-term patterns which is widely regarded as difficulty. And we conclude that PR-DTW has the capability of prevention of End Effect, anti-noise and segmentation. Moreover, by extracting the top ten patterns predicting stock’s rise and fall in short term (10 days) according to the ranking of stock’s rising probability in the next three days, we find out the short-term patterns obtained by PR-DTW have prospective directive to the stock trend analysis in short term.

Kriging-based unconstrained global optimization through multi-point sampling

ABSTRACTThe generalized efficient global optimization (GEGO) method is able to solve expensive black-box problems. However, selecting one sampling point per cycle may result in large time consumption and loss of convergence accuracy. To this end, a kriging-based multi-point unconstrained global optimization (KMUGO) method is proposed. It extends the GEGO method with the improved constant liar (CL) strategy. For each cycle, the kriging model is first constructed or updated by the existing sampled data. Then, the enhanced alternative CL strategy is used to find multiple points, which will be further screened to identify the final expensive-evaluation points. Test results of numerical problems and an engineering simulation case show that KMUGO can deliver better convergence than GEGO, multi-point sampling method-based kriging (MPSK), hybrid and adaptive meta-model-based global optimization (HAM) and the kriging-based global optimization method using multi-point infill search criterion (KGOMISC).