Unconstrained Global Optimization Research Articles

A New Integral Function Algorithm for Global Optimization and Its Application to the Data Clustering Problem

The filled function method is an approach to finding global minimum points of multidimensional unconstrained global optimization problems. The conventional parametric filled functions have computational weaknesses when they are employed in some benchmark optimization functions. This paper proposes a new integral function algorithm based on the auxiliary function approach. The proposed method can successfully be used to find the global minimum point of a function of several variables. Some testing global optimization problems have been used to show the ability of this recommended method. The integral function algorithm is then implemented to solve the center-based data clustering problem. The results show that the proposed algorithm can solve the problem successfully.

A Non-Parameter Filled Function Method for Unconstrained Global Optimization Problems

In the paper, we give a new non-parameter filled function method for finding global minimizer of global optimization programming problems, the filled function consists of a inverse cosine function and a logarithm function, and without parameter. Its theoretical residences are proved. A new filled function algorithm is given based on the proposed new parameterless filled function, The results of numerical with ten experiments verify the efficient and reliability for the algorithm.

An Eikonal Equation with Vanishing Lagrangian Arising in Global Optimization

We show a connection between global unconstrained optimization of a continuous function f and weak KAM theory for an eikonal-type equation arising also in ergodic control. A solution v of the critical Hamilton–Jacobi equation is built by a small discount approximation as well as the long time limit of an associated evolutive equation. Then v is represented as the value function of a control problem with target, whose optimal trajectories are driven by a differential inclusion describing the gradient descent of v. Such trajectories are proved to converge to the set of minima of f, using tools in control theory and occupational measures. We prove also that in some cases the set of minima is reached in finite time.

A physics-based machine-learning approach for modeling the temperature-dependent yield strengths of medium- or high-entropy alloys

Machine learning is becoming a powerful tool to accurately predict temperature-dependent yield strengths (YS) of structural materials, particularly for multi-principal-element systems. However, successful machine-learning predictions depend on the use of reasonable machine-learning models. Here, we present a comprehensive and up-to-date overview of a bilinear log model for predicting temperature-dependent YS of medium-entropy or high-entropy alloys (MEAs or HEAs). In this model, a break temperature, Tbreak, is introduced, which can guide the design of MEAs or HEAs with attractive high-temperature properties. Unlike assuming black-box structures, our model is based on the underlying physics, incorporated in the form of a priori information. A technique for the unconstrained global optimization is employed to enable the concurrent optimization of model parameters over low- and high-temperature regimes, showing that the break temperature is consistent across the YS and ultimate strength for a variety of HEA compositions. A high-level comparison between YS of MEAs/HEAs and those of Nickel-based superalloys reveals superior strength properties of selected refractory HEAs. For reliable operations, the temperature of a structural component, such as a turbine blade, made from refractory alloys, may need to stay below Tbreak. Once above Tbreak, phase transformations may start taking place, and the alloy may begin losing structural integrity.

A New Parameterless Filled Function Method for Global Optimization

The filled function method is an effective way to solve global optimization problems. However, its effectiveness is greatly affected by the selection of parameters, and the non-continuous or non-differentiable properties of the constructed filled function. To overcome the above-mentioned drawbacks, in this paper, a new parameterless filled function is proposed that is continuous and differentiable. Theoretical proofs have been made to show the properties of the proposed filled function. Based on the new filled function, a filled function algorithm is proposed to solve unconstrained global optimization problems. Experiments are carried out on widely used test problems and an application of supply chain problems with equality and inequality constraints. The numerical results show that the proposed filled function is effective.

A Class of Parameter-Free Filled Functions for Unconstrained Global Optimization

The deficiencies of existing filled functions have been extensively reviewed in the literature. Their use of an exponent function, the existence of adjustable parameters, their discontinuities, and their difficulty in determining an initial minimization point are significant issues that must be addressed. Although a new generation of filled functions, i.e., parameter-free filled functions, has been proposed, these filled functions still include an exponential term or some discontinuous points in their feasible domains. In this paper, these two adverse effects were diminished by providing a more general form, a new parameter free, and nonexponential filled function, which is continuously differentiable. The results of the analysis demonstrated that, under a specific condition, the properties of the filled functions presented in this paper meet the definition of a filled function. A comparison of numerical results confirmed that the proposed filled function performs better compared to some other parametric and nonparametric filled functions.

Modified Particle Swarm Optimization with Chaotic Initialization Scheme for Unconstrained Optimization Problems

Different variants of particle swarm optimization (PSO) algorithms were introduced in recent years with various improvements to tackle different types of optimization problems more robustly. However, the conventional initialization scheme tends to generate an initial population with relatively inferior solution due to the random guess mechanism. In this paper, a PSO variant known as modified PSO with chaotic initialization scheme is introduced to solve unconstrained global optimization problems more effectively, by generating a more promising initial population. Experimental studies are conducted to assess and compare the optimization performance of the proposed algorithm with four existing well-establised PSO variants using seven test functions. The proposed algorithm is observed to outperform its competitors in solving the selected test problems.

A new filled function for global minimization and system of nonlinear equations

Filled function methods have been considered as an effective approach for solving global optimization problems. However, most filled functions have the drawbacks of discontinuity, non-differentiability, and they could be sensitive to parameters and have exponential or logarithmic terms which may reduce their efficiency. In this paper, we propose a continuously differentiable filled function without parameters and exponential/logarithmic terms to overcome these problems. The continuous differentiability of the presented filled function makes its minimization an easy process and allows using efficient indirect local search methods. The proposed filled function has no parameters to adjust. Adjustment of the parameters is not an easy task, since the parameters may take different values for different problems. Moreover, the new filled function is numerically stable, since there are no exponential or logarithmic terms. Theoretical features of the considered filled function are investigated and a new algorithm for unconstrained global optimization problems is designed. The numerical results show that this method can successfully be used to solve global optimization problems, with a large number of variables. Furthermore, we extend the proposed filled function method to solve systems of nonlinear equations. Finally, some test problems for systems of nonlinear equations are reported, with satisfactory numerical results.

A dimensionality reduction technique for unconstrained global optimization of functions with low effective dimensionality

Abstract We investigate the unconstrained global optimization of functions with low effective dimensionality, which are constant along certain (unknown) linear subspaces. Extending the technique of random subspace embeddings in Wang et al. (2016, J. Artificial Intelligence Res., 55, 361–387), we study a generic Random Embeddings for Global Optimization (REGO) framework that is compatible with any global minimization algorithm. Instead of the original, potentially large-scale optimization problem, within REGO, a Gaussian random, low-dimensional problem with bound constraints is formulated and solved in a reduced space. We provide novel probabilistic bounds for the success of REGO in solving the original, low effective-dimensionality problem, which show its independence of the (potentially large) ambient dimension and its precise dependence on the dimensions of the effective and embedding subspaces. These results significantly improve existing theoretical analyses by providing the exact distribution of a reduced minimizer and its Euclidean norm and by the general assumptions required on the problem. We validate our theoretical findings by extensive numerical testing of REGO with three types of global optimization solvers, illustrating the improved scalability of REGO compared with the full-dimensional application of the respective solvers.

Dense conjugate initialization for deterministic PSO in applications: ORTHOinit+

This paper describes a class of novel initializations in Deterministic Particle Swarm Optimization (DPSO) for approximately solving costly unconstrained global optimization problems. The initializations are based on choosing specific dense initial positions and velocities for particles. These choices tend to induce in some sense orthogonality of particles’ trajectories, in the early iterations, in order to better explore the search space. Our proposal is inspired by both a theoretical analysis on a reformulation of PSO iteration, and by possible limits of the proposals reported in Campana et al. (2010); Campana et al. (2013). We explicitly show that, in comparison with other initializations from the literature, our initializations tend to scatter PSO particles, at least in the first iterations. The latter goal is obtained by imposing that the initial choice of particles’ position/velocity satisfies specific conjugacy conditions, with respect to a matrix depending on the parameters of PSO. In particular, by an appropriate condition on particles’ velocities, our initializations also resemble and partially extend a general paradigm in the literature of exact methods for derivative-free optimization. Moreover, we propose dense initializations for DPSO, so that the final approximate global solution obtained is possibly not too sparse, which might cause troubles in some applications. Numerical results, on both Portfolio Selection and Computational Fluid Dynamics problems, validate our theory and prove the effectiveness of our proposal, which applies also in case different neighborhood topologies are adopted in DPSO.

Non parameter-filled function for global optimization

It has been generally recognized that most of the existing parametric filled function methods used for finding the global mininizer of unconstrained global optimization problems have computational weaknesses. In this paper, a new non parameter-filled function is proposed. This type of auxiliary function behaves as a bridge that can deliver one minimizer to another local minimizer, if one exists. To prove that the proposed filled function satisfies the filling properties required by the filled function definition, the analytical properties are explored. Through several test examples, the capability of this proposed method is demonstrated and the computational weaknesses of the parametric filled function are proved to be surmounted by this new non parameter-filled function.

A new parameter free filled function for solving unconstrained global optimization problems

The filled function method is an efficient approach for finding a global minimizer of global optimization problems. This paper introduces a new filled function which overcomes the drawbacks of sensitivity to parameters, containing exponential or logarithmic terms, discontinuity and non-differentiability for some previous filled functions. It proposes a filled function without any parameters to be adjusted. This filled function has no exponential or logarithmic terms which make the filled function numerically unstable. Also it is continuously differentiable, so gradient information is available in order to use effective local minimization algorithms. Theories of the proposed filled function are investigated and an algorithm for unconstrained global optimization is presented. Numerical results on many test problems with large number of variables are reported. A comparison with some existing algorithms shows that this algorithm is efficient and reliable.

Filled Fonksiyonunda Yeni Bir Stokastik Arama Yöntemi

In this study, a new stochastic search approach is presented as a faster and more efficient alternative to classic filled function search strategy. An unconstrained global optimization method based on clustering and parabolic approximation (GOBC-PA) has been used as a stochastic method for accelerating the L type filled function as a deterministic method. Searching the basin regions of the filled function is performed by GOBC-PA. The methods used in this study are preferred due to their popularity, speed and robustness. The objective function of the stochastic method is the epsilon value of the gradient that gives the location of basin region. Therefore, the whole purpose of the stochastic method is not to find the global optimum but to find the basin region. The role of finding the global minimum has been left to the deterministic method. The developed method has been tested against classical filled function using 11 benchmark functions and process repeated 10 times. When the obtained results are examined, it is seen that the stochastic search approach has superiority over the mean error, standard deviation and elapsed time values according to the classical approach. These results show that the combination of deterministic and stochastic methods can be more successful in finding the global minimum against the classic deterministic method.

A pattern representation of stock time series based on DTW

Time series analysis based on pattern discovery has received a lot of interests in the fields of economic physics and machine learning due to its simplicity and ability to reveal complex nonlinear behavior in stock market. Dynamic Time Warping (DTW) is a useful tool to extract morphological characteristics of time series for its capacity to cope with time shifts and warpings. In this paper, we propose a new time series representation method for stock time series based on dynamic time warping (DTW) called PR-DTW. A combinatorial optimization model with strict constraints is built to get the pattern representation of stock time series. To simplify the calculation, we construct another unconstrained global optimization problem whose optimal solution includes the optimal solution of the original combinatorial optimization problem based on a theorem proved in this paper. Particle Swarm Optimization algorithm is used to solve the global optimization problem, then the results can be converted into the optimal solution of the combinatorial optimization problem through a few simple formulas given in the theorem. The results of three classifiers (1NN, Decision Tree, Multi-layer Perceptron) implemented on 15 sectors in Chinese A-share market unanimously demonstrate that PR-DTW has the capability of extracting time series short-term patterns which is widely regarded as difficulty. And we conclude that PR-DTW has the capability of prevention of End Effect, anti-noise and segmentation. Moreover, by extracting the top ten patterns predicting stock’s rise and fall in short term (10 days) according to the ranking of stock’s rising probability in the next three days, we find out the short-term patterns obtained by PR-DTW have prospective directive to the stock trend analysis in short term.

Kriging-based unconstrained global optimization through multi-point sampling

ABSTRACTThe generalized efficient global optimization (GEGO) method is able to solve expensive black-box problems. However, selecting one sampling point per cycle may result in large time consumption and loss of convergence accuracy. To this end, a kriging-based multi-point unconstrained global optimization (KMUGO) method is proposed. It extends the GEGO method with the improved constant liar (CL) strategy. For each cycle, the kriging model is first constructed or updated by the existing sampled data. Then, the enhanced alternative CL strategy is used to find multiple points, which will be further screened to identify the final expensive-evaluation points. Test results of numerical problems and an engineering simulation case show that KMUGO can deliver better convergence than GEGO, multi-point sampling method-based kriging (MPSK), hybrid and adaptive meta-model-based global optimization (HAM) and the kriging-based global optimization method using multi-point infill search criterion (KGOMISC).

Solving one-dimensional unconstrained global optimization problem using parameter free filled function method

It is generally known that almost all filled function methods for one-dimensional unconstrained global optimization problems have computational weaknesses. This paper introduces a relatively new parameter free filled function, which creates a non-ascending bridge from any local isolated minimizer to other first local isolated minimizer with lower or equal function value. The algorithm’s unprecedented function can be used to determine all extreme and inflection points between the two considered consecutive local isolated minimizers. The proposed method never fails to carry out its job. The results of the several testing examples have shown the capability and efficiency of this algorithm while at the same time, proving that the computational weaknesses of the filled function methods can be overcomed.

An unconstrained global optimization framework for real symmetric eigenvalue problems

In this work, we interpret real symmetric eigenvalue problems in an unconstrained global optimization framework. More precisely, given two N×N matrices, a symmetric matrix A, and a symmetric positive definite matrix B, we propose and analyze a nonconvex functional F whose local minimizers are, indeed, global minimizers. These minimizers correspond to eigenvectors of the generalized eigenvalue problem Ax=λBx associated with its smallest eigenvalue. To minimize the proposed functional F, we consider the gradient descent method and show its global convergence. Furthermore, we provide explicit error estimates for eigenvalues and eigenvectors at the kth iteration of the method in terms of the gradient of F at the kth iterate xk. At the end, we provide a few numerical experiments to confirm our analysis and to compare with other methods, which reveals interesting numerical aspects of our proposed model.

A Newton’s method characterization for real eigenvalue problems

The current work is a continuation of Kim (An unconstrained global optimization framework for real symmetric eigenvalue problems, submitted), where an unconstrained optimization problem was proposed and a first order method was shown to converge to a global minimizer that is an eigenvector corresponding to the smallest eigenvalue with no eigenvalue estimation given. In this second part, we provide local and global convergence analyses of the Newton’s method for real symmetric matrices. Our proposed framework discovers a new eigenvalue update rule and shows that the errors in eigenvalue and eigenvector estimations are comparable, which extends to nonsymmetric diagonalizable matrices as well. At the end, we provide numerical experiments for generalized eigenvalue problems and for the trust region subproblem discussed in Adachi et al. (SIAM J Optim 27(1):269–291, 2017) to confirm efficiency and accuracy of our proposed method.

A Memetic Chaotic Gravitational Search Algorithm for unconstrained global optimization problems

Metaheuristic optimization algorithms address two main tasks in the process of problem solving: i) exploration (also called diversification) and ii) exploitation (also called intensification). Guaranteeing a trade-off between these operations is critical to good performance. However, although many methods have been proposed by which metaheuristics can achieve a balance between the exploration and exploitation stages, they are still worse than exact algorithms at exploitation tasks, where gradient-based mechanisms outperform metaheuristics when a local minimum is approximated. In this paper, a quasi-Newton method is introduced into a Chaotic Gravitational Search Algorithm as an exploitation method, with the purpose of improving the exploitation capabilities of this recent and promising population-based metaheuristic. The proposed approach, referred to as a Memetic Chaotic Gravitational Search Algorithm, is used to solve forty-five benchmark problems, both synthetic and real-world, to validate the method. The numerical results show that the adding of quasi-Newton search directions to the original (Chaotic) Gravitational Search Algorithm substantially improves its performance. Also, a comparison with the state-of-the-art algorithms: Particle Swarm Optimization, Genetic Algorithm, Rcr-JADE, COBIDE and RLMPSO, shows that the proposed approach is promising for certain real-world problems.

Solving Unconstrained Global Optimization Problem Using Parameter Free Filled Function in Cooperation with Jameson Gradient Based Method

Global optimization problem still becomes a challenges due to the problem on locating the global optimum of multimodal function. How to reach the better minimizer from the current minimizer and how to decide that the obtained minimizer is the desired one are both major challenges on solving global optimization problem. Filled function method is one of the recent considered deterministic easy applied methods which concerned to the mentioned problems. The basic concept of filled function method is firstly by minimizing the objective function (first phase) then to build such an auxiliary function which to be minimized (second phase) in order to locate a point with lower function value than the current minimizer of the objective function. In the second phase, a local minimization method can be applied. Newton’s method is considered to be fast method on finding the zero of gradient of quadratic function, but may be very expensive or infeasible to determine the Hessian matrix in the case of complex problems. The Jameson gradient based method is the search procedures which avoid the need to store an estimate of the Hessian as well as its inverse and do not require exact line searches. In this paper, an algorithm in cooperation of parameter free filled function method and Jameson gradient method are introduced for solving global optimization problem with two variables. The algorithm is implemented to some benchmark test function. The numerical performance of the method on solving two-dimensional global optimization problems is presented.