Global Optimization Functions Research Articles

Sample-and-Bound for Non-convex Optimization

Standard approaches for global optimization of non-convex functions, such as branch-and-bound, maintain partition trees to systematically prune the domain. The tree size grows exponentially in the number of dimensions. We propose new sampling-based methods for non-convex optimization that adapts Monte Carlo Tree Search (MCTS) to improve efficiency. Instead of the standard use of visitation count in Upper Confidence Bounds, we utilize numerical overapproximations of the objective as an uncertainty metric, and also take into account of sampled estimates of first-order and second-order information. The Monte Carlo tree in our approach avoids the usual fixed combinatorial patterns in growing the tree, and aggressively zooms into the promising regions, while still balancing exploration and exploitation. We evaluate the proposed algorithms on high-dimensional non-convex optimization benchmarks against competitive baselines and analyze the effects of the hyper parameters.

Preprint: Global, and local optimization beamforming for broadband sources.

This paper presents an alternative energy function for global optimization (GO) beamforming, tailored to acoustic broadband sources. Given that properties, such as the source location, multipole rotation, or flow conditions, are parameterized over the frequency, a cross spectral matrix-fitting can be performed for all frequencies at once. A numerical analysis shows that the nonlinear energy function for the standard GO problem is equivalent to the source's point spread function (PSF) and contains local minima at the locations of the grating lobes and side lobes. The energy function is improved with the proposed broadband energy, as it averages the PSF. Further, it simplifies the process of identifying sources and reconstructing their spectra from the results. The paper shows that the method is superior on synthetic monopoles compared to standard GO and CLEAN based on source coherence (CLEAN-SC). For real-world data, the results of the proposed method and CLEAN-SC are similar, and outperform standard GO. The main difference is that source assumption violations cause noisy maps for CLEAN-SC and cause wrong spectral estimations of the proposed method. By using reasonable initial values, the GO problem reduces to a local optimization problem with similar results. Further, the proposed method is able to identify synthetic multipoles with different pole amplitudes and unknown pole rotations.

Global optimization and structural analysis of Coulomb and logarithmic potentials on the unit sphere using a population-based heuristic approach

Global optimization of high-dimensional non-convex functions arises in many important applications and researches. In this paper, we focus on the minimum energy configuration problem on the unit sphere, whose goal is to determine the minimum-energy configuration of N particles interacting via the standard Coulomb potential or the discrete logarithmic potential on the unit sphere. This is a classic global optimization problem with many important applications in mathematics, physics, biology and chemistry, and is one of the 18 great mathematical problems for the twenty-first century listed by the Fields Medal winner (Smale, 1998). To determine the minimum-energy configuration for these two potential functions, we propose a population-based global optimization algorithm that combines a two-phase local optimization method with early-stopping strategy, a random perturbation operator, a distance-based population update strategy using a novel distance function to measure the difference between solutions, and a population rebuilding method. Computational results show that the proposed algorithm is very competitive compared to the best known results in the literature. In particular, it improves the best-known result for one instance of the classic Thomson problem (i.e., N=360) widely studied in the literature. The configurations of instances with up to N=500 particles are systematically optimized for the first time by the proposed algorithm for the discrete logarithmic potential. The comparative study shows that the putative optimal configurations have a high similarity for the Coulomb and logarithmic potentials and that the defects play an important role in reducing the energy of configuration. The energies of particles on the putative optimal configurations are analyzed for the studied logarithmic potential.

NeuralBO: A black-box optimization algorithm using deep neural networks

Bayesian Optimization (BO) is an effective approach for the global optimization of black-box functions when function evaluations are expensive. Most prior works use Gaussian processes to model the black-box function, however, the use of kernels in Gaussian processes leads to two problems: first, the kernel-based methods scale poorly with the number of data points and second, kernel methods are usually not effective on complex structured high dimensional data due to curse of dimensionality. Therefore, we propose a novel black-box optimization algorithm where the black-box function is modeled using a neural network. Our algorithm does not need a Bayesian neural network to estimate predictive uncertainty and is therefore computationally favorable. We analyze the theoretical behavior of our algorithm in terms of regret bound using advances in NTK theory showing its efficient convergence. We perform experiments with both synthetic and real-world optimization tasks and show that our algorithm is more sample efficient compared to existing methods.

A Metaheuristic Approach to Parameter Estimation of a Flexible Parametric Mixture Cure Rate Model with Interval-Censored Data

A flexible parametric mixture cure model, called bi-lognormal cure rate model or simply BLN model, is defined and studied. The BLN model can be effectively used to analyze survival dataset in the presence of long-term survivors, especially when the dataset presents the underlying phenomenon of latent competing risks or when there is evidence that a bimodal hazard function is appropriated to described it, which are advantages over other cure rate models found in the literature. We discuss the maximum likelihood estimation for the model parameters considering interval-censored data through the differential evolution algorithm that is a nature-inspired computing metaheuristic used for global optimization of functions defined in multidimensional spaces. This approach is also used because the likelihood function of the model is multimodal and the direct application of gradient methods in this case is not ideal, since such methods are local search methods with a high chance of getting stuck at a local maximum when the starting point is chosen outside the basin of attraction of a global maximum. In addition, a simulation study was implemented to compare the performance of differential evolution algorithm with the performance of the Newton-Raphson algorithm in terms of bias, root mean square error, and the coverage probability of the asymptotic confidence intervals for the parameters. Finally, an application of the BLN model to real data is presented to illustrate that it can provide a better fit than other mixture cure rate models.

The digitization work of cement plant in China

This article is to introduce the digitization work of smart cement plant in China. To save power and promote the product quality, delicacy management in variety ways of digital transformation is the best tool for the new built, or existing cement plants. New technologies, such as special integrated robots, online equipment, information plant form, cement process digital applicant (SAAS), Artificial Intelligence (AI) and Advanced Process Control (APC) are implemented to approve the traditional ways of operation.This paper introduces the application of intelligent factory technology in China, and focuses on the intelligent optimization control system. The system has self-learning and global optimization functions, with strong adaptability and robustness. In combination with quality prediction, kiln condition identification and quantitative identification of fire watching TV video images, the kiln system is feed-forward controlled to solve the control problem of long time delay, large inertia and strong coupling of the burning system. The system gives the global optimal recommended value of the control target through calculation, with which to ensure the best control and operation of the system.With the application of comprehensive intelligent construction, the cement plant will usually save 2–5 % of energy and 20 % staff simplification, and make the management mode more concise and efficient. Meanwhile, the cement groups are also developing the production information plat form, by merging the Enterprise Resource Planning (ERP) data, sales business to lower down the total cost. This paper tries to introduce the digital efforts that are carried out in the smart cement plants and shows the methodology of implementation.

Optimization Analysis of Overlap Ratio in Wide and Heavy Plate Slitting Shear

In studying planar multi-bar mechanisms with multiple degrees of freedom, mathematical modeling is undoubtedly a way to get closer to the expected trajectory. Compared with the analytical method, the optimization method has higher accuracy in solving nonlinear equations, and it can be searched and iterated in an extensive range until it meets the real engineering solution. The research on the overlap ratio of the Slitting Shear essentially aims to study the kinematics of the two-DOF mechanism. In this study, a one-DOF mathematical model of mechanical motion was established for the Slitting Shear without considering the overlap ratio. The mathematical model was then verified through simulation. The overlap ratio was introduced based on the above mathematic model, and the three-DOF mathematic model of the shear mechanism was established. Finally, the setting of the overlap ratio was optimized using the conjugate gradient method and the global optimal NMinmize function. Under the condition of satisfying the overlap ratio, the recommended value for the overlap ratio setting of the two-DOF mechanism, which approximates the pure rolling cut, was determined.

Artificial Ecosystem-Based Optimization with Dwarf Mongoose Optimization for Feature Selection and Global Optimization Problems

Meta-Heuristic (MH) algorithms have recently proven successful in a broad range of applications because of their strong capabilities in picking the optimal features and removing redundant and irrelevant features. Artificial Ecosystem-based Optimization (AEO) shows extraordinary ability in the exploration stage and poor exploitation because of its stochastic nature. Dwarf Mongoose Optimization Algorithm (DMOA) is a recent MH algorithm showing a high exploitation capability. This paper proposes AEO-DMOA Feature Selection (FS) by integrating AEO and DMOA to develop an efficient FS algorithm with a better equilibrium between exploration and exploitation. The performance of the AEO-DMOA is investigated on seven datasets from different domains and a collection of twenty-eight global optimization functions, eighteen CEC2017, and ten CEC2019 benchmark functions. Comparative study and statistical analysis demonstrate that AEO-DMOA gives competitive results and is statistically significant compared to other popular MH approaches. The benchmark function results also indicate enhanced performance in high-dimensional search space.

Alternative variable-fidelity acquisition functions for efficient global optimization of black-box functions

Alternative variable-fidelity acquisition functions for efficient global optimization of black-box functions

Wasserstein enabled Bayesian optimization of composite functions

Bayesian optimization (BO) based on the Gaussian process model (GP-BO) has become the most used approach for the global optimization of black-box functions and computationally expensive optimization problems. BO has proved its sample efficiency and its versatility in a wide range of engineering and machine learning problems. A limiting factor in its applications is the difficulty of scaling over 15–20 dimensions. In order to mitigate this drawback, it has been remarked that optimization problems can have a lower intrinsic dimensionality. Several optimization strategies, built on this observation, map the original problem into a lower dimension manifold. In this paper we take a novel approach mapping the original problem into a space of discrete probability distributions endowed with a Wasserstein metric. The Wasserstein space is a non-linear manifold whose elements are discrete probability distributions. The input of the Gaussian process is given by discrete probability distributions and the acquisition function becomes a functional in the Wasserstein space. The minimizer of the acquisition functional in the Wasserstein space is then mapped back to the original space using a neural network. Computational results for three test functions with dimensionality ranging from 5 to 100, show that the exploration in the Wasserstein space is significantly more effective than that performed by plain Bayesian optimization in the Euclidean space and its advantage grows with the dimensions of the search space.

Visual/inertial odometry with tight coupling of multi-bit pose information

To address the problem that the loosely coupled frame pose scale needs to be initialized manually, making the subsequent algorithm less accurate and less real-time, while the tightly coupled OKVIS does not include closed-loop detection and map building, an adaptive vision/inertial odometry method with tightly coupled multi-bit pose information is proposed, incorporating optimized map points to provide more accurate data for the subsequent algorithm, with repositioning, closed-loop detection and global a complete visual/inertial odometry system with repositioning, closed-loop detection, and global posture map optimization functions. The adaptive vision/inertial odometry with tightly coupled multi-location posture information is experimentally validated to have better real-time performance, accuracy, and robustness than the loosely coupled algorithm.

A Method to Optimize the Inverse Operation of Electric Field Based on A Smart Algorithm

Along with the progress of electric network scale, power load and transmission lines are also growing rapidly in China. In addition, the transmission state detection is getting more and more attention. The paper explored the electric field under the HV (high voltage) line of transmission and researched the inverse operation principle of electric field. To solve the multiple parameter solutions in electric field inversion, the paper proposed two smart algorithms. Firstly, the fitness function of voltage value is extracted, therefore inversion algorithm global optimization function of the electric field is proposed. Secondly, an example of HV transmission line verified the effectiveness of the inverse optimization algorithm. Last, the case results indicate that the optimization methods are effective and accurate.

Solving Optimization Problems Using an Extended Gradient-Based Optimizer

This paper proposes an improved method for solving diverse optimization problems called EGBO. The EGBO stands for the extended gradient-based optimizer, which improves the local search of the standard version of the gradient-based optimizer (GBO) using expanded and narrowed exploration behaviors. This improvement aims to increase the ability of the GBO to explore a wide area in the search domain for the giving problems. In this regard, the local escaping operator of the GBO is modified to apply the expanded and narrowed exploration behaviors. The effectiveness of the EGBO is evaluated using global optimization functions, namely CEC2019 and twelve benchmark feature selection datasets. The results are analyzed and compared to a set of well-known optimization methods using six performance measures, such as the fitness function’s average, minimum, maximum, and standard deviations, and the computation time. The EGBO shows promising results in terms of performance measures, solving global optimization problems, recording highlight accuracies when selecting significant features, and outperforming the compared methods and the standard version of the GBO.

Deep Kernel Learning-Based Bayesian Optimization with Adaptive Kernel Functions

Bayesian optimization (BO) is a widely used data-driven method for the global optimization of black-box objective functions with noise. A key component of BO is surrogate model that is a probabilistic model used for approximating black-box objective functions. However, classical BO methods often utilize Gaussian processes with stationary covariance functions as surrogate models, which is difficult to accurately model complex systems with rapidly oscillating target functions, and therefore deteriorates the performance of BO. To this end, a Bayesian optimization approach based on deep kernel learning (DKL) is investigated for the optimization of complex objective functions with large oscillations, in which a Gaussian process based on deep kernel learning is utilized to capture the characteristics of data with non-stationary and hierarchical covariance functions via deep neural networks. The optimization performance of the DKL-based BO approach with different network structures is quantified and compared with a state-of-the-art BO method based on Gaussian process under the scenarios of different noise levels.

Continuous global optimization of multivariable functions based on Sergeev and Kvasov diagonal approach

Abstract. One of modern global optimization algorithms is method of Strongin and Piyavskii modified by Sergeev and Kvasov diagonal approach. In recent paper we propose an extension of this approach to continuous multivariable functions defined on the multidimensional parallelepiped. It is known that Sergeev and Kvasov method applies only to a Lipschitz continuous function though it effectively extends one-dimensional algorithm to multidimensional case. So authors modify We modify mentioned method to a continuous functions using introduced by Vanderbei ε-Lipschitz property that generalizes conventional Lipschitz inequality. Vanderbei proved that a real valued function is uniformly continuous on a convex domain if and only if it is ε-Lipschitz. Because multidimensional parallelepiped is a convex compact set, we demand objective function to be only continuous on a search domain. We describe extended Strongin’s and Piyavskii’s methods in the Sergeev and Kvasov modification and prove the sufficient conditions for the convergence. As an example of proposed method’s application, at the end of this article we show numerical optimization results of different continuous but not Lipschitz functions using three known partition strategies: “partition on 2”, “partition on 2N” and “effective”. For the first two of them we present formulas for computing a new iteration point and for recalculating the ε-Lipschitz constant estimate. We also show algorithm modification that allows to find a new search point on any algorithm’s step.

An Effective Metaheuristic Approach for Building Energy Optimization Problems

Mathematical optimization can be a useful strategy for minimizing energy usage while designing low-energy buildings. To handle building energy optimization challenges, this study provides an effective hybrid technique based on the pelican optimization algorithm (POA) and the single candidate optimizer (SCO). The suggested hybrid algorithm (POSCO) benefits from both the robust local search power of the single candidate method and the efficient global search capabilities of the pelican optimization. To conduct the building optimization task, the optimization method was developed and integrated with the EnergyPlus codes. The effectiveness of the proposed POSCO method was verified using mathematical test functions, and the outcomes were contrasted with those of conventional POA and other effective optimization techniques. Application of POSCO for global function optimization reveals that, among the thirteen considered functions, the proposed method was best at finding the global solution for seven functions, while providing superior results for the other functions when compared with competitive techniques. The suggested POSCO is applied for reducing an office buildings’ annual energy use. Comparing POSCO to POA procedures, the building energy usage is reduced. Furthermore, POSCO is compared to simple POA and other algorithms, with the results showing that, at specific temperatures and lighting conditions, the POSCO approach outperforms selected state-of-the-art methods and reduces building energy usage. As a result, all data suggests that POSCO is a very promising, dependable, and feasible optimization strategy for dealing with building energy optimization models. Finally, the building energy optimization findings for various climatic conditions demonstrate that the changes to the weather dataset had limited effect on the efficiency of the optimization procedure.

Efficient smooth minorants for global optimization of univariate functions with the first derivative satisfying the interval Lipschitz condition

AbstractIn 1998, the paper Sergeyev (Math Program 81(1):127–146, 1998) has been published where a smooth piece-wise quadratic minorant has been proposed for multiextremal functions f(x) with the first derivative $$f'(x)$$ f ′ ( x ) satisfying the Lipschitz condition with a constant L, i.e., $$f'(x)$$ f ′ ( x ) cannot increase with the slope higher than L and decrease with the slope smaller than $$-L$$ - L . This minorant has been successfully applied in several efficient global optimization algorithms and used in engineering applications. In the present paper, it is supposed that the first derivative $$f'(x)$$ f ′ ( x ) cannot increase with the slope higher than a constant $$\beta $$ β and decrease with the slope smaller than $$\alpha $$ α . The interval $$[\alpha ,\beta ]$$ [ α , β ] is called the Lipschitz interval (clearly, in this case the Lipschitz constant $$L = \max \{|\alpha |, |\beta | \}$$ L = max { | α | , | β | } ). For this class of functions, smooth piece-wise estimators (minorants and majorants) have been proposed and applied in global optimization. Both theoretically and experimentally (on 200 randomly generated test problems) it has been shown that in cases where $$ |\alpha | \ne |\beta |$$ | α | ≠ | β | the new estimators can give a significant improvement w.r.t. those proposed in Sergeyev (Math Program 81(1):127–146, 1998), for example, in the framework of branch-and-bound global optimization methods.

An Improved New Caledonian Crow Learning Algorithm for Global Function Optimization.

The New Caledonian crow learning algorithm (NCCLA) is a novel metaheuristic algorithm inspired by the learning behavior of New Caledonian crows learning to make tools to obtain food. However, it suffers from the problems of easily falling into local optima and insufficient convergence accuracy and convergence precision. To further improve the convergence performance of NCCLA, an improved New Caledonian crow learning algorithm (INCCLA) is proposed in this paper. By determining the parent individuals based on the cosine similarity, the juveniles are guided to search toward different ranges to maintain the population diversity; a novel hybrid mechanism of complete and incomplete learning is proposed to balance the exploration and exploitation capabilities of the algorithm; the update strategy of juveniles and parent individuals is improved to enhance the convergence speed and precision of the algorithm. The test results of the CEC2013 and CEC2020 test suites show that, compared with the original NCCLA algorithm and four of the best metaheuristics to date, INCCLA has significant advantages in terms of convergence speed, convergence precision, and stability.

Three-Dimensional Force Sensor Based on Fiber Bragg Grating for Medical Puncture Robot

In medical puncture robots, visible light, infrared and ultrasound images are currently used to guide punctures. The lack of information about the interaction forces between the puncture needle and soft tissue in different directions during the puncture process can easily lead to soft tissue being damaged. The current three-dimensional force sensors are large and can only be mounted on the base of the puncture needle, which does not allow for easy integration. Moreover, the force transfer to the base introduces various disturbing forces and the measurement accuracy is low. To reduce the risk of soft tissue being damaged and to enhance the intelligent control strategy of the puncture robot, this paper designs a three-dimensional force sensor based on fiber Bragg gratings. The sensor is very small and can be integrated into the back end of the puncture needle to accurately measure the interaction forces between the puncture needle and the soft tissue in different directions. The puncture needle wall is designed with notched bending of a multilayer continuous beam, which can increase the sensitivity of axial stiffness, while maintaining the sensitivity of the sensor to lateral bending and torsion, and also reduce the crosstalk between the axial and lateral forces. The finite element method is used to optimize its structural parameters, and a BP neural network based on the global optimal fitness function is proposed to solve the decoupling problem between the three-dimensional forces, which effectively improves the detection accuracy of the force sensor. The experimental results show that the measurement error of the sensor is less than 1.5%, which can accurately measure the interaction force between the puncture needle and the soft tissue and improve the safety of the puncture process.

An Efficient Improved Greedy Harris Hawks Optimizer and Its Application to Feature Selection.

To overcome the lack of flexibility of Harris Hawks Optimization (HHO) in switching between exploration and exploitation, and the low efficiency of its exploitation phase, an efficient improved greedy Harris Hawks Optimizer (IGHHO) is proposed and applied to the feature selection (FS) problem. IGHHO uses a new transformation strategy that enables flexible switching between search and development, enabling it to jump out of local optima. We replace the original HHO exploitation process with improved differential perturbation and a greedy strategy to improve its global search capability. We tested it in experiments against seven algorithms using single-peaked, multi-peaked, hybrid, and composite CEC2017 benchmark functions, and IGHHO outperformed them on optimization problems with different feature functions. We propose new objective functions for the problem of data imbalance in FS and apply IGHHO to it. IGHHO outperformed comparison algorithms in terms of classification accuracy and feature subset length. The results show that IGHHO applies not only to global optimization of different feature functions but also to practical optimization problems.