Multi-start Algorithms Research Articles

Solving continuous quadratic programming for the discrete balanced graph partitioning problem using simulated annealing algorithm hybridized local search and multi-start algorithms

In a discrete balanced graph partitioning problem (DBGPP), a simple undirected weighted graph in that both vertices and edges are weighted is given. The task is dividing vertices into two disjoint subsets such that the sum of the weights of cut edges is minimized and the sum of the weights of vertices in each subset must equal or be as close as possible to each other. Here in order to solve the DBGPP, we first transform the problem into continuous quadratic programming and then show that the new problem has a binary solution, which is the optimal solution for the DBGPP. We also give necessary and sufficient conditions of continuous quadratic programming to identify stationary and local optimal solutions. In addition, we propose a local search to find a binary solution of the continuous quadratic programming. An approximated solution of the DBGPP is obtained using a hybrid simulated annealing algorithm. In our proposed algorithm, the structure of the objective function in a continues problem is considered as the evaluator function. Due to the Dolan–Moré performance profiles and the non-parametric statistical one-sided Wilcoxon signed rank test, we demonstrate the efficiency of our proposed approach in comparison with other available methods.

Design and Optimization of a Rack and Pinion Type WEC Using an Auxiliary Vibrating System

Research on wave energy converters with Rack and pinion type Power Take-Off (PTO) has been increasing over the last few years. A few control methods are used to optimize the performance of the said Wave Energy Converters (WECs). This paper presents a novel auxiliary vibrating system that can be implemented to improve the power input to a wave energy converter with a rack and pinion type PTO in regular waves. The design of the WEC system includes a floater, a double rack and pinion arrangement, a vibrating system, and a Mechanical Motion Rectifier (MMR) consisting of two one-way bearings that can convert the bidirectional wave motion to a unidirectional rotation of the output shaft. Once the waves move the floater upwards, this compresses the vibrating system which absorbs some of the energy and then the vibrating system helps the floater return to its original position by releasing the stored energy. The vibrating system also serves as a control method for limiting rack movement, so the impact of the waves is not detrimental to the system. This article aims to approximate the optimized power input to the system and investigate whether the implementation of a novel vibrating system improves the system power input. Allowing the WEC’s natural frequency to reach the wave’s natural frequency is important as it allows for maximum power absorption. The use of vibration systems to tune the WEC’s natural frequency close to the waves’ is novel and serves as the main factor in choosing this research. The WEC was modeled as 2 spring mass damper systems. Then the characteristic equations of the systems were extracted from the equations of motion and solved analytically to obtain the responses. One-factor-at-a-time (OFAT) method together with two different algorithms (Genetic and Multi-Start algorithms) from MATLAB code were used to optimize the response. The optimized power input to the system was then approximated. For system one, the maximum amplitude of the response was seen at a system mass of 500 kg and stiffness in the range of 100<k<240 N/m. The same was achieved for system two at a system mass of 500 kg and stiffness in the range of 100<k<138. The effect of the stiffness and mass on the response and input power has also been discussed.

Service Chain Caching and Workload Scheduling in Mobile Edge Computing

The computing and storage resources of an edge node in a mobile edge computing (MEC) system are limited. One edge computing node cannot cache all the services supported by the entire MEC system. A service chain caching and scheduling strategy should be designed to improve the service processing capacities of the edge computing nodes under high computing load conditions. This article combines the open Markov queuing network for the first time to model the process of edge node analysis and processing of computationally intensive tasks as a mixed integer nonlinear programming problem. Moreover, a mathematical model for the service chains is established. We analyze the average service response time during task execution to optimize the performance of service chain caching and scheduling. Experiments show that the outer approximation (OA) algorithm obtains better results than MindtPy and Multistart. The average response time obtained by the OA algorithm is 2.2% and 6.9% lower than those obtained by the MindtPy and Multistart algorithms, respectively. The OA algorithm using this mathematical model can effectively reduce the average service response time and improve the resource utilization of the edge server.

Comparison of B-Spline Surface and Free-Form Deformation Geometry Control for Aerodynamic Optimization

Two aerodynamic shape optimization geometry control methods, B-spline surface control and free-form deformation, are applied to three optimization problems and compared on the bases of optimal shape performance and problem setup ease of use. For both methods, the geometry is parameterized using B-spline surfaces with mesh movement accomplished using an efficient integrated technique. Gradients for the optimization algorithm are computed using the adjoint method. The first problem is a wing twist optimization under inviscid, subsonic flow, achieving an elliptical load distribution. The second is a lift-constrained drag minimization of a wing under transonic flow based on the Reynolds-averaged Navier–Stokes equations. The third involves lift-to-drag-ratio maximization, based on the Reynolds-averaged Navier–Stokes equations, beginning from a classically shaped blended wing–body aircraft and converging to a lifting-fuselage configuration. B-spline surface control is often found to result in slightly better performance; however, in general, both methods perform equally well. Free-form deformation provides a more general approach to problem setup, decoupling geometry control from parameterization. Overall, the results suggest that B-spline surface control is better suited for simple geometries, such as wings, whereas free-form deformation is better suited for complex geometries, such as unconventional aircraft, and for implementation with multistart algorithms and adaptive geometry control approaches.

Non-dominated sorting differential evolution algorithm for the minimization of route based fuel consumption multiobjective vehicle routing problems

In this paper, three parallel multi-start non-dominated sorting differential evolution algorithms (PMS-NSDEs) are proposed for the solution of four multiobjective route based fuel consumption vehicle routing problems (MRFCVRPs) and their results are compared with the results of a parallel multi-start NSGA II algorithm. All these algorithms use more than one initial population of solutions. In each algorithm a variable neighborhood search algorithm for the improvement of each solution separately is used. The problems that are formulated with two competitive objective functions are the multiobjective symmetric and asymmetric delivery route based fuel consumption vehicle routing problem (MSDRFCVRP and MADRFCVRP) and the multiobjective symmetric and asymmetric pick-up route based fuel consumption vehicle routing problem (MSPRFCVRP and MAPRFCVRP). The objective functions correspond to the optimization of the time needed for the vehicle to travel between two customers or between the customer and the depot and to the route based fuel consumption of the vehicle considering the traveled distance, the load of the vehicle, the slope of the road, the speed and the direction of the wind, and the driver’s behavior when the decision maker plans delivery or pick-up routes. A number of modified Vehicle Routing Problem instances are used in order to measure the quality of the proposed algorithms.

F-Flip strategies for unconstrained binary quadratic programming

Unconstrained binary quadratic programming (UBQP) provides a unifying modeling and solution framework for solving a remarkable range of binary optimization problems, including many accompanied by constraints. Current methods for solving UBQP problems customarily rely on neighborhoods consisting of flip moves that select one or more binary variables and “flip” their values to the complementary value (from 1 to 0 or from 0 to 1). We introduce a class of approaches called f-flip strategies that include a fractional value f as one of those available to the binary variables during intermediate stages of solution. A variety of different f-flip strategies, particularly within the context of multi-start algorithms, are proposed for pursuing intensification and diversification goals in metaheuristic algorithms, accompanied by special rules for evaluating and executing f-flips efficiently.

A review of distances for the Mallows and Generalized Mallows estimation of distribution algorithms

The Mallows (MM) and the Generalized Mallows (GMM) probability models have demonstrated their validity in the framework of Estimation of distribution algorithms (EDAs) for solving permutation-based combinatorial optimisation problems. Recent works, however, have suggested that the performance of these algorithms strongly relies on the distance used under the model. The goal of this paper is to review three common distances for permutations, Kendall's-$$\tau $$?, Cayley and Ulam, and compare their performance under MM and GMM EDAs. Moreover, with the aim of predicting the most suitable distance for solving any given permutation problem, we focus our attention on the relation between these distances and the neighbourhood systems in the field of local search optimisation. In this sense, we demonstrate that the performance of the MM and GMM EDAs is strongly correlated with that of multistart local search algorithms when using related neighbourhoods. Furthermore, by means of fitness landscape analysis techniques, we show that the suitability of a distance to solve a problem is clearly characterised by the generation of high smoothness fitness landscapes.

An algorithm for clusterwise linear regression based on smoothing techniques

We propose an algorithm based on an incremental approach and smoothing techniques to solve clusterwise linear regression (CLR) problems. This algorithm incrementally divides the whole data set into groups which can be easily approximated by one linear regression function. A special procedure is introduced to generate an initial solution for solving global optimization problems at each iteration of the incremental algorithm. Such an approach allows one to find global or approximate global solutions to the CLR problems. The algorithm is tested using several data sets for regression analysis and compared with the multistart and incremental Spath algorithms.

Resource allocation decision making in the military health system

The necessity to efficiently balance and re-allocate system resources among hospitals in a hospital network is paramount, especially as health systems experience increasing demand and costs for health services. In this paper, we proffer a resource allocation-based optimization model that adjusts resources (system inputs) automatically, which provides decision makers (such as health care managers and policy-makers) with a decision-support tool for re-allocating resources in large health systems that are centrally controlled and funded, such as the Military Health System. In these systems, inputs are fixed at certain levels and may only be adjusted within medical treatment facilities, while outputs must be maintained. We provide a mathematical formulation and example solutions from a case study using real-world data from sixteen U.S. Army hospitals. We also find utility in the use of multi-start evolutionary algorithms to store multiple optimal solutions for consideration by decision makers.

Global optimization of the Saccharomyces cerevisiae: Fermentation process

In this article, steady-state optimization of the Saccharomyces cerevisiae fermentation process problem is performed revealing the existence of multiple optimum solutions. The globally optimum solution was determined using the NEOS global optimization solver LINDO. A branch and bound strategy (bnb20.m) and the global search and multistart algorithms in the MATLAB global optimization toolbox were successful in determining locally optimum solutions and these results are validated by plotting the objective function against the decision variables. While in some cases all the strategies were successful in obtaining the globally optimum solutions, an example is presented where the most beneficial product value, which is not a stationary point and lies on the feasible boundary, is obtained by the LINDO global optimization solver (but not the other routines) as the globally optimum solution. The Jones-Kompala model was used to model the steady-state of the fermentation process. While several articles have been published demonstrating the existence of nonlinearities and bifurcations in this model, the challenges posed by this model to optimization has never been investigated so far and this work attempts to do so. Both dilution rate and the oxygen mass transfer coefficient were used as the decision variables individually and together.

A multistart gradient-based algorithm with surrogate model for global optimization

Gradient-based optimization algorithms are probably the most efficient option for the solution of a local optimization problem. These methods are intrinsically limited in the search of a local optimum of the objective function: if a global optimum is searched, the application of local optimization algorithms can be still successful if the algorithm is initialized starting from a large number of different points in the design space (multistart algorithms). As a counterpart, the cost of the exploration is further increased, linearly with the number of adopted starting points. Consequently, the use of a multistart local optimization algorithm is rarely adopted, mainly for two reasons: i) the large computa- tional cost and ii) the absence of a guarantee about the success of the search (in fact, there is not a general indication about the minimum number of starting points able to guarantee the success of global optimization). In this paper, techniques for reducing the computational cost of the full process together with some techniques able to maximize the efficiency of a parallel multistart search are described and tested. An extensive use of surrogate models is applied in order to drastically reduce the computational effort in practical applications, where the computational cost of a single objective function evaluation is high. Declustering techniques are also adopted in order to exploit at best the different local searches.

Properties and numerical testing of a parallel global optimization algorithm

In the framework of multistart and local search algorithms that find the global minimum of a real function f(x), $ x \in S\subseteq R^n$ , Gaviano et alias proposed a rule for deciding, as soon as a local minimum has been found, whether to perform or not a new local minimization. That rule was designed to minimize the average local computational cost $eval_1(\cdotp)$ required to move from the current local minimum to a new one. In this paper the expression of the cost $eval_2(\cdotp)$ of the entire process of getting a global minimum is found and investigated; it is shown that $eval_2(\cdotp)$ has among its components $eval_1(\cdotp)$ and can be only monotonically increasing or decreasing; that is, it exhibits the same property of $eval_1(\cdotp)$ . Moreover, a counterexample is given that shows that the optimal values given by $eval_1(\cdotp)$ and $eval_2(\cdotp)$ might not agree. Further, computational experiments of a parallel algorithm that uses the above rule are carried out in a MatLab environment.

On Multi-Start Algorithms for Optimization of High School Timetables

The paper deals with the problem of high-school time-tabling that is important in applications, but hard for solving. The algorithm is presented for timetabling based on Multi-start and Simulated Annealing with parameters adapted using the Bayes approach. The algorithm proposed is compared with other timetabling algorithms using the web-based software. A multi-start algorithm is a simple way to provide the convergence, if the number of uniformly distributed starting points is large. A disadvantage is slow convergence.Therefore, the first aim of this paper is experimental comparisons of the efficiency of different versions of multi-start algorithms in the optimization of timetables. To obtain representative results, the algorithms should be compatible with the Lithuanian high school practice and flexible enough for adaptation to different high schools.The second aim is a web-based implementation of these algorithms in a way convenient for high schools. The web-based software is important for evaluation and comparison of algorithms by independent experts, as well, since the efficiency of algorithms depends on subjective parameters specific to each school, so on-line calculations are needed to obtain representative data. It is useful for scientific cooperation and applications to different schools. In addition, the software for evaluating of real timetables is included to compare with the results of optimization.

Development and assessment of the SHARP and RandSHARP algorithms for the arc routing problem

The Capacitated Arc Routing Problem (CARP) is a combinatorial optimization problem similar to the well-known Capacitated Vehicle Routing Problem (CVRP). In the CARP, the customers' demands are located on the edges (arcs) of a general (not necessarily complete) graph. This is in contrast to the CVRP, where demand is located on the nodes of a complete graph. While the CVRP has been extensively studied over the past couple of decades, the number of articles and research results on the CARP is significantly lower. To partially fill this gap in the literature, a new heuristic and two new algorithms for the CARP are proposed and evaluated. Our Savings-based Heuristic for the ARP (SHARP) is inspired on the popular Clarke and Wright savings heuristic for the CVRP. The SHARP procedure outperforms the classical Path Scanning heuristic (PSH) and thus it can be integrated into most meta-heuristics to provide a ‘good’ and fast initial solution to medium-to-large size ARPs which cannot be solved using exact approaches. Using both SHARP and PSH as a base, two randomized algorithms are developed in this paper. These are multi-start algorithms which introduce a biased randomization process to each heuristic. This randomization process uses biased probability distributions, such as the geometric one, in order to induce some degree of randomness in the solution-construction stage while keeping most of the heuristic ‘common sense’. In order to state their efficiency, both randomized algorithms are compared and evaluated using a standard set of benchmarks. The results show that our randomized version of SHARP, RandSHARP, is quite competitive and far superior to the PSH-based randomized algorithm.

An ejection chain algorithm for the quadratic assignment problem

AbstractIn this study, we present a new tabu search algorithm for the quadratic assignment problem (QAP) that utilizes an embedded neighborhood construction called an ejection chain. Our ejection chain approach provides a combinatorial leverage effect, where the size of the neighborhood grows multiplicatively while the effort of finding a best move in the neighborhood grows only additively. Our results illustrate that significant improvement in solution quality is obtained in comparison to the traditional swap neighborhood. We also develop two multistart tabu search algorithms utilizing the ejection chain approach in order to demonstrate the power of embedding this neighborhood construction within a more sophisticated heuristic framework. Comparisons to the best large neighborhood approaches from the literature are presented. © 2009 Wiley Periodicals, Inc. NETWORKS, 2010

Calibration of subsurface batch and reactive-transport models involving complex biogeochemical processes

In this study, the calibration of subsurface batch and reactive-transport models involving complex biogeochemical processes was systematically evaluated. Two hypothetical nitrate biodegradation scenarios were developed and simulated in numerical experiments to evaluate the performance of three calibration search procedures: a multi-start non-linear regression algorithm (i.e. multi-start Levenberg–Marquardt), a global search heuristic (i.e. particle swarm optimization), and a hybrid algorithm that combines the particle swarm procedure with a regression-based “polishing” step. Graphical analysis of the selected calibration problems revealed heterogeneous regions of extreme parameter sensitivity and insensitivity along with abundant numbers of local minima. These characteristics hindered the performance of the multi-start non-linear regression technique, which was generally the least effective of the considered algorithms. In most cases, the global search and hybrid methods were capable of producing improved model fits at comparable computational expense. In other cases, the multi-start and hybrid calibration algorithms yielded comparable fitness values but markedly differing parameter estimates and associated uncertainty measures.

Global optimization of NURBs-based metamodels

The emergence of metamodels as approximate objective function representations offers the ability to ‘design’ metamodels with favourable optimization characteristics without compromising the accurate representational capabilities of arbitrary function topologies and modalities. With non-uniform rational B-splines (NURBs) as a metamodel basis, favourable optimization properties can be obtained which allow the intelligent selection of starting points for multistart optimization algorithms and which constrain optimization searches to metamodel regions containing the global metamodel optimum. In this article NURBs-based metamodels are used to define an optimization algorithm (HyPerOp) which guarantees the discovery of the global metamodel optimum with known computational effort. Emphasis is placed on demonstrating how NURBs’ properties contribute to a favourable objective function approximation. Through a large non-linear optimization trial problem set, the claim that HyPerOp is guaranteed to find the global metamodel optimum is demonstrated and the performance of HyPerOp with respect to random multistart approaches is evaluated.

Multistart algorithms for seeking feasibility

This paper describes modifications to two multistart algorithms for global optimization which enable them to find feasible solutions to a system of nonlinear constraints more efficiently. The multistart algorithms, called OptQuest-NLP (OQNLP) and Multistart-NLP (MSNLP), start a local NLP Solver from a set of starting points and return the best solution found. Candidate starting points are generated either by a scatter search heuristic or by a randomized process. Two adaptive filters choose a small subset of the candidate points as starting points. The modifications to facilitate feasibility seeking include replacing the exact penalty function used to measure the goodness of a starting point with the sum of infeasibilities, and terminating when a feasible solution is found. We describe experimental results on a large and diverse set of smooth nonlinear nonconvex problems coded in the GAMS modeling language. These are chosen so that a single application of a selected solver from the user-specified starting point terminates infeasible, yet the problems all have feasible solutions. Our results show that MSNLP's feasibility mode is able to find feasible solutions to almost all problems. It is moderately faster than MSNLP not using feasibility mode, and is somewhat better a finding feasible solutions when they exist. It is now an option within MSNLP, and can be invoked by inserting an appropriate record into the options file.

Local optima smoothing for global optimization

It is widely believed that in order to solve large-scale global optimization problems, an appropriate mixture of local approximation and global exploration is necessary. Local approximation, if first-order information on the objective function is available, is efficiently performed by means of local optimization methods. Unfortunately, global exploration, in absence of some kind of global information on the problem, is a ‘blind’ procedure, aimed at placing observations as evenly as possible in the search domain. Often, this procedure reduces to uniform random sampling (like in Multistart algorithms or in techniques based on clustering). In this paper, we propose a new framework for global exploration which tries to guide random exploration towards the region of attraction of low-level local optima. The main idea originated by the use of smoothing techniques (based on Gaussian convolutions): the possibility of applying a smoothing transformation not to the objective function but to the result of local searches seems to have never been explored yet. Although an exact smoothing of the results of local searches is impossible to implement, in this paper we propose a computational approximation scheme which has proven to be very efficient and (maybe more important) extremely robust in solving large-scale global optimization problems with huge numbers of local optima and, in particular, for problems displaying a ‘funnel’ structure.

Development of an efficient algorithm for global optimization by simplex elimination

An efficient multi-start algorithm for global optimization is developed by introducing multi-dimensional simplexes as new expression units of attraction regions. The region elimination method generally consists of making a set of eliminated regions called attraction regions, checking adjacency between the current design point and the attraction region, and quitting local optimization for the attracted design points. The efficiency of the elimination method is considerably enhanced by supplementing general simplexes and their neighborhoods to conventional units of attraction regions of points and lines. To show the effectiveness of the proposed algorithm, mathematical problems from the literature are solved and the results are compared with several well-known multi-start algorithms. The present algorithm produces the global optimum in all problems more efficiently than the variants of the multi-start method. Several types of truss, frame, and composite material structures are optimized as engineering applications. Many local optima are found and the differences among the local optima are not negligibly small. These results suggest that an efficient and reliable global optimizer is strongly required in some fields of engineering optimization.