Continuous Optimization Method Research Articles

Splitting for optimization

The splitting method is a well-known method for rare-event simulation, where sample paths of a Markov process are split into multiple copies during the simulation, so as to make the occurrence of a rare event more frequent. Motivated by the splitting algorithm we introduce a novel global optimization method for continuous optimization that is both very fast and accurate. Numerical experiments demonstrate that the new splitting-based method outperforms known methods such as the differential evolution and artificial bee colony algorithms for many bench mark cases.

Best subset selection via a modern optimization lens

In the period 1991–2015, algorithmic advances in Mixed Integer Optimization (MIO) coupled with hardware improvements have resulted in an astonishing 450 billion factor speedup in solving MIO problems. We present a MIO approach for solving the classical best subset selection problem of choosing $k$ out of $p$ features in linear regression given $n$ observations. We develop a discrete extension of modern first-order continuous optimization methods to find high quality feasible solutions that we use as warm starts to a MIO solver that finds provably optimal solutions. The resulting algorithm (a) provides a solution with a guarantee on its suboptimality even if we terminate the algorithm early, (b) can accommodate side constraints on the coefficients of the linear regression and (c) extends to finding best subset solutions for the least absolute deviation loss function. Using a wide variety of synthetic and real datasets, we demonstrate that our approach solves problems with $n$ in the 1000s and $p$ in the 100s in minutes to provable optimality, and finds near optimal solutions for $n$ in the 100s and $p$ in the 1000s in minutes. We also establish via numerical experiments that the MIO approach performs better than Lasso and other popularly used sparse learning procedures, in terms of achieving sparse solutions with good predictive power.

GACE: A meta-heuristic based in the hybridization of Genetic Algorithms and Cross Entropy methods for continuous optimization

Metaheuristics have proven to get a good performance solving difficult optimization problems in practice. Despite its success, metaheuristics still suffers from several problems that remains open as the variability of their performance depending on the problem or instance being solved. One of the approaches to deal with these problems is the hybridization of techniques. This paper presents a hybrid metaheuristic that combines a Genetic Algorithm (GA) with a Cross Entropy (CE) method to solve continuous optimization functions. The algorithm divides the population into two sub-populations, in order to apply GA in one sub-population and CE in the other. The proposed method is tested on 24 continuous benchmark functions, with four different dimension configurations. First, a study to find the best parameter configuration is done. The best configuration found is compared with several algorithms in the literature in order to demonstrate the competitiveness of the proposal. The results shows that GACE is the best performing method for instances with high dimensionality. Statistical tests have been applied, to support the conclusions obtained in the experimentation.

Trust-Region Methods Without Using Derivatives: Worst Case Complexity and the NonSmooth Case

Trust-region methods are a broad class of methods for continuous optimization that found application in a variety of problems and contexts. In particular, they have been studied and applied for problems without using derivatives. The analysis of trust-region derivative-free methods has focused on global convergence, and they have been proven to generate a sequence of iterates converging to stationarity independently of the starting point. Most of such an analysis is carried out in the smooth case, and, moreover, little is known about the complexity or global rate of these methods. In this paper, we start by analyzing the worst case complexity of general trust-region derivative-free methods for smooth functions. For the nonsmooth case, we propose a smoothing approach, for which we prove global convergence and bound the worst case complexity effort. For the special case of nonsmooth functions that result from the composition of smooth and nonsmooth/convex components, we show how to improve the existing resu...

Gaussian variable neighborhood search for the file transfer scheduling problem

This paper presents new modifications of Variable Neighborhood Search approach for solving the file transfer scheduling problem. To obtain better solutions in a small neighborhood of a current solution, we implement two new local search procedures. As Gaussian Variable Neighborhood Search showed promising results when solving continuous optimization problems, its implementation in solving the discrete file transfer scheduling problem is also presented. In order to apply this continuous optimization method to solve the discrete problem, mapping of uncountable set of feasible solutions into a finite set is performed. Both local search modifications gave better results for the large size instances, as well as better average performance for medium and large size instances. One local search modification achieved significant acceleration of the algorithm. The numerical experiments showed that the results obtained by Gaussian modifications are comparable with the results obtained by standard VNS based algorithms, developed for combinatorial optimization. In some cases Gaussian modifications gave even better results.

Particle Swarm Optimization for Integrated Fixture Layout

Fixture is a work-holding or supporting device used in the manufacturing industry to hold the workpiece. Fixtures are used to securely locate (position in a specific location or orientation) and support the work, ensuring that all parts produced using the fixture will maintain conformity and interchangeability. The location of fixture elements is called as fixture layout. The fixture layout plays major role in the work piece deformation during the machining operation. Hence optimization of fixture layout to minimize the work piece deformation is one of the critical aspects in the fixture design process. Minimization the workpiece deformation which is the objective function in the present work is calculated using Finite Element Method (FEM) and the fixture layout is optimized using Discrete fixture layout optimization method (DFLOM), Continuous fixture layout optimization method (CFLOM) and Integrated fixture layout optimization method (IFLOM).The workpiece deformation is minimum in Particle Swarm Optimization (PSO) based IFLOM is reported for the selected fixture. In this paper the PSO is used as an optimization tool to optimize the workpiece deformation.

Least quantile regression via modern optimization

We address the Least Quantile of Squares (LQS) (and in particular the Least Median of Squares) regression problem using modern optimization methods. We propose a Mixed Integer Optimization (MIO) formulation of the LQS problem which allows us to find a provably global optimal solution for the LQS problem. Our MIO framework has the appealing characteristic that if we terminate the algorithm early, we obtain a solution with a guarantee on its sub-optimality. We also propose continuous optimization methods based on first-order subdifferential methods, sequential linear optimization and hybrid combinations of them to obtain near optimal solutions to the LQS problem. The MIO algorithm is found to benefit significantly from high quality solutions delivered by our continuous optimization based methods. We further show that the MIO approach leads to (a) an optimal solution for any dataset, where the data-points $(y_i,\mathbf{x}_i)$'s are not necessarily in general position, (b) a simple proof of the breakdown point of the LQS objective value that holds for any dataset and (c) an extension to situations where there are polyhedral constraints on the regression coefficient vector. We report computational results with both synthetic and real-world datasets showing that the MIO algorithm with warm starts from the continuous optimization methods solve small ($n=100$) and medium ($n=500$) size problems to provable optimality in under two hours, and outperform all publicly available methods for large-scale ($n={}$10,000) LQS problems.

Solving air traffic conflict problems via local continuous optimization

This paper first introduces an original trajectory model using B-splines and a new semi-infinite programming formulation of the separation constraint involved in air traffic conflict problems. A new continuous optimization formulation of the tactical conflict-resolution problem is then proposed. It involves very few optimization variables in that one needs only one optimization variable to determine each aircraft trajectory. Encouraging numerical experiments show that this approach is viable on realistic test problems. Not only does one not need to rely on the traditional, discretized, combinatorial optimization approaches to this problem, but, moreover, local continuous optimization methods, which require relatively fewer iterations and thereby fewer costly function evaluations, are shown to improve the performance of the overall global optimization of this non-convex problem.

Fixture layout optimization in multi-station assembly processes using augmented ant colony algorithm

In recent years, ant colony algorithms (ACAs) are used to solve the fixture layout optimization problem for a single workpiece machined in a single manufacturing stage. Assembly processes, however, are normally multi-station manufacturing processes, whose fixture layout optimization problem is much more complex. The purpose of this research is to develop an augmented ACA based on continuous optimization methods to optimize fixture layouts for 2D rigid parts in multi-station assembly processes. The algorithm is augmented by changing the mutation step size in the global search, the limiting step size in the local search, the new pheromone value's expression of the ant, etc. The augmented ACA is used to properly select the coordinates of two locating pins to minimize the sensitivity index. A case about three-station automotive side aperture assembly processes is studied to verify the effectiveness of the augmented ACA. The results show that the augmented ACA can generate more accurate results with a faster rate of convergence and a better stability than the basic ACA. This work could also be applied to fixture layouts optimization problems for 3D rigid parts in multi-station manufacturing processes.

Small Solid Propellant Launch Vehicle Mixed Design Optimization Approach

For a small country with limited research budget and lack of advanced space technology, it is imperative to find new approaches for the development of low-cost launch vehicles (LV), which is, among all possibilities, an interesting option for rapid access to space, focused on integration of acquired components complemented with indigenously developed subsystems. This approach requires the cooperation of developed countries with huge experience and knowledge in LV development and operations. The main objective is to develop a small three stage solid propellant LV capable of delivering a payload of 100 kg to a circular low earth orbit of 600 km altitude, with the first and second stage solid rocket motors (SRM) hypothetically acquired from different countries and the third one designed and produced domestically in accordance with the production and technological capability. This approach provides main advantages such as: reduction in total time to access the space and to master the basic knowledge of launch operations. For this purpose, an integer continuous genetic algorithm global optimization method was selected and implemented, the SRM characteristics of the first and second stage were considered as integer variables, whereas the design variables of the third stage SRM and the trajectory variable were considered as continuous. A multi discipline feasible (MDF) framework was implemented along with the propulsion, aerodynamic, mass and trajectory models. Despite their particular characteristics and constraints, the results show highly acceptable values, and the approach proved to be reliable for conceptual design level.

An optimality criteria based method for discrete design optimization taking into account buildability constraints

This paper presents a heuristic design optimization method specifically developed for practicing structural engineers. Practical design optimization problems are often governed by buildability constraints. The majority of optimization methods that have recently been proposed for design optimization under buildability constraints are based on evolutionary computing. While these methods are generally easy to implement, they require a large number of function evaluations (finite element analyses), and they involve algorithmic parameters that require careful tuning. As a consequence, both the computation time and the engineering time are high. The discrete design optimization algorithm presented in this paper is based on the optimality criteria method for continuous optimization. It is faster than an evolutionary algorithm and it is free of tuning parameters. The algorithm is successfully applied to two classical benchmark problems (the design of a ten-bar truss and an eight-story frame) and to a practical truss design optimization problem.

Prediction of Machining Fixture Layout through FEM and ANN and Comparison of Optimal Fixture Layouts of GA and ACA

In machine manufacturing, workholding plays a vital role in immobilizing the workpiece to provide maximum accuracy. The workpiece should be securely located and held against the cutting tool to reduce workpiece deformation thereby reducing dimensional and form errors. Fixtures are workholding devices used in machining and assembly for holding, supporting and locating a workpiece in a given orientation. Fixturing elements such as locators and clamps are used to deprive the workpiece of all degrees of freedom so that it is constrained at all times to prevent any movement. The optimal positioning of locators and clamps highly influences the workpiece deformation and thereby the machining errors. A two-dimensional workpiece geometry involving a slot milling operation has been considered for the research work for which the workpiece deformation for different fixture layouts is determined using finite element method (FEM). The different layouts and the corresponding workpiece deformations from the simulation results are made available as the database for artificial neural network (ANN) to develop a numerical model that recognizes a pattern between the position of fixturing elements and the workpiece deformations. Using the recognized pattern from ANN, genetic algorithm (GA) based continuous optimization method and ant colony algorithm (ACA) based continuous optimization method have been used to determine the optimal position of locators and clamps to minimize the workpiece deformation and thereby the dimensional errors

Global Optimization of Protein–peptide Docking by a Filling Function Method

Molecular docking programs play a crucial role in drug design and development. In recent years, much attention has been devoted to the protein---peptide docking problem in which docking of a flexible peptide with a given protein is sought. In this work, we present a docking algorithm which is based on the use of a filling function method for continuous global optimization. In particular, the protein---peptide docking position is found by minimizing the conformational potential free energy function based on a new approximate mathematical model. The resulting global optimization problem presents some difficulties, since it is a large-scale one and the objective function is non-convex, so that it has many local minima. To solve the problem, we adopt a global optimization method based on the use of a filling function to escape from local solutions. Moreover, in order to obtain more accurate results, we search the correct docking position by performing a two-phase optimization process. In particular, in a first step, only the carbon $$\mathrm{C}_\alpha $$ C ? atoms of the protein and peptide are considered, thus obtaining an approximate docking solution. Then, the energy function is completed by considering all the peptide and protein atoms so that, starting from the solution of the first phase, the new minimization process gives a more accurate result. We present numerical results on a set of benchmark docking pairs and their comparison with those obtained by the known software package PacthDock for molecular docking.

A review of hybrid desalination systems for co-production of power and water: analyses, methods, and considerations

The production of freshwater from seawater is a growing necessity throughout the world. In arid areas with high temperature and salinity seawater, thermal desalination and power plants (dual-purpose/cogeneration plants) are often employed for the production of power and water. In other areas, reverse osmosis is commonly employed. However, both technologies are inherently challenged with economic and performance issues. As a response to these issues, hybrid desalination, that is, employing both thermal and mechanical desalination methods, has been increasingly utilized over thermal desalination plants alone. In this article, an overview of thermal desalination, seawater reverse osmosis (SWRO), and co-generation of power and water is presented, specifically with regards to the motivation for utilizing hybrid plants, for example, process limitations and areas of potential improvement. In addition, a review of the considerations for design and economics of hybrid desalination plants is presented, for example, existing system configurations, thermoeconomic analyses, and improvements of seawater pretreatment are discussed. Finally, studies for the optimization of hybrid desalination systems are reviewed. Specifically, the use of objective functions, continuous optimization methods, and optimal hardware configurations are discussed with respect to the key considerations of hybrid desalination plants.

A bi‐value coding parameterization scheme for the discrete optimal orientation design of the composite laminate

SUMMARYThe discrete optimal orientation design of the composite laminate can be treated as a material selection problem dealt with by using the concept of continuous topology optimization method. In this work, a new bi‐value coding parameterization (BCP) scheme of closed form is proposed to this aim. The basic idea of the BCP scheme is to ‘code’ each material phase using integer values of +1 and –1 so that each available material phase has one unique ‘code’ consisting of +1 and/or –1 assigned to design variables. Theoretical and numerical comparisons between the proposed BCP scheme and existing schemes show that the BCP has the advantage of an evident reduction of the number of design variables in logarithmic form. The benefit is particularly remarkable when the number of candidate materials becomes important in large‐scale problems. Numerical tests with up to 36 candidate material orientations are illustrated for the first time to indicate the reliability and efficiency of the BCP scheme in solving this kind of problem. It proves that the BCP is an interesting and valuable scheme to achieve the optimal orientations for large‐scale design problems. Besides, a four‐layer laminate example is tested to demonstrate that the proposed BCP scheme can easily be extended to multilayer problems. Copyright © 2012 John Wiley & Sons, Ltd.

Tabu search algorithm combined with global perturbation for packing arbitrary sized circles into a circular container

The arbitrary sized circle packing problem (ACP) is concerned about how to pack a number of arbitrary sized circles into a smallest possible circular container without overlapping. As a classical NP-hard problem, ACP is theoretically important and is often encountered in practical applications. Based on the already existing Quasi-physical method, this paper proposes a hybrid algorithm named GP-TS which combines tabu search with global perturbation to solve the two-dimensional ACP. The Quasi-physical method is a continuous optimization method which is used to obtain a local optimal configuration from any initial configuration. The tabu search procedure iteratively updates the incumbent configuration with its best neighboring configuration according to some forbidden rule and aspiration criterion. If the configuration obtained by the tabu search procedure does not satisfy the constraints, the global perturbation operator is subsequently applied in order that the search jumps out of the current local optimum without destroying the incumbent configuration too much. After that, the tabu search procedure is launched again. GP-TS is performed by repeating this process until the stop criterion is met. Computational experiments based on 3 sets of representative instances show that GP-TS can improve many best known results within reasonable time.

Approximate Fourier phase information in the phase retrieval problem: what it gives and how to use it

This work evaluates the importance of approximate Fourier phase information in the phase retrieval problem. The main discovery is that a rough phase estimate (up to π/2 rad) allows development of very efficient algorithms whose reconstruction time is an order of magnitude faster than that of the current method of choice--the hybrid input-output (HIO) algorithm. Moreover, a heuristic explanation is provided of why continuous optimization methods like gradient descent or Newton-type algorithms fail when applied to the phase retrieval problem and how the approximate phase information can remedy this situation. Numerical simulations are presented to demonstrate the validity of our analysis and success of our reconstruction method even in cases where the HIO algorithm fails, namely, complex-valued signals without tight support information.

A Tabu–Harmony Search-Based Approach to Fuzzy Linear Regression

We propose an unconstrained global continuous optimization method based on tabu search and harmony search to support the design of fuzzy linear regression (FLR) models. Tabu and harmony search strategies are used for diversification and intensification of FLR, respectively. The proposed approach offers the flexibility to use any kind of an objective function based on client's requirements or requests and the nature of the dataset and then attains its minimum error. Moreover, we elaborate on the error produced by this method and compare it with the errors resulting from the other known estimation methods. To study the performance of the method, three categories of datasets are considered: Numeric inputs-symmetric fuzzy outputs, symmetric fuzzy inputs-symmetric fuzzy outputs, and numeric inputs-asymmetric fuzzy outputs. Through a series of experiments, we demonstrate that in terms of the produced error with different model-fitting measurements, the proposed method outperforms or is Pareto-equivalent to the existing methods reported in the literature.

Continuous optimization methods for ground-state searches of spin clusters

Computational methods were developed for ground-state searches of Heisenberg model spin clusters in which spin sites were represented by classical spin vectors. Simulated annealing, continuous genetic algorithm, and particle swarm optimization methods were applied for solving the problems. Because the results of these calculations were influenced by the settings of optimization parameters, effective parameter settings were also investigated. The results indicated that a continuous genetic algorithm is the most effective method for ground-state searches of Heisenberg model spin clusters, and that a mutation operator plays an important role in this genetic algorithm. These results provide useful information for solving physically or chemically important continuous optimization problems.

Continuous global optimization in surface reconstruction from an oriented point cloud

We introduce a continuous global optimization method to the field of surface reconstruction from discrete noisy cloud of points with weak information on orientation. The proposed method uses an energy functional combining flux-based data-fit measures and a regularization term. A continuous convex relaxation scheme assures the global minima of the geometric surface functional. The reconstructed surface is implicitly represented by the binary segmentation of vertices of a 3D uniform grid and a triangulated surface can be obtained by extracting an appropriate isosurface. Unlike the discrete graph-cut solution, the continuous global optimization entails advantages like memory requirements, reduction of metrication errors for geometric quantities, and allowing globally optimal surface reconstruction at higher grid resolutions. We demonstrate the performance of the proposed method on several oriented point clouds captured by laser scanners. Experimental results confirm that our approach is robust to noise, large holes and non-uniform sampling density under the condition of very coarse orientation information.