General Optimization Algorithm Research Articles

ROPTLIB

Riemannian optimization is the task of finding an optimum of a real-valued function defined on a Riemannian manifold. Riemannian optimization has been a topic of much interest over the past few years due to many applications including computer vision, signal processing, and numerical linear algebra. The substantial background required to successfully design and apply Riemannian optimization algorithms is a significant impediment for many potential users. Therefore, multiple packages, such as Manopt (in Matlab) and Pymanopt (in Python), have been developed. This article describes ROPTLIB, a C++ library for Riemannian optimization. Unlike prior packages, ROPTLIB simultaneously achieves the following goals: (i) it has user-friendly interfaces in Matlab, Julia, and C++; (ii) users do not need to implement manifold- and algorithm-related objects; (iii) it provides efficient computational time due to its C++ core; (iv) it implements state-of-the-art generic Riemannian optimization algorithms, including quasi-Newton algorithms; and (v) it is based on object-oriented programming, allowing users to rapidly add new algorithms and manifolds.

Efficient calculation of constraint back-offs for optimization under uncertainty: A case study on maleic anhydride synthesis

In the present work we propose an efficient and general algorithm for optimization under uncertainty based on the work of Srinivasan et al. (2003). We use specialized cubature rules to speed up the uncertainty propagation step which results in a significant reduction of the overall computational effort. The approach is illustrated by studying the optimal design of a fixed bed reactor for the synthesis of maleic anhydride from raffinate II feedstock, where the amount of n-butane and n-butenes in the feed is assumed to be uncertain. Applying the algorithm results in a robustified reactor design which shows significantly less temperature constraint violations and runaway conditions while still satisfying reactor performance criteria such as minimally required conversion and maximum allowable pressure drop.

Parallel two-phase methods for global optimization on GPU

Developing general global optimization algorithms is a difficult task, specially for functions with a huge number of local minima in high dimensions. Stochastic metaheuristic algorithms can provide the only alternative for the solution of such problems since they are aimed at guaranteeing global optimality. However, the main drawback of these algorithms is that they require a large number of function evaluations in order to skip/discard local optima, thus exhibiting a low convergence order and, as a result, a high computational cost. Furthermore, the situation can become even worse with the increase of dimension. Usually the number of local minima highly increases, as well as the computational cost of the function evaluation, thus increasing the difficulty for covering the whole search space. On the other hand, deterministic local optimization methods exhibit faster convergence rates, requiring a lower number of functions evaluations and therefore involving a lower computational cost, although they can get stuck into local minima. A way to obtain faster global optimization algorithms is to mix local and global methods in order to benefit from higher convergence rates of local ones, while retaining the global approximation properties. Another way to speedup global optimization algorithms comes from the use of efficient parallel hardware architectures. Nowadays, a good alternative is to take advantage of graphics processing units (GPUs), which are massively parallel processors and have become quite accessible cheap alternative for high performance computing. In this work a parallel implementation on GPUs of some hybrid two-phase optimization methods, that combine the metaheuristic Simulated Annealing algorithm for finding a global minimum, with different local optimization methods, namely a conjugate gradient algorithm and a version of Nelder–Mead method, is presented. The performance of parallelized versions of the above hybrid methods are analyzed for a set of well known test problems. Results show that GPUs represent an efficient alternative for the parallel implementation of two-phase global optimization methods.

Mirror descent search and its acceleration

In recent years, attention has been focused on the relationship between black-box optimization problem and reinforcement learning problem. In this research, we propose the Mirror Descent Search (MDS) algorithm which is applicable both for black box optimization problems and reinforcement learning problems. Our method is based on the mirror descent method, which is a general optimization algorithm. The contribution of this research is roughly twofold. We propose two essential algorithms, called MDS and Accelerated Mirror Descent Search (AMDS), and two more approximate algorithms: Gaussian Mirror Descent Search (G-MDS) and Gaussian Accelerated Mirror Descent Search (G-AMDS). This research shows that the advanced methods developed in the context of the mirror descent research can be applied to reinforcement learning problem. We also clarify the relationship between an existing reinforcement learning algorithm and our method. With two evaluation experiments, we show our proposed algorithms converge faster than some state-of-the-art methods.

Riemannian Optimization and Approximate Joint Diagonalization for Blind Source Separation

We consider the blind source separation (BSS) problem and the closely related approximate joint diagonalization (AJD) problem of symmetric positive definite (SPD) matrices. These two problems can be reduced to an optimization problem with three key components: the criterion to minimize, the constraint on the solution, and the optimization algorithm to solve it. This paper contains two contributions that allow us to treat these issues independently. We build the first complete Riemannian optimization framework suited for BSS and AJD handling three classical constraints, and allowing to use a large panel of general optimization algorithms on manifolds. We also perform a thorough study of the AJD problem of SPD matrices from an information geometry point of view. We study AJD criteria based on several divergences of the set of SPD matrices, provide three optimization strategies to minimize them, and analyze their properties. Our numerical experiments on simulated and pseudoreal electroencephalographic data show the interest of the Riemannian optimization framework and of the different AJD criteria we consider.

A simplicial homology algorithm for Lipschitz optimisation

The simplicial homology global optimisation (SHGO) algorithm is a general purpose global optimisation algorithm based on applications of simplicial integral homology and combinatorial topology. SHGO approximates the homology groups of a complex built on a hypersurface homeomorphic to a complex on the objective function. This provides both approximations of locally convex subdomains in the search space through Sperner’s lemma and a useful visual tool for characterising and efficiently solving higher dimensional black and grey box optimisation problems. This complex is built up using sampling points within the feasible search space as vertices. The algorithm is specialised in finding all the local minima of an objective function with expensive function evaluations efficiently which is especially suitable to applications such as energy landscape exploration. SHGO was initially developed as an improvement on the topographical global optimisation (TGO) method. It is proven that the SHGO algorithm will always outperform TGO on function evaluations if the objective function is Lipschitz smooth. In this paper SHGO is applied to non-convex problems with linear and box constraints with bounds placed on the variables. Numerical experiments on linearly constrained test problems show that SHGO gives competitive results compared to TGO and the recently developed Lc-DISIMPL algorithm as well as the PSwarm, LGO and DIRECT-L1 algorithms. Furthermore SHGO is compared with the TGO, basinhopping (BH) and differential evolution (DE) global optimisation algorithms over a large selection of black-box problems with bounds placed on the variables from the SciPy benchmarking test suite. A Python implementation of the SHGO and TGO algorithms published under a MIT license can be found from https://bitbucket.org/upiamcompthermo/shgo/ .

Synthesis of Coupling Matrix for Diplexers Based on a Self-Adaptive Differential Evolution Algorithm

Diplexer coupling matrix synthesis often involves both analytical methods and optimization techniques. At present, general purpose optimization algorithms are used, but they need strong supporting information (e.g., high-quality starting points and very narrow search ranges) from analytical methods, which is not available or too complex to be obtained in many cases. Aiming to obtain the desired coupling matrix with highly reduced supporting information to relieve the pressure of analytical methods, a new optimization algorithm, called self-adaptive differential evolution for coupling matrix synthesis (SADEC), is proposed. Considering the landscape characteristics of diplexer coupling matrix synthesis problems, a new self-adaptive multipopulation search framework and a self-adaptive algorithm parameter control strategy are proposed and organized in a particular way. The performance of SADEC is demonstrated by two all-resonator-based narrowband diplexers using large search ranges only with the requirement of matching the diplexer topology and no ad hoc analysis is included. Experiments and comparisons show the high performance of SADEC and clear advantages compared with the state-of-the-art global optimization methods. SADEC is also applicable to filter coupling matrix synthesis and is downloadable.

Design and simulation of MEMS-actuated adjustable optical wedge for laser beam scanners

This paper introduces both optical and mechanical design and simulation of large static deflection MOEMS actuator. The designed device is in the form of an adjustable optical wedge (AOW) laser scanner. The AOW is formed of 1.5-mm-diameter plano-convex lens separated by air gap from plano-concave fixed lens. The convex lens is actuated by staggered vertical comb drive and suspended by rectangular cross-section torsion beam. An optical analysis and simulation of air separated AOW as well as detailed design, analysis, and static simulation of comb -drive are introduced. The dynamic step response of the full system is also introduced. The analytical solution showed a good agreement with the simulation results. A general global minimum optimization algorithm is applied to the comb-drive design to minimize driving voltage. A maximum comb-drive mechanical deflection angle of 12 deg in each direction was obtained under DC actuation voltage of 32 V with a settling time of 90 ms, leading to 1-mm one-dimensional (1-D) steering of laser beam with continuous optical scan angle of 5 deg in each direction. This optimization process provided a design of larger deflection actuator with smaller driving voltage compared with other conventional devices. This enhancement could lead to better performance of MOEMS-based laser beam scanners for imaging and low-speed applications.

Solve train stowage planning problem of steel coil using a pointer-based discrete differential evolution

Train stowage planning problem of steel coil (TSPP) is to determine loading locations of the coils on train railcars, which is rarely studied, and an important optimisation problem in real iron and steel industry. In this paper, first, according to the actual situations of a steel products transportation department, the problem is formulated as an integer programming (IP) mathematical model in which multiple destination stations, varied stowing modes and different railcar types are considered. Then, on the basis of a general discrete optimisation algorithm framework, i.e. pointer-based discrete differential evolution (PDDE), a novel variant (T-PDDE) is proposed for effectively solving the TSPP. In particular, to deal with the issue of transformation between stowage plan and algorithm individual, a problem-based coding method is designed. To further enhance the algorithm performance, a double levels evolution strategy and an opposite-based local search are developed based on the features of problem. Finally, with the practical and simulative data, extensive comparison experiments are carried out to evaluate the proposed algorithm. The numerical results demonstrate the superiority of T-PDDE on solving TSPP.

Genetic Algorithm Based Feeder Reconfiguration and Control of Reactive Power for Loss Reduction

Power distribution network consists of a group of radial feeders which can be connected together by several tie-switches and tie-lines. The power loss reduction in the network is a major concern of electric distribution utilities. Among conventional methods, optimal reconfiguration and capacitor placement are two effective methods which can be applied on the network. In this paper a feeder reconfiguration algorithm is build and present for
 the purpose of power loss reduction in distribution systems. The methodology developed combine optimization technique. The network reconfiguration problem is formulated as single objective optimization problem with equality and inequality constraints. The proposed solution to this problem is based on a general combinatorial optimization algorithm known as genetic algorithm, and the load flow equations in distribution
 system. The GA optimizes the system-switching pattern for losses reduction. Decides the system topology, and considered taken the size and location of capacitor placement in the radial distribution networks. The two selection method used in genetic algorithm, first method roulette wheel
 selection and second method tournament selection. Tests show the different in execution time of the program to arrive the best fitness function.
 The proposed algorithm has been implemented in technical MATLAB package and tested for practical network (48-bus) which is taken from the distribution system of Palestine Street in Baghdad city. Tests show that GA is suitable algorithm as optimization technique, high accuracy, and avoid local minimum by searching in several regions to arrive to the global optimum solution. Thus, the outcome of the study shows an efficient technique to solve the problem of network reconfiguration for loss reduction in the distribution power system.

Optimal polyhedral description of 3D polycrystals: Method and application to statistical and synchrotron X-ray diffraction data

A methodology is presented for optimal polyhedral description of 3D polycrystals from experimental properties. This is achieved by determining, by optimization, appropriate attributes of the seeds of Laguerre tessellations. The resulting tessellations are optimal in the sense that no further improvements are possible using convex geometries. The optimization of Laguerre tessellation combines a new, computationally-efficient algorithm for updating tessellations between iterations to a generic optimization algorithm. The method is applied to different types of experimental data, either statistical, such as grain size distributions, or grain-based, as provided by synchrotron X-ray diffraction experiments. It is then shown how the tessellations can be meshed for finite-element simulations. The new method opens the way to more systematic and quantitative analyses of microstructural effects on properties. The presented algorithms are implemented and distributed in the free (open-source) software package Neper.

Recent advances in stellarator optimization

Computational optimization has revolutionized the field of stellarator design. To date, optimizations have focused primarily on optimization of neoclassical confinement and ideal MHD stability, although limited optimization of other parameters has also been performed. The purpose of this paper is to outline a select set of new concepts for stellarator optimization that, when taken as a group, present a significant step forward in the stellarator concept. One of the criticisms that has been leveled at existing methods of design is the complexity of the resultant field coils. Recently, a new coil optimization code—COILOPT++, which uses a spline instead of a Fourier representation of the coils,—was written and included in the STELLOPT suite of codes. The advantage of this method is that it allows the addition of real space constraints on the locations of the coils. The code has been tested by generating coil designs for optimized quasi-axisymmetric stellarator plasma configurations of different aspect ratios. As an initial exercise, a constraint that the windings be vertical was placed on large major radius half of the non-planar coils. Further constraints were also imposed that guaranteed that sector blanket modules could be removed from between the coils, enabling a sector maintenance scheme. Results of this exercise will be presented. New ideas on methods for the optimization of turbulent transport have garnered much attention since these methods have led to design concepts that are calculated to have reduced turbulent heat loss. We have explored possibilities for generating an experimental database to test whether the reduction in transport that is predicted is consistent with experimental observations. To this end, a series of equilibria that can be made in the now latent QUASAR experiment have been identified that will test the predicted transport scalings. Fast particle confinement studies aimed at developing a generalized optimization algorithm are also discussed. A new algorithm developed for the design of the scraper element on W7-X is presented along with ideas for automating the optimization approach.

Conformational Space Annealing explained: A general optimization algorithm, with diverse applications

Many problems in science and engineering can be formulated as optimization problems. One way to solve these problems is to develop tailored problem-specific approaches. As such development is challenging, an alternative is to develop good generally-applicable algorithms. Such algorithms are easy to apply, typically function robustly, and reduce development time. Here we provide a description for one such algorithm called Conformational Space Annealing (CSA) along with its python version, PyCSA. We previously applied it to many optimization problems including protein structure prediction and graph community detection. To demonstrate its utility, we have applied PyCSA to two continuous test functions, namely Ackley and Eggholder functions. In addition, in order to provide complete generality of PyCSA to any types of an objective function, we demonstrate the way PyCSA can be applied to a discrete objective function, namely a parameter optimization problem. Based on the benchmarking results of the three problems, the performance of CSA is shown to be better than or similar to the most popular optimization method, simulated annealing. For continuous objective functions, we found that, L-BFGS-B was the best performing local optimization method, while for a discrete objective function Nelder–Mead was the best. The current version of PyCSA can be run in parallel at the coarse grained level by calculating multiple independent local optimizations separately. The source code of PyCSA is available from http://lee.kias.re.kr.

A meta optimisation analysis of particle swarm optimisation velocity update equations for watershed management learning

Particle swarm optimisation (PSO) is a general purpose optimisation algorithm used to address hard optimisation problems. The algorithm operates as a result of a number of particles converging on what is hoped to be the best solution. How the particles move through the problem space is therefore critical to the success of the algorithm. This study utilises meta optimisation to compare a number of velocity update equations to determine which features of each are of benefit to the algorithm. A number of hybrid velocity update equations are proposed based on other high performing velocity update equations. This research also presents a novel application of PSO to train a neural network function approximator to address the watershed management problem. It is found that the standard PSO with a linearly changing inertia, the proposed hybrid Attractive Repulsive PSO with avoidance of worst locations (AR PSOAWL) and Adaptive Velocity PSO (AV PSO) provide the best performance overall. The results presented in this paper also reveal that commonly used PSO parameters do not provide the best performance. Increasing and negative inertia values were found to perform better.

Rainbow boxes: A new technique for overlapping set visualization and two applications in the biomedical domain

Overlapping set visualization is a well-known problem in information visualization. This problem considers elements and sets containing all or part of the elements, a given element possibly belonging to more than one set. A typical example is the properties of the 20 amino-acids. A more complex application is the visual comparison of the contraindications or the adverse effects of several similar drugs. The knowledge involved is voluminous, each drug has many contraindications and adverse effects, some of them are shared with other drugs. Another real-life application is the visualization of gene annotation, each gene product being annotated with several annotation terms indicating the associated biological processes, molecular functions and cellular components.In this paper, we present rainbow boxes, a novel technique for visualizing overlapping sets, and its application to the presentation of the properties of amino-acids, the comparison of drug properties, and the visualization of gene annotation. This technique requires solving a combinatorial optimization problem; we propose a specific heuristic and we evaluate and compare it to general optimization algorithms. We also describe a user study comparing rainbow boxes to tables and showing that the former allowed physicians to find information significantly faster. Finally, we discuss the limits and the perspectives of rainbow boxes.

Calibration of Hygrothermal Simulations by the Help of a Generic Optimization Tool

Hygrothermal simulation represents an established method for the performance evaluation of building constructions. Some validation cases are known where the validity of these models was approved in form of a comparison between measured and simulated quantities under controlled boundary conditions. A further step is commonly not explained but also included, the calibration of uncertainly known model settings, e.g. surface exchange coefficients or measured material properties with a relevant uncertainty range. This paper summarizes a calibration procedure, which was carried out for the data analysis of a hygrothermal test stand with two- and three-dimensional junctions, including brickwork and joist ends. The calibration procedure is based on the definition of quantitative criteria (target values) that describe the accordance between measured and simulated data series. Starting with the thermal two-dimensional simulation model, uncertain parameters are defined with their start values, ranges and steps of variation. This is used as an input for the identification of the best-fitting parameter combination via generic optimization algorithm. The final calibrated simulation model shows a remarkably better accordance to the measurement results. The procedure is systematic and reproducible.

Realizing the empirical mode decomposition by the adaptive stochastic resonance in a new periodical model and its application in bearing fault diagnosis

We investigate a multi-frequency signal that cannot be decomposed by empirical mode decomposition directly. Moreover, this kind of signal in the noisy background cannot be decomposed successfully by the traditional stochastic resonance with bistable system yet. We propose a new method which using the empirical mode decomposition combined the adaptive stochastic resonance in a new periodical model to solve this problem. The results show that the proposed method decomposes the multi-frequency signal perfectly. Meanwhile, the general scale transformation and random particle swarm optimization algorithm are used to help obtain a better result in the process of optimization. Through using this new method, the simulation results are satisfactory. More importantly, this new method also shows good performance in the application of bearing fault diagnosis.

Comparative analysis of the nonlinear unified strength criterion for rocks and other three-dimensional Hoek–Brown strength criteria

The Hoek–Brown strength criterion, which has been widely used in rock engineering, is one of the most influential rock strength criteria. Because this criterion does not take into account the effect of intermediate principal stress and because there are deficiencies in the theory and its practical application, for nearly 30 years many scholars have carried out research related to this topic. Yu and Zan took the advantages of the unified strength theory and integrated this theory with the Hoek–Brown strength criterion, thereby proposing the nonlinear unified strength criterion for rocks. This criterion takes into account the difference between the tensile strength and the compressive strength of rock, i.e. the intermediate principal stress effect, and the researchers determined that the limit loci in the meridian plane are nonlinear. The parameters of the nonlinear unified strength criterion are the same as those of the Hoek–Brown strength criterion obtained from conventional triaxial tests for rock blocks. The current study compared and analyzed the relationship between the nonlinear unified strength criterion and other three-dimensional Hoek–Brown strength criteria proposed by Pan and Hudson, Zhang and Zhu, and Jiang. Using the comparison of failure limit lines of the various strength criteria in the π-plane, the nonlinear unified strength criterion is superior to other strength criteria. Using the 1stOpt software of the general global optimization algorithm to fit the strength criterion parameters, the comparison of the results of the nonlinear unified strength criterion with the true triaxial test data from six types of rocks shows good agreement, and the true triaxial test of rock strength can be evaluated with high accuracy.

Suppression of Multiphoton Resonances in Driven Quantum Systems via Pulse Shape Optimization.

This Letter demonstrates control over multiphoton absorption processes in driven two-level systems, which include, for example, superconducting qubits or laser-irradiated graphene, through spectral shaping of the driving pulse. Starting from calculations based on Floquet theory, we use differential evolution, a general purpose optimization algorithm, to find the Fourier coefficients of the driving function that suppress a given multiphoton resonance in the strong field regime. We show that the suppression of the transition probability is due to the coherent superposition of high-order Fourier harmonics which closes the dynamical gap between the Floquet states of the two-level system.

Two-Scale Topology Optimization with Microstructures

In this article, we present a novel two-scale framework to optimize the structure and the material distribution of an object given its functional specifications. Our approach utilizes multi-material microstructures as low-level building blocks of the object. We start by precomputing the material property gamut—the set of bulk material properties that can be achieved with all material microstructures of a given size. We represent the boundary of this material property gamut using a level set field. Next, we propose an efficient and general topology optimization algorithm that simultaneously computes an optimal object topology and spatially varying material properties constrained by the precomputed gamut. Finally, we map the optimal spatially varying material properties onto the microstructures with the corresponding properties to generate a high-resolution printable structure. We demonstrate the efficacy of our framework by designing, optimizing, and fabricating objects in different material property spaces on the level of a trillion voxels, that is, several orders of magnitude higher than what can be achieved with current systems.