General-purpose Optimization Algorithms Research Articles

An Overview of Adaptive Designs and Some of Their Challenges, Benefits, and Innovative Applications.

Adaptive designs are increasingly developed and used to improve all phases of clinical trials and in biomedical studies in various ways to address different statistical issues. We first present an overview of adaptive designs and note their numerous advantages over traditional clinical trials. In particular, we provide a concrete demonstration that shows how recent adaptive design strategies can further improve an adaptive trial implemented 13 years ago. Despite their usefulness, adaptive designs are still not widely implemented in clinical trials. We offer a few possible reasons and propose some ways to use them more broadly in practice, which include greater availability of software tools and interactive websites to generate optimal adaptive trials freely and effectively, including the use of metaheuristics to facilitate the search for an efficient trial design. To this end, we present several web-based tools for finding various adaptive and nonadaptive optimal designs and discuss nature-inspired metaheuristics. Metaheuristics are assumptions-free general purpose optimization algorithms widely used in computer science and engineering to tackle all kinds of challenging optimization problems, and their use in designing clinical trials is just emerging. We describe a few recent such applications and some of their capabilities for designing various complex trials. Particle swarm optimization is an exemplary nature-inspired algorithm, and similar to others, it has a simple definition but many moving parts, making it hard to study its properties analytically. We investigated one of its hitherto unstudied issues on how to bring back out-of-range candidates during the search for the optimum of the search domain and show that different strategies can impact the success and time of the search. We conclude with a few caveats on the use of metaheuristics for a successful search.

Safeguarded Learned Convex Optimization

Applications abound in which optimization problems must be repeatedly solved, each time with new (but similar) data. Analytic optimization algorithms can be hand-designed to provably solve these problems in an iterative fashion. On one hand, data-driven algorithms can "learn to optimize" (L2O) with much fewer iterations and similar cost per iteration as general-purpose optimization algorithms. On the other hand, unfortunately, many L2O algorithms lack converge guarantees. To fuse the advantages of these approaches, we present a Safe-L2O framework. Safe-L2O updates incorporate a safeguard to guarantee convergence for convex problems with proximal and/or gradient oracles. The safeguard is simple and computationally cheap to implement, and it is activated only when the data-driven L2O updates would perform poorly or appear to diverge. This yields the numerical benefits of employing machine learning to create rapid L2O algorithms while still guaranteeing convergence. Our numerical examples show convergence of Safe-L2O algorithms, even when the provided data is not from the distribution of training data.

Steam allocation for SAGD: Multi-pad multi-criteria short and long-term real-time performance management

Optimizing steam allocation is critical for long-term performance goals in Steam-Assisted Gravity Drainage (SAGD) recovery. The steam supply mainly contributes to the economics, steam chamber conformance, greenhouse gas emissions, and ultimate bitumen recovery. Therefore, SAGD real-time optimization (RTO) must balance steam chamber development and economics to achieve long-term goals. However, in RTO, general-purpose optimization algorithms decide based on short-term responses, unlike long-term optimization processes. Using economic Key Performance Indicators (KPI) such as Net Present Value (NPV) in a single objective, the RTO determines the smallest amount of steam that results in the highest economic returns. Injecting a small amount of steam reduces steam chamber heat loss, growth, and long-term ultimate bitumen recovery. As a result, a suitable selection of critical KPIs is required to balance the development and economics of the steam chamber.This work presents a study of multiple combinations of SAGD KPIs for a multi-pad real-time steam allocation. The KPI combinations used in this study are NPV, cumulative steam-oil-ratio (cSOR), recovery factor (RF), and average pad temperature (aveTemp). A Box-Jenkins-based data-driven model simulates the SAGD recovery process, and a first principle simulator is used as a substitute for an actual reservoir. The Alternating Direction Method of Multipliers (ADMM) is proposed for multi and many-objective optimization for real-time steam allocation across multiple pads. A compromise programming approach guides the decision-makers choice of the optimum control setpoints of the non-dominated solutions on each horizon. At the end of three and half years of RTO, the optimal KPI combination is ranked based on the aggregated performance of NPV, cSOR, and RF of SAGD multi-pad steam allocation. A synthetic Western Canadian reservoir model with four pads and 33 well pairs developed from publicly available data is used for this study.The aggregated performance of this study's results showed that joint optimization of an economic and a steam chamber growth KPIs of two is better than a combination of three or four. The choice of the joint economic and steam chamber growth KPIs depends on the average pad temperature at the start of RTO to achieve optimal fluid balance in the long term. cSOR or NPV with RF joint KPI performs optimally at average pad temperatures above 90 °C, while cSOR or NPV with aveTemp below 90 °C at the start of RTO to achieve injected and produced fluid balance. Optimum economic return, recovery, and greenhouse gas emission of the SAGD process are achieved with cSOR KPI as the primary economic objective.

Appropriate noise addition to metaheuristic algorithms can enhance their performance

Nature-inspired swarm-based algorithms are increasingly applied to tackle high-dimensional and complex optimization problems across disciplines. They are general purpose optimization algorithms, easy to implement and assumption-free. Some common drawbacks of these algorithms are their premature convergence and the solution found may not be a global optimum. We propose a general, simple and effective strategy, called heterogeneous Perturbation–Projection (HPP), to enhance an algorithm’s exploration capability so that our sufficient convergence conditions are guaranteed to hold and the algorithm converges almost surely to a global optimum. In summary, HPP applies stochastic perturbation on half of the swarm agents and then project all agents onto the set of feasible solutions. We illustrate this approach using three widely used nature-inspired swarm-based optimization algorithms: particle swarm optimization (PSO), bat algorithm (BAT) and Ant Colony Optimization for continuous domains (ACO). Extensive numerical experiments show that the three algorithms with the HPP strategy outperform the original versions with 60–80% the times with significant margins.

A probability model for predicting the transport efficiency in vertical pipes considering the particle size distribution

Improving transport efficiency is one of the core issues in the field of deep-sea mining. Slip velocities have strong effects on the transport efficiency of deep-sea mineral nodules in the hydraulic conveying process. The present work proposes a probability model for predicting the transport velocity and efficiency, which can reflect the influence of different particle size distribution of large particles. By comparing with a numerical simulation, experimental data and empirical models, the proposed model is well verified and demonstrates advantages in accuracy. To acquire further knowledge of the optimal transport efficiency and the predominant factors, such as the volumetric concentration, the pipe diameter and the mixture velocity, effects of various parameters are explored in detail through the present model, including the medium particle diameter, the average dry solid production, the operating power, the particle density, and the fluid density. Furthermore, four general-purpose optimization algorithms, including the global search (GS), the MultiStart (MS), the pattern search (PS) and the genetic algorithm (GA), are combined to obtain the optimal transport efficiency and compared in an application study. The proposed model provides an efficient tool to fast optimize the transport efficiency of mineral nodules in hydraulic conveying pipes. • Probability model to predict velocity considering particle size distribution • Complete procedure to obtain optimal efficiency and corresponding variables • Four general-purpose optimization algorithms compared for hydraulic conveying • Superior prediction accuracy validated by experimental data and empirical models • Efficient tool to fast optimize transport efficiency of conveying mineral nodules

Multi-objective evolutionary algorithms are generally good: Maximizing monotone submodular functions over sequences

Evolutionary algorithms (EAs) are general-purpose optimization algorithms, inspired by natural evolution. Recent theoretical studies have shown that EAs can achieve good approximation guarantees for solving the problem classes of submodular optimization, which have a wide range of applications, such as maximum coverage, sparse regression, influence maximization, document summarization and sensor placement, just to name a few. Though they have provided some theoretical explanation for the general-purpose nature of EAs, the considered submodular objective functions are defined only over sets or multisets. To complement this line of research, this paper studies the problem class of maximizing monotone submodular functions over sequences, where the objective function depends on the order of items. We prove that for each kind of previously studied monotone submodular objective functions over sequences, i.e., prefix monotone submodular functions, weakly monotone and strongly submodular functions, and DAG monotone submodular functions, a simple multi-objective EA, i.e., GSEMO, can always reach or improve the best known approximation guarantee after running polynomial time in expectation. Note that these best-known approximation guarantees can be obtained only by different greedy-style algorithms before. Empirical studies on various applications, e.g., accomplishing tasks, maximizing information gain, search-and-tracking and recommender systems, show the excellent performance of the GSEMO.

A Splitting Scheme for Flip-Free Distortion Energies

We introduce a robust optimization method for flip-free distortion energies used, for example, in parametrization, deformation, and volume correspondence. This method can minimize a variety of distortion energies, such as the symmetric Dirichlet energy and our new symmetric gradient energy. We identify and exploit the special structure of distortion energies to employ an operator splitting technique, leading us to propose a novel Alternating Direction Method of Multipliers (ADMM) algorithm to deal with the non-convex, non-smooth nature of distortion energies. The scheme results in an efficient method where the global step involves a single matrix multiplication and the local steps are closed-form per-triangle/per-tetrahedron expressions that are highly parallelizable. The resulting general-purpose optimization algorithm exhibits robustness to flipped triangles and tetrahedra in initial data as well as during the optimization. We establish the convergence of our proposed algorithm under certain conditions and demonstrate applications to parametrization, deformation, and volume correspondence.

Data assimilation with soft constraints (DASC) through a generalized iterative ensemble smoother

This work investigates an ensemble-based workflow to simultaneously handle generic, nonlinear equality and inequality constraints in reservoir data assimilation problems. The proposed workflow is built upon a recently proposed umbrella algorithm, called the generalized iterative ensemble smoother (GIES), and inherits the benefits of ensemble-based data assimilation algorithms in geoscience applications. Unlike the traditional ensemble assimilation algorithms, the proposed workflow admits cost functions beyond the form of nonlinear-least-squares, and has the potential to develop an infinite number of constrained assimilation algorithms. In the proposed workflow, we treat data assimilation with constraints as a constrained optimization problem. Instead of relying on a general-purpose numerical optimization algorithm to solve the constrained optimization problem, we derive an (approximate) closed form to iteratively update model variables, but without the need to explicitly linearize the constraint systems. The established model update formula bears similarities to that of an iterative ensemble smoother (IES). Therefore, in terms of theoretical analysis, it becomes relatively easy to transit from an ordinary IES to the proposed constrained assimilation algorithms, and in terms of practical implementation, it is also relatively straightforward to implement the proposed workflow for users who are familiar with the IES, or other conventional ensemble data assimilation algorithms like the ensemble Kalman filter (EnKF). Apart from the aforementioned features, we also develop efficient methods to handle two noticed issues that would be of practical importance for ensemble-based constrained assimilation algorithms. These issues include localization in the presence of constraints, and the (possible) high dimensionality induced by the constraint systems. We use one 2D and one 3D case studies to demonstrate the performance of the proposed workflow. In particular, the 3D example contains experiment settings close to those of real field case studies. In both case studies, the proposed workflow achieves better data assimilation performance in comparison to the choice of using an original IES algorithm. As such, the proposed workflow has the potential to further improve the efficacy of ensemble-based data assimilation in practical reservoir data assimilation problems.

On the comparison of optimization algorithms for the random-field Potts model

For many systems with quenched disorder the study of ground states can crucially contribute to a thorough understanding of the physics at play, be it for the critical behavior if that is governed by a zero-temperature fixed point or for uncovering properties of the ordered phase. While ground states can in principle be computed using general-purpose optimization algorithms such as simulated annealing or genetic algorithms, it is often much more efficient to use exact or approximate techniques specifically tailored to the problem at hand. For certain systems with discrete degrees of freedom such as the random-field Ising model, there are polynomial-time methods to compute exact ground states. But even as the number of states increases beyond two as in the random-field Potts model, the problem becomes NP hard and one cannot hope to find exact ground states for relevant system sizes. Here, we compare a number of approximate techniques for this problem and evaluate their performance.

Multiobjective Evolutionary Algorithms Are Still Good: Maximizing Monotone Approximately Submodular Minus Modular Functions.

As evolutionary algorithms (EAs) are general-purpose optimization algorithms, recent theoretical studies have tried to analyze their performance for solving general problem classes, with the goal of providing a general theoretical explanation of the behavior of EAs. Particularly, a simple multiobjective EA, that is, GSEMO, has been shown to be able to achieve good polynomial-time approximation guarantees for submodular optimization, where the objective function is only required to satisfy some properties and its explicit formulation is not needed. Submodular optimization has wide applications in diverse areas, and previous studies have considered the cases where the objective functions are monotone submodular, monotone non-submodular, or non-monotone submodular. To complement this line of research, this article studies the problem class of maximizing monotone approximately submodular minus modular functions (i.e., g-c) with a size constraint, where g is a so-called non-negative monotone approximately submodular function and c is a so-called non-negative modular function, resulting in the objective function (g-c) being non-monotone non-submodular in general. Different from previous analyses, we prove that by optimizing the original objective function (g-c) and the size simultaneously, the GSEMO fails to achieve a good polynomial-time approximation guarantee. However, we also prove that by optimizing a distorted objective function and the size simultaneously, the GSEMO can still achieve the best-known polynomial-time approximation guarantee. Empirical studies on the applications of Bayesian experimental design and directed vertex cover show the excellent performance of the GSEMO.

A comparison of general-purpose optimization algorithms for finding optimal approximate experimental designs

Several common general purpose optimization algorithms are compared for finding A- and D-optimal designs for different types of statistical models of varying complexity, including high dimensional models with five and more factors. The algorithms of interest include exact methods, such as the interior point method, the Nelder–Mead method, the active set method, the sequential quadratic programming, and metaheuristic algorithms, such as particle swarm optimization, simulated annealing and genetic algorithms. Several simulations are performed, which provide general recommendations on the utility and performance of each method, including hybridized versions of metaheuristic algorithms for finding optimal experimental designs. A key result is that general-purpose optimization algorithms, both exact methods and metaheuristic algorithms, perform well for finding optimal approximate experimental designs.

Maximizing submodular or monotone approximately submodular functions by multi-objective evolutionary algorithms

Evolutionary algorithms (EAs) are a kind of nature-inspired general-purpose optimization algorithm, and have shown empirically good performance in solving various real-word optimization problems. During the past two decades, promising results on the running time analysis (one essential theoretical aspect) of EAs have been obtained, while most of them focused on isolated combinatorial optimization problems, which do not reflect the general-purpose nature of EAs. To provide a general theoretical explanation of the behavior of EAs, it is desirable to study their performance on general classes of combinatorial optimization problems. To the best of our knowledge, the only result towards this direction is the provably good approximation guarantees of EAs for the problem class of maximizing monotone submodular functions with matroid constraints. The aim of this work is to contribute to this line of research. Considering that many combinatorial optimization problems involve non-monotone or non-submodular objective functions, we study the general problem classes, maximizing submodular functions with/without a size constraint and maximizing monotone approximately submodular functions with a size constraint. We prove that a simple multi-objective EA called GSEMO-C can generally achieve good approximation guarantees in polynomial expected running time.

Efficiency optimization of a three-coil resonant energy link

AbstractThis paper presents an effective and time saving procedure for designing a three-coil resonant inductive wireless power transfer (WPT) link. The proposed approach aims at optimizing the power transfer efficiency of the link for given constraints imposed by the specific application of interest. The WPT link is described as a two-port network with equivalent lumped elements analytically expressed as function of the geometrical parameters. This allows obtaining a closed-form expression of the efficiency that can be maximized by acting on the geometrical parameters of the link by using a general purpose optimization algorithm. The proposed design procedure allows rapidly finding the desired optimal solution while minimizing the computational efforts. Referring to the case of an application constraining the dimensions of the receiver, analytical data are validated through full-wave simulations and measurements.

Synthesis of Coupling Matrix for Diplexers Based on a Self-Adaptive Differential Evolution Algorithm

Diplexer coupling matrix synthesis often involves both analytical methods and optimization techniques. At present, general purpose optimization algorithms are used, but they need strong supporting information (e.g., high-quality starting points and very narrow search ranges) from analytical methods, which is not available or too complex to be obtained in many cases. Aiming to obtain the desired coupling matrix with highly reduced supporting information to relieve the pressure of analytical methods, a new optimization algorithm, called self-adaptive differential evolution for coupling matrix synthesis (SADEC), is proposed. Considering the landscape characteristics of diplexer coupling matrix synthesis problems, a new self-adaptive multipopulation search framework and a self-adaptive algorithm parameter control strategy are proposed and organized in a particular way. The performance of SADEC is demonstrated by two all-resonator-based narrowband diplexers using large search ranges only with the requirement of matching the diplexer topology and no ad hoc analysis is included. Experiments and comparisons show the high performance of SADEC and clear advantages compared with the state-of-the-art global optimization methods. SADEC is also applicable to filter coupling matrix synthesis and is downloadable.

Real-time contact force distribution using a polytope hierarchy in the grasp wrench set

Abstract This paper presents an efficient approach to contact force distribution, which is aimed at computing optimal contact forces to generate the required wrench for grasping an object. It has been derived in the previous work that this problem can be reduced to computing the intersection of the ray originating from the origin along the required wrench with the boundary of the grasp wrench set. Noticing that the grasp wrench set is fixed once contact positions are determined, we propose an algorithm to pre-compute a hierarchy of polytopes in the grasp wrench set and a list of facets from the interior of the grasp wrench set to its boundary. Then, the ray’s intersection can be quickly found by searching the list of facets and optimal contact forces can be computed in real time. Numerical examples show that the online computation of the proposed approach is one order of magnitude faster than the latest algorithm to compute the ray’s intersection and two orders of magnitude faster than general-purpose optimization algorithms. This approach to contact force distribution is an iterative solution that can run until reaching a desired accuracy.

Suppression of Multiphoton Resonances in Driven Quantum Systems via Pulse Shape Optimization.

This Letter demonstrates control over multiphoton absorption processes in driven two-level systems, which include, for example, superconducting qubits or laser-irradiated graphene, through spectral shaping of the driving pulse. Starting from calculations based on Floquet theory, we use differential evolution, a general purpose optimization algorithm, to find the Fourier coefficients of the driving function that suppress a given multiphoton resonance in the strong field regime. We show that the suppression of the transition probability is due to the coherent superposition of high-order Fourier harmonics which closes the dynamical gap between the Floquet states of the two-level system.

ADMM ⊇ Projective Dynamics: Fast Simulation of Hyperelastic Models with Dynamic Constraints.

We apply the alternating direction method of multipliers (ADMM) optimization algorithm to implicit time integration of elastic bodies, and show that the resulting method closely relates to the recently proposed projective dynamics algorithm. However, as ADMM is a general purpose optimization algorithm applicable to a broad range of objective functions, it permits the use of nonlinear constitutive models and hard constraints while retaining the speed, parallelizability, and robustness of projective dynamics. We further extend the algorithm to improve the handling of dynamically changing constraints such as sliding and contact, while maintaining the benefits of a constant, prefactored system matrix. We demonstrate the benefits of our algorithm on several examples that include cloth, collisions, and volumetric deformable bodies with nonlinear elasticity and skin sliding effects.

Examination of Three Different ACOPF Formulations With Generator Capability Curves

This paper examines three different formulations of AC optimal power flow problem and compares performance of well-established general-purpose optimization algorithms for each, over different initial conditions. Polar power–voltage, rectangular power–voltage, and rectangular current–voltage are formulated to evaluate ACOPF solution characteristic. The formulation here maintains line flows as explicit variables and employs summations of these quantities to impose conservation conditions at each node. Two representations of line thermal limits are considered, one using real power (to allow comparisons to dc power flow approximations), and a more physically based ampacity limit using current magnitude. Nonlinear generator capability curves are represented (“D-curves”), including options to allow active and reactive limits dependent on generator voltage. A uniform objective function is used throughout, that of minimizing quadratic generator operating cost curves. Numerical performance case studies are performed for these formulations over six different classes of initial conditions, evaluating computational time and also robustness of convergence.

A parallel hybrid optimization algorithm for fitting interatomic potentials

In this work we present the parallel implementation of a hybrid global optimization algorithm assembled specifically to tackle a class of time consuming interatomic potential fitting problems. The resulting objective function is characterized by large and varying execution times, discontinuity and lack of derivative information. The presented global optimization algorithm corresponds to an irregular, two-level execution task graph where tasks are spawned dynamically. We use the OpenMP tasking model to express the inherent parallelism of the algorithm on shared-memory systems and a runtime library which implements the execution environment for adaptive task-based parallelism on multicore clusters. We describe in detail the hybrid global optimization algorithm and various parallel implementation issues. The proposed methodology is then applied to a specific instance of the interatomic potential fitting problem for the metal titanium. Extensive numerical experiments indicate that the proposed algorithm achieves the best parallel performance. In addition, its serial implementation performs well and therefore can also be used as a general purpose optimization algorithm.

High resolution range-reflectivity estimation of radar targets via compressive sampling and Memetic Algorithm

Recent results of Compressive Sampling (CS) have demonstrated its feasibility in high-resolution radar targets estimation and imaging [2,10,14,15,17,19,23,29,30,32–34]. However, the signal recovery is reduced to seeking a sparse solution to an underdetermined linear system of equations. It is potentially very difficult because even finding a solution that approximates the true minimum is NP-hard. In this paper, we introduce Memetic Algorithm (MA) to solve this non-convex l0-norm minimization problem, and design a compressive receiver for high-resolution range-reflectivity estimation of multiple radar targets. A double-population MA is proposed, where the position population is used to evaluate the ranges, and the coefficient population is used to realize a local search of target reflectivities. By combining the global search with a local searching operation to exploit the available knowledge in the recovery, the proposed MA outperforms the general purpose optimization algorithms in terms of the quality of solution. Some experiments are taken to investigate the performance of this compressive receiver at different sampling rates, and the results show the superiority to its counterparts in both noiseless environment and noisy, cluttered environment.