Gradient-free Optimization Algorithms Research Articles

Intensification of a microbial electrolysis cell for biohydrogen production

Hydrogen can be the fuel of the future, and there are multiple approaches for its production and storage. Most hydrogen production comes from non-renewable sources, primarily natural gas reforming, which is efficient but unsustainable. Microbial electrolysis cells (MEC) are alternative developing technologies that exploit conductor-like properties of electrogenic bacteria to both oxidize organic contaminants and get protons reduced in a cathode, thus producing hydrogen. Here, the aim is to design a model-based optimization framework to intensify the MEC efficiency. The constructed process model is proposed by considering transport phenomena, microbial kinetics, and a secondary current distribution model to estimate steady-state reactor performance and electrical efficiency. The results show a value of 25 % for electrical efficiency as hydrogen recovered. Global sensitivity analysis was performed and determined a strong dependency on inlet liquid velocity, and a built-in, gradient-free optimization algorithm was implemented to the model to maximize electrical efficiency under a defined range of inlet liquid velocity, inlet substrate concentration, and applied voltage. The proposed model predicts a maximum increase of efficiency of about 267 % by increasing inlet velocity and substrate concentration. Intensification of the cell's performance can be greatly achieved when halving hydraulic retention time and by having a more concentrated feed.

Gray code based gradient-free optimization algorithm for parameterized quantum circuit

A Gray code based gradient-free optimization (GCO) algorithm is proposed to update the parameters of parameterized quantum circuits (PQCs) in this work. Each parameter of PQCs is encoded as a binary string, named as a gene, and a genetic-based method is adopted to select the offsprings. The individuals in the offspring are decoded in Gray code way to keep Hamming distance, and then are evaluated to obtain the best one with the lowest cost value in each iteration. The algorithm is performed iteratively for all parameters one by one until the cost value satisfies the stop condition or the number of iterations is reached. The GCO algorithm is demonstrated for classification tasks in Iris and MNIST datasets, and their performance are compared by those with the Bayesian optimization algorithm and binary code based optimization algorithm. The simulation results show that the GCO algorithm can reach high accuracies steadily for quantum classification tasks. Importantly, the GCO algorithm has a robust performance in the noise environment.

Comparative study on gradient-free optimization methods for inverse source-term estimation of radioactive dispersion from nuclear accidents

In this study, we rigorously assess the performance of three gradient-free optimization algorithms—Ensemble Kalman Inversion (EKI), Particle Swarm Optimization (PSO), and Genetic Algorithm (GA)—for estimating source terms in diverse radionuclide release scenarios. Our analysis encompasses both single and multiple sources with varying radionuclide compositions, delving into the influence of decay constants and radioactivity on source estimation accuracy. Although estimating a single radionuclide from a single source exhibits outstanding results, estimating multiple radionuclides from a single source proves more arduous due to the limited information available for discerning gamma dose rates. Contrary to expectations, increasing the number of observation stations does not consistently improve the likelihood of finding accurate solutions in ill-posed inverse problems. Impressively, under our simulation settings, EKI demonstrates competitive performance in terms of convergence, accuracy, and runtime compared to PSO and GA, with GPU parallelization further bolstering computational efficiency. We explore strategies for enhancing source term estimation, including incorporating prior information, applying uncertainty removal techniques, and optimizing observation placement. Additionally, this study underscores the intricate role of relative error in determining multi-radionuclide estimation accuracy from gamma dose measurements. By employing the Gaussian plume model under steady-state conditions, our research lays the groundwork for future applications of Lagrangian dispersion models with real-time data integration. The insights gleaned from our study promise to advance environmental radioactivity monitoring and catalyze the development of cutting-edge, real-time source estimation technologies in full-scale systems.

Geometric Constellation Shaping for Fiber-Optic Channels via End-to-End Learning

End-to-end learning has become a popular method to optimize a constellation shape of a communication system. When the channel model is differentiable, end-to-end learning can be applied with conventional backpropagation algorithm for optimization of the shape. A variety of optimization algorithms have also been developed for end-to-end learning over a non-differentiable channel model. In this paper, we compare gradient-free optimization method based on the cubature Kalman filter, model-free optimization and backpropagation for end-to-end learning on a fiber-optic channel modeled by the split-step Fourier method. The results indicate that the gradient-free optimization algorithms provide a decent replacement to backpropagation in terms of performance at the expense of computational complexity. Furthermore, the quantization problem of finite bit resolution of the digital-to-analog and analog-to-digital converters is addressed and its impact on geometrically shaped constellations is analysed. Here, the results show that when optimizing a constellation with respect to mutual information, a minimum number of quantization levels is required to achieve shaping gain. For generalized mutual information, the gain is maintained throughout all of the considered quantization levels. Also, the results implied that the autoencoder can adapt the constellation size to the given channel conditions.

Inverse design of dispersion for photonic devices based on LSTM and gradient-free optimization algorithms hybridization

Inverse design of dispersion for photonic devices based on LSTM and gradient-free optimization algorithms hybridization

Prestress and size optimization of double-curvature cable domes using an incremental-prestressing iterative technique

A major challenge in designing new forms of cable domes is to keep balance between structural weight and stiffness. Increasing prestress level should decrease the maximum displacement and this, in turn, may lead to heavier structures and higher costs. Accordingly, the required cross sections and prestress level that achieve stability and strength constraints vary whenever this weight-stiffness balance changes, typically as moving along a Pareto front that represents the solution of a two-objective optimization of weight and displacement. Apart from complicated derivation of the gradient of optimization function or using time-consuming gradient-free optimization algorithms, this study proposes a time-efficient iterative technique aims at developing a trade-off curve between the optimized weight and corresponding displacement of cable domes that simulates the Pareto front solution of two-objective optimization. This curve provides insights into the extent to which the optimized weight would increase when decreasing the maximum displacement, giving the designer a decision support tool to select the most preferred solution. The proposed technique is based on prestressing the dome with an appropriate initial level of prestress, then increasing the level of prestress incrementally until reaching the target stiffness. By each increment of prestress level, the dome is designed to the minimum weight that achieves the stability and strength constraints through sub-iterations. The variables of this incremental-prestressing (IP) iterative technique are prestress level and size of elements with constraints on stability of struts and strength of cables. The resulting pairs of optimized weight and corresponding displacement at all increments of prestress level are, then, fitted to a power function formulating the trade-off curve. The proposed IP technique is used to design a simple illustrative example as well as a realistic double-curvature cable dome. The efficiency of the proposed technique is verified by comparing the resulting trade-off curve of cable dome example to the Pareto optimal solutions obtained by a two-objective optimization of weight and displacement using NSGA-II algorithm. Different geometrical parameters are considered to highlight the time-efficiency of the IP technique for further usage as a robust tool in large parametric studies in which surrogate functions are developed to predict the optimized weight and prestress level of cable domes at specific target stiffness.

Automated Discovery and Optimization of 3D Topological Photonic Crystals

Topological photonic crystals have received considerable attention for their ability to manipulate and guide light in unique ways. They are typically designed by hand based on the careful analysis of their bands and mode profiles, but recent theoretical advances have revealed new and powerful insights into the connection between band symmetry, connectivity, and topology. Here we propose a combined global and local optimization framework that integrates a flexible symmetry-constrained level-set parametrization with standard gradient-free optimization algorithms to optimize topological photonic crystals, a problem setting where the objective function may be highly nonconvex and noncontinuous. Our framework can be applied to any symmetry-identifiable band topology, and we demonstrate its applicability to several prominent kinds of three-dimensional band topologies, namely, Γ-enforced nodal lines, Weyl points, and Chern insulators. Requiring no prior examples of topological photonic crystals or prior knowledge on the connection between structure and band topology, our approach indicates a path toward the automated discovery of novel topological photonic crystal designs.

Single-qubit quantum classifier based on gradient-free optimization algorithm

A single-qubit quantum classifier (SQC) based on a gradient-free optimization (GFO) algorithm, named the GFO-based SQC, is proposed to overcome the effects of barren plateaus caused by quantum devices. Here, a rotation gate RX (ϕ) is applied on the single-qubit binary quantum classifier, and the training data and parameters are loaded into ϕ in the form of vector multiplication. The cost function is decreased by finding the value of each parameter that yields the minimum expectation value of measuring the quantum circuit. The algorithm is performed iteratively for all parameters one by one until the cost function satisfies the stop condition. The proposed GFO-based SQC is demonstrated for classification tasks in Iris and MNIST datasets and compared with the Adam-based SQC and the quantum support vector machine (QSVM). Furthermore, the performance of the GFO-based SQC is discussed when the rotation gate in the quantum device is under different types of noise. The simulation results show that the GFO-based SQC can reach a high accuracy in reduced time. Additionally, the proposed GFO algorithm can quickly complete the training process of the SQC. Importantly, the GFO-based SQC has a good performance in noisy environments.

Quasi-Gradient Nonlinear Simplex Optimization Method in Electromagnetics

Particle swarm optimization (PSO), genetic algorithm (GA), and nonlinear simplex optimization method (SOM) are some of the most prominent gradient-free optimization algorithms in engineering. When it comes to a common group of electromagnetic optimization problems wherein less than 10 optimization parameters are present in the problem domain, SOM features faster convergence rate vs PSO and GA. Nevertheless, PSO and GA still outperform SOM by having more accuracy in finding the global minimum. To improve the accuracy of SOM in problems with few optimization parameters, a quasi-gradient (Q-G) search direction is added to the conventional algorithm. An extra decision is made by the proposed algorithm to move alongside the reflection or quasi-gradient direction during the error-reduction operations. This modification will improve the accuracy of SOM, which otherwise fails in the examples presented in this article, to levels similar to PSO and GA, while retaining approximately 33% faster convergence speed with relatively small number of parameters, and 20% faster convergence speed with larger number of optimization parameters. Following a standard benchmark test verification, the proposed algorithm successfully solves a suite of electromagnetic optimization problems. Representative examples include the optimization of absorber dimensions in an anechoic chamber, and estimation of the properties of an unknown embedded object by scattered microwave signals.

Adaptive search space decomposition method for pre- and post-buckling analyses of space truss structures

The paper proposes a novel adaptive search space decomposition method and a novel gradient-free optimization-based formulation for the pre- and post-buckling analyses of space truss structures. Space trusses are often employed in structural engineering to build large steel constructions, such as bridges and domes, whose structural response is characterized by large displacements. Therefore, these structures are vulnerable to progressive collapses due to local or global buckling effects, leading to sudden failures. The method proposed in this paper allows the analysis of the load-equilibrium path of truss structures to permanent and variable loading, including stable and unstable equilibrium stages and explicitly considering geometric nonlinearities. The goal of this work is to determine these equilibrium stages via optimization of the Lagrangian kinematic parameters of the system, determining the global equilibrium. However, this optimization problem is non-trivial due to the undefined parameter domain and the sensitivity and interaction among the Lagrangian parameters. Therefore, we propose to formulate this problem as a nonlinear, multimodal, unconstrained, continuous optimization problem and develop a novel adaptive search space decomposition method, which progressively and adaptively re-defines the search domain (hypersphere) to evaluate the equilibrium of the system using a gradient-free optimization algorithm. We tackle three benchmark problems and evaluate a medium-sized test representing a real structural problem in this paper. The results are compared to those available in the literature regarding displacement–load curves and deformed configurations. The accuracy and robustness of the adopted methodology show a high potential for gradient-free algorithms to analyze space truss structures.

A framework for simultaneous design of wind turbines and cable layout in offshore wind

Abstract. An optimization framework for simultaneous design of wind turbines (WTs) and cable layout for a collection system of offshore wind farms (OWFs) is presented in this paper. The typical approach used in both research and practical design is sequential, with an initial annual energy production (AEP) maximization, followed then by the collection system design. The sequential approach is robust and effective. However it fails to exploit the synergies between optimization blocks. Intuitively, one of the strongest trade-offs is between the WTs and cable layout, as they generally compete; i.e. spreading out WTs mitigates wake losses for larger AEP but also results in longer submarine cables in the collection system and higher costs. The proposed optimization framework implements a gradient-free optimization algorithm to smartly move the WTs within the project area subject to minimum distance constraint, while a fast heuristic algorithm is called in every function evaluation in order to calculate a cost estimation of the cable layout. In a final stage, a refined cable layout design is obtained by iteratively solving a mixed integer linear programme (MILP), modelling all typical engineering constraints of this particular problem. A comprehensive performance analysis of the cost estimation from the fast heuristic algorithm with respect to the exact model is carried out. The applicability of the method is illustrated through a large-scale real-world case study. Results shows that (i) the quality of the cable layout estimation is strongly dependent on the separation between WTs, where dense WT layouts present better performance parameters in terms of error, correlation, and computing time, and (ii) the proposed simultaneous design approach provides up to 6 % of improvement on the quality of fully feasible wind farm designs, and broadly, a statistically significant enhancement is ensured in spite of the stochasticity of the optimization algorithm.

Post-Optimum Sensitivity Analysis with Automatically Tuned Numerical Gradients Applied to Swept Wind Turbine Blades

Post-Optimum Sensitivity Analysis (POSA) extends numerical design optimization to provide additional information on how the design and performance would change if various parameters and constraints were varied. POSA is challenging since it typically requires accurate gradients and gradient-based optimization problems that provide Lagrange multipliers. To overcome this problem, this paper introduces a technique to automatically tune gradients with statistical methods and algorithms to calculate the Lagrange multipliers after an optimization. This allows these methods to be applied to problems with noisy gradients or problems solved with gradient-free optimization algorithms. These methods have been applied to swept wind turbine blades. Swept blades can reduce wind turbine loads by twisting out of the wind when the wind speed increases. The methods have shown that introducing design freedom in the sweep, blade root flap-wise bending moments and blade tip deflection has a weaker influence on the design. Instead, blade root torsion moment and material failure become the driving constraints.

Metamaterials to control both longitudinal and transverse waves simultaneously

Mechanical metamaterials are used in various applications to control bulk wave propagation through elastic materials. Controlling both the longitudinal and transverse waves at the same time by designing a single phononic device is challenging through theoretical, computational, and experimental perspectives. Suppression of system generated second harmonics due to interaction of monochromatic longitudinal wave and during one-way two-wave mixing of two longitudinal waves by designing effective phononic material is demonstrated in previous work by the author [Ghodake, J. Acoust. Soc. Am. 150, A149 (2021)]. In this talk, the design of linear periodic metamaterials which can control both the longitudinal and transverse waves, their second harmonics during propagation of monochromatic waves, and one-way two-wave mixing between longitudinal and transverse waves, is discussed by solving inverse problems using finite element analysis, effective boundary conditions, and optimization techniques. Along with the widths of periodic elastic materials, the number of repeated periodic cells (N) in a phononic lattice is defined as a design parameter. Different strong and weak contains are implemented so that in every iteration either during or after the iteration N will remain integer. A gradient-free optimization algorithm is used during constrained optimization. Strong constraint solves the inverse problem in a few minutes.

Developing a Lung Model in the Age of COVID-19: A Digital Image Correlation and Inverse Finite Element Analysis Framework.

Pulmonary diseases, driven by pollution, industrial farming, vaping, and the infamous COVID-19 pandemic, lead morbidity and mortality rates worldwide. Computational biomechanical models can enhance predictive capabilities to understand fundamental lung physiology; however, such investigations are hindered by the lung’s complex and hierarchical structure, and the lack of mechanical experiments linking the load-bearing organ-level response to local behaviors. In this study we address these impedances by introducing a novel reduced-order surface model of the lung, combining the response of the intricate bronchial network, parenchymal tissue, and visceral pleura. The inverse finite element analysis (IFEA) framework is developed using 3-D digital image correlation (DIC) from experimentally measured non-contact strains and displacements from an ex-vivo porcine lung specimen for the first time. A custom-designed inflation device is employed to uniquely correlate the multiscale classical pressure-volume bulk breathing measures to local-level deformation topologies and principal expansion directions. Optimal material parameters are found by minimizing the error between experimental and simulation-based lung surface displacement values, using both classes of gradient-based and gradient-free optimization algorithms and by developing an adjoint formulation for efficiency. The heterogeneous and anisotropic characteristics of pulmonary breathing are represented using various hyperelastic continuum formulations to divulge compound material parameters and evaluate the best performing model. While accounting for tissue anisotropy with fibers assumed along medial-lateral direction did not benefit model calibration, allowing for regional material heterogeneity enabled accurate reconstruction of lung deformations when compared to the homogeneous model. The proof-of-concept framework established here can be readily applied to investigate the impact of assorted organ-level ventilation strategies on local pulmonary force and strain distributions, and to further explore how diseased states may alter the load-bearing material behavior of the lung. In the age of a respiratory pandemic, advancing our understanding of lung biomechanics is more pressing than ever before.

Covariance matrix adaptation evolution strategy for topology optimization of foundations under static and dynamic loadings

This paper studies the topology optimization for jet-grout distribution as a structural material beneath a foundation under static and dynamic loadings. The material distribution is optimized in a manner to minimize the vertical displacement, beneath the center of the foundation, under static loading and vertical velocity under dynamic loading. For this purpose, the efficient gradient-free optimization algorithm called Covariance Matrix Adaption Evolution Strategy (CMA-ES) is used and coupled with finite element method. The important parameters that affect the optimal jet-grout layouts such as loading frequency, soil type, and foundation rigidity are explored. The results show that the jet-grout mainly tends to concentrate near the edges of the foundation. It is also observed that variations in the dynamic loading frequency, foundation rigidity, soil stiffness, and soil-foundation interaction change the layout of the structural material and mitigation capacities (MC). Furthermore, it is found that topology optimization is more operational in increasing MC value in the soft soil and type of soil-foundation interaction does not affect the ultimate MC value. The results show that as loading frequency increases, jet-grout moves to the left and bifurcates at the bottom. Moreover, flexible foundations have lower MC values in both static and dynamic loading cases.

Natural evolutionary strategies for variational quantum computation

Natural evolutionary strategies (NES) are a family of gradient-free black-box optimization algorithms. This study illustrates their use for the optimization of randomly initialized parameterized quantum circuits (PQCs) in the region of vanishing gradients. We show that using the NES gradient estimator the exponential decrease in variance can be alleviated. We implement two specific approaches, the exponential and separable NES, for parameter optimization of PQCs and compare them against standard gradient descent. We apply them to two different problems of ground state energy estimation using variational quantum eigensolver and state preparation with circuits of varying depth and length. We also introduce batch optimization for circuits with larger depth to extend the use of ES to a larger number of parameters. We achieve accuracy comparable to state-of-the-art optimization techniques in all the above cases with a lower number of circuit evaluations. Our empirical results indicate that one can use NES as a hybrid tool in tandem with other gradient-based methods for optimization of deep quantum circuits in regions with vanishing gradients.

A Framework to Optimize Electrode Morphology

Since the invention of lithium-ion batteries, porous electrodes have been manufactured in essentially the same way: granulating active materials into powder, mixing them with some additives such as carbon black and binder, and pasting them on conductive substrate. Optimizations have been focusing on tuning parameters such as particle sizes, porosity and electrode thickness, with the assumption of a uniform electrode. Some recent work demonstrated better cell performance by engineering the shape of the electrode with features such as nano cylinders, tunnels and fractal branches. 3D printing has been used to fabricate the structure of batter electrode. On the simulation side, however, there have been few reports on freeform electrode shape optimization. Most shape optimizations assume a particular electrode pattern with a few adjustable dimensional parameters. Here we report a method to optimize porosity distribution of electrode without a prior assumption of the pattern. An NMC 333 electrode with lithium metal as the counter electrode is considered as an example. To find the optimal solid material distribution for maximum specific energy, we represented the NMC material distribution by discrete nodal design variables. To solve this high dimensional optimization problem, we used a gradient-free optimization algorithm, self-directed online learning optimization. We show that the method can produce an electrode pattern with the specific energy more than 15% higher than that of the uniform electrode. This generic distribution optimization approach provides a powerful tool for the design and optimization of porous electrodes. Figure 1

Robust implementation of generative modeling with parametrized quantum circuits

Although the performance of hybrid quantum-classical algorithms is highly dependent on the selection of the classical optimizer and the circuit ansätze (Benedetti et al, npj Quantum Inf 5:45, 2019; Hamilton et al, 2018; Zhu et al, 2018), a robust and thorough assessment on-hardware of such features has been missing to date. From the optimizer perspective, the primary challenge lies in the solver’s stochastic nature, and their significant variance over the random initialization. Therefore, a robust comparison requires one to perform several training curves for each solver before one can reach conclusions about their typical performance. Since each of the training curves requires the execution of thousands of quantum circuits in the quantum computer, such a robust study remained a steep challenge for most hybrid platforms available today. Here, we leverage on Rigetti’s Quantum Cloud Services (QCS™) to overcome this implementation barrier, and we study the on-hardware performance of the data-driven quantum circuit learning (DDQCL) for three different state-of-the-art classical solvers, and on two-different circuit ansätze associated to different entangling connectivity graphs for the same task. Additionally, we assess the gains in performance from varying circuit depths. To evaluate the typical performance associated with each of these settings in this benchmark study, we use at least five independent runs of DDQCL towards the generation of quantum generative models capable of capturing the patterns of the canonical Bars and Stripes dataset. In this experimental benchmarking, the gradient-free optimization algorithms show an outstanding performance compared to the gradient-based solver. In particular, one of them had better performance when handling the unavoidable noisy objective function to be minimized under experimental conditions.

Topological design of microstructures using periodic material-field series-expansion and gradient-free optimization algorithm

Light-weight cellular materials with periodic repetitive microstructures are widely used in various fields due to their superior mechanical/multi-physical performances. As the microstructural design problem is known to have multiple local minima, most gradient-based topology optimization methods significantly depend on the initial guess of the microstructural geometry, thus requiring the designer’s experiences. This paper presents an effective gradient-free framework for periodic microstructure design, which exhibits powerful global searching capabilities and requires no sensitivity information. The proposed framework combines the material-field series-expansion (MFSE) topology representation of periodic microstructures and the sequential Kriging-based optimization algorithm. The MFSE method decouples the topological representation from the finite element discretization, and describes a relatively complex microstructural topology with high-quality boundary description using a greatly reduced number of design variables. Based on the Kriging surrogate model, a solution scheme is suggested to solve material microstructural topology optimization successively in a sequence of sub-optimization problems with self-adaptive design spaces. With the present gradient-free optimization method, high-performance cellular materials that approach the H-S upper bound with porosities from 0.2 to 0.6, or that achieve negative Poisson’s ratios of -0.94 in the principal directions for materials with square symmetry, are obtained without prior knowledge of the optimum microstructural topology.

Sequential minimal optimization for quantum-classical hybrid algorithms

We propose a sequential minimal optimization method for quantum-classical hybrid algorithms, which converges faster, is robust against statistical error, and is hyperparameter-free. Specifically, the optimization problem of the parameterized quantum circuits is divided into solvable subproblems by considering only a subset of the parameters. In fact, if we choose a single parameter, the cost function becomes a simple sine curve with period $2\pi$, and hence we can exactly minimize with respect to the chosen parameter. Furthermore, even in general cases, the cost function is given by a simple sum of trigonometric functions with certain periods and hence can be minimized by using a classical computer. By repeatedly performing this procedure, we can optimize the parameterized quantum circuits so that the cost function becomes as small as possible. We perform numerical simulations and compare the proposed method with existing gradient-free and gradient-based optimization algorithms. We find that the proposed method substantially outperforms the existing optimization algorithms and converges to a solution almost independent of the initial choice of the parameters. This accelerates almost all quantum-classical hybrid algorithms readily and would be a key tool for harnessing near-term quantum devices.