Good Initial Guess Research Articles

Solving Systems of Quadratic Equations via Exponential-Type Gradient Descent Algorithm

We consider the rank minimization problem from quadratic measurements, i.e., recovering a rank $r$ matrix $X \in \mathbb{R}^{n \times r}$ from $m$ scalar measurements $y_i=a_i^{\top} XX^{\top} a_i,\;a_i\in \mathbb{R}^n,\;i=1,\ldots,m$. Such problem arises in a variety of applications such as quadratic regression and quantum state tomography. We present a novel algorithm, which is termed exponential-type gradient descent algorithm, to minimize a non-convex objective function $f(U)=\frac{1}{4m}\sum_{i=1}^m(y_i-a_i^{\top} UU^{\top} a_i)^2$. This algorithm starts with a careful initialization, and then refines this initial guess by iteratively applying exponential-type gradient descent. Particularly, we can obtain a good initial guess of $X$ as long as the number of Gaussian random measurements is $O(nr)$, and our iteration algorithm can converge linearly to the true $X$ (up to an orthogonal matrix) with $m=O\left(nr\log (cr)\right)$ Gaussian random measurements.

Active-Set based Inexact Interior Point QP Solver for Model Predictive Control

Interior point methods are applicable to a large class of problems and can be very reliable for convex optimization, even without a good initial guess for the optimal solution. Active-set methods, on the other hand, are often restricted to linear or quadratic programming but they have a lower computational cost per iteration and superior warm starting properties. The present paper proposes an approach for improving the numerical conditioning and warm starting properties of interior point methods, based on an active-set identification strategy and inexact Newton-type optimization techniques. In addition, we show how this reduces the average computational cost of the linear algebra operations in each interior point iteration. We developed an efficient C code implementation of the active-set based interior point method (ASIPM) and show that it can be competitive with state of the art solvers for a standard case study of model predictive control stabilizing an inverted pendulum on a cart.

Bifidelity Gradient-Based Approach for Nonlinear Well-Logging Inverse Problems

Solving a nonlinear inverse problem is challenging in computational science and engineering. Sampling-based methods require a large number of model evaluations; gradient-based methods require fewer model evaluations but only find the local minima. Multifidelity optimization combines the low-fidelity model and the high-fidelity model to achieve both high accuracy and high efficiency. In this article, we present a bifidelity approach to solve nonlinear inverse problems. In the bifidelity inversion method, the low-fidelity model is used to acquire a good initial guess, and the high-fidelity model is used to locate the global minimum. Combined with a multistart optimization scheme, the proposed approach significantly increases the possibility of finding the global minimum for nonlinear inverse problems with many local minima. The method is tested with two toy problems and then applied to an electromagnetic well-logging inverse problem, which is difficult to solve using traditional gradient-based methods. The bifidelity method provides promising inversion results and can be easily applied to traditional gradient-based methods.

Grid Search Based Tire-Road Friction Estimation

The tire-road friction coefficient (μ max ) is an important input for vehicle dynamics control system and automated driving modules. However, reliable and accurate measurement of this parameter is difficult and costly in mass-produced vehicles and thus estimation is necessary. In this research, an innovative optimization based framework to estimate μ max is proposed. The observation problem is formulated as a non-convex optimization. A novelty of the framework is that the μmax can be accurately estimated in real time together with side slip angle as a by-product without requiring a good initial guess for the non-convex optimization. A key observation is that the time derivative of μmax and side slip angle can be assumed as zero and computed based on measurement, respectively. This allows the observed variables to be updated at a relatively low frequency w.r.t. the solution of the optimization problem. During the interval between each two neighbouring updating time, the observer estimates the μ max and side slip angle by integrating sensor information based on the last update. To find the global optima approximately, a grid search method is implemented for solving non-convex optimization. The estimation results from the proposed observer and a linearization based observer (lbo) are finally compared under various tire-road conditions with simulations and experiments. The results showed that 1) the proposed observer can always guarantee stability in a wide range of vehicle operations while lbo cannot. 2) w.r.t. root mean square of estimation error, the proposed observer performs overall better than lbo in μmax estimation.

On Fluorophore Imaging by Diffusion Equation Model: Decompositions and Optimizations

Fluorescence imaging is a target-specific molecular imaging technology using absorption coefficient of fluorophore. For this imaging model governed by an inverse problem for the coupled diffusion system, which describes the interaction of the excitation field from several boundary sources and the corresponding emission field, we reformulate it as an optimization problem. For solving this non-quadratic optimizing problem, we propose a decomposition scheme, which extracts the horizontal information of the target from the boundary measurement data directly. The realizability of this hybrid imaging scheme is rigorously proved mathematically for cubic and ellipsoid targets, by constructing an indicator function for the horizontal location of the target explicitly. Then based on this horizonal location as a good initial guess for the iteration process, the cost functional is optimized efficiently using the trust domain scheme. Numerical implementations are provided to show the validity of the proposed scheme.

Acceleration of implicit schemes for large systems of nonlinear differential-algebraic equations

When solving large systems of nonlinear differential-algebraic equations by implicit schemes, each integration step requires the solution of a system of large nonlinear algebraic equations. The latter is solved by an inexact Newton method which, in its turn, leads to a set of large linear systems commonly solved by a Krylov subspace iterative method. The efficiency of the whole process depends on the initial guesses for the inexact Newton and the Krylov subspace methods. An inexpensive approach is proposed and justified that computes good initial guesses for these methods. It requires a subspace of small dimension and the use of line search and trust region for the inexact Newton method and Petrov-Galerkin for the Krylov subspace method. Numerical examples are included to illustrate the effectiveness of the proposed approach.

A Fast and Accurate Planar-Feature-Based Global Scan Registration Method

Scan registration is the basis of LiDAR-based mapping and localization. When a good initial guess is available, classical local registration algorithms such as iterative closet point (ICP) and normal distributions transform (NDT) can be utilized. However, scan registration without odometer or inertial measurement unit (IMU) is still an open problem. After reviewing the existing methods, we summarize a modular planar-feature-based registration framework. Based on the idea of modular design, we present a novel global scan registration method based on planar features. A modified renormalization-based algorithm considering the sensor noise model is derived and used for accurate plane fitting and covariance computing. An efficient adaptive plane segmentation algorithm is employed to extract planar patches which is based on voxel growing. A novel projected-image-based method is utilized for fast plane attribute calculation. The attributes of individual planes and relation constrains between the planes are employed to robustly build plane correspondences. The plane-plane singular value decomposition (SVD) method is used to estimate the transform, and normalization of the point coordinates is adopted to further improve accuracy. A consensus metric is introduced to determine plane correspondences and transform solution updating, and to select the optimal solution using the distance from the centroid of one plane to its conjugate plane. Extensive comparative experiments were performed on four datasets covering different structured scenarios to evaluate the performance of the proposed registration method. The experimental results demonstrate that the proposed method achieves a better balance between accuracy and efficiency, and has more stable and robust performance across different scenarios compared to all tested state-of-the-art algorithms.

Fast Generation of Optimal Asteroid Landing Trajectories Using Deep Neural Networks

To improve the autonomy and reliability of asteroid landings, an intelligent approach to fast and reliable generation of optimal landing trajectories is proposed. Unfortunately, the indirect methods require good initial guesses for shootings. To address this issue, deep neural networks (DNNs) are developed to approximate the costates of the indirect methods and supply good initial solutions to achieve high success rate of shootings. This article focuses on the following three contributions. First, the original asteroid landing problems are connected with two-dimensional (2-D) asteroid-free transfer problems using model continuation and state transformation techniques. Second, DNNs are developed and optimized to approximate the costates of the 2-D transfers with high accuracy. Third, a systematical solution for asteroid landing trajectory optimization is developed, wherein the DNNs provide good initial solutions, and the accurate solutions for landings are obtained through a solution continuation process. Additionally, an alternative solution supplying strategy based on error statistics of DNN approximation is presented to take over the solution supplying when the predictions of DNNs fail to converge to the optima. Evaluations of the DNNs for solution supplying and simulations of landings on 433 Eros and 101955 Bennu are given to substantiate the effectiveness of the proposed techniques and demonstrate the performance of the developed algorithm.

Use of Artificial Neural Network for Initial design of steel structures

Design of steel structures is an iterative process that often turns complicated when requires initial guess. A good initial design guess that comes from past design experience can considerably reduce the number of subsequent analysis and design cycles. It is difficult to form declarative rules to express human intuitions and past experience. Another problem in design process is arriving at optimal solution, where optimization process in most cases is computationally expensive and time consuming. Artificial Neural Network (ANN) is a promising computational model that can perform cognitive tasks, such as learning and optimization. This paper focuses on application of ANN in design so as to make design process more efficient. The aim is to train ANN to arrive at optimal solution by considering design constraints along with practical. This is illustrated with an example of design of compression member which primarily requires initial guess. The economy in design is tried to obtain by providing training set of optimal solutions to a multilayer neural network. The design process followed to generate training set is in accordance with IS: 800-2007. The solutions given by ANN are approximate but have reasonable agreement with the expected solutions.

Qnm: A Python package for calculating Kerr quasinormal modes, separation constants, and spherical-spheroidal mixing coefficients

$\mathtt{qnm}$ is an open-source Python package for computing the Kerr quasinormal mode frequencies, angular separation constants, and spherical-spheroidal mixing coefficients. The $\mathtt{qnm}$ package includes a Leaver solver with the Cook-Zalutskiy spectral approach to the angular sector, and a caching mechanism to avoid repeating calculations. We provide a large cache of low $\ell, m, n$ modes, which can be downloaded and installed with a single function call, and interpolated to provide good initial guess for root-polishing at new values of spin.

Real-Time Optimal Control for Spacecraft Orbit Transfer via Multiscale Deep Neural Networks

This study is motivated by the requirement of on-board trajectory optimization with guaranteed convergence and real-time performance for optimal spacecraft orbit transfers. To this end, a real-time optimal control approach is proposed using deep learning technologies to obtain minimum-time trajectories of solar sail spacecraft for orbit transfer missions. First, the minimum-time two-dimensional orbit transfer problem is solved by an indirect method, and the costate normalization technique is introduced to increase the probability of finding the optimal solutions. Second, by making novel use of deep learning technologies, three deep neural networks are designed and trained offline by the obtained optimal solutions to generate the guidance commands in real time during flight. Consequently, the long-standing difficulty of on-board trajectory generation is resolved. Then, an interactive network training strategy is presented to increase the success rate of finding optima by supplying good initial guesses for the indirect method. Moreover, a multiscale network cooperation strategy is designed to deal with the recognition deficiency of deep neural networks (DNNs) with small input values, which helps achieve highly precise control of terminal orbit insertion. Numerical simulations are given to substantiate the efficiency of these techniques, and illustrate the effectiveness and robustness of the proposed DNN-based trajectory control for future on-board applications.

Combining Linear Algebra and Numerical Optimization for Gray-Box Affine State-Space Model Identification

Accurately estimating the parameters of a gray-box linear time-invariant state-space representation is still a challenging problem especially if the number of unknowns exceeds ten. The standard nonlinear optimization-based procedures often fail because the initial guesses are not in the domain of attraction of the user-defined cost function global minimum. Herein, a new solution is developed to give access to estimated parameters which are accurate estimates of the unknown parameters directly or, at worst, good initial guesses for any standard nonlinear optimization-based method. By assuming that 1) the gray-box model to identify is minimal and structurally identifiable, 2) the parameter dependency is affine, and 3) the available datasets have been transformed into a reliable and accurate black-box state-space model, our procedure consists in determining the unique similarity transformation between the black-box and gray-box representations of the system to identify. More specifically, we introduce a technique which first extracts a system of equations linear in terms of the similarity transformation; second, it solves this system by using standard linear algebra tools; finally, completes this two-step procedure by a numerical optimization solution (dedicated now to a very small number of unknowns) if the second step does not give access to all the similarity transformation parameters effectively. In this contribution, we first describe our new procedure, prove its capabilities of giving accurate estimates under specific practical constraints, and finally demonstrates its effectiveness with simulation examples.

A steepest descent algorithm for the computation of traveling dissipative solitons

An algorithm is proposed to calculate traveling dissipative solitons for the FitzHugh–Nagumo equations. It is based on the application of the steepest descent method to a certain functional. This approach can be used to find solitons whenever the problem has a variational structure. Since the method seeks the lowest energy configuration, it has robust performance qualities. It is global in nature, so that initial guesses for both the pulse profile and the wave speed can be quite different from the correct solution. Also, bifurcations have a minimal effect on the performance. In the literature, there is a conjecture that no stable traveling pulse exists for a 2-component system in 2D unbounded domains. In many instances, such numerical studies investigate only solutions with a small speed, as they rely on good initial guesses based on stable standing pulse profiles. Studying a modified problem with a 2D strip domain $${\mathbb {R}} \times [-L,L]$$ with zero Dirichlet boundary conditions at $$y=\pm L$$, by using our algorithm we establish the existence of fast-moving solitons. With an appropriate set of physical parameters in this unbounded rectangular strip domain, we observe the co-existence of single-soliton and 2-soliton solutions together with additional unstable traveling pulses. The algorithm automatically calculates these various pulses as the energy minimizers at different wave speeds. In addition to finding individual solutions, we anticipate that this approach could be used to augment or initiate continuation algorithms. We also note that the rectangular strip domain can serve as a first step to investigating waves in the whole of $${\mathbb {R}}^2$$.

Blind Search for Complex Chemical Pathways Using Harmonic Linear Discriminant Analysis.

Disentangling the mechanistic details of a chemical reaction pathway is a hard problem that often requires a considerable amount of chemical intuition and a component of luck. Experiments struggle in observing short-life metastable intermediates, while computer simulations often rely upon a good initial guess. In this work, we propose a method that, from the simulations of a reactant and a product state, searches for reaction mechanisms connecting the two by exploring the configuration space through metadynamics, a well-known enhanced molecular dynamics method. The key quantity underlying this search is based on the use of an approach called harmonic linear discriminant analysis which allows a systematic construction of collective variables. Given the reactant and product states, we choose a set of descriptors capable of discriminating between the two states. In order to not prejudge the results, generic descriptors are introduced. The fluctuations of the descriptors in the two states are used to construct collective variables. We use metadynamics in an exploratory mode to discover the intermediates and the transition states that lead from reactant to product. The search is at first conducted at a low theory level. The calculation is then refined, and the energy of the intermediates and transition states discovered during metadynamics is computed again using a higher level of theory. The method's aim is to offer a simple reaction mechanism search procedure that helps in saving time and is able to find unexpected mechanisms that defy well established chemical paradigms. We apply it to two reactions, showing that a high level of complexity can be hidden even in seemingly trivial and small systems. The method can be applied to larger systems, such as reactions in solution or catalysis.

A hybridization of cuckoo search and particle swarm optimization for solving nonlinear systems

In numerical computations, one of the most strenuous problems is to solve systems of nonlinear equations. It is known that traditional numerical methods such as Newton methods and their variants require differentiability and/or good initial guess for the solutions. In practice, it will be difficult to get this initial solution and costly in term of the time to compute Jacobian. Therefore, there is a need to develop an algorithm to avoid the requirements of these traditional methods. This study proposes a new hybrid algorithm by incorporating cuckoo search (CS) with particle swarm optimization (PSO), called CSPSO, for solving systems of nonlinear equations. The goal of the hybridization between CS and PSO is to incorporate the best attributes of two algorithms together to structure a good-quality algorithm. One of the disadvantages to CS, it requires a large number of function evaluations to get the optimal solution, and to PSO, it is trapped into local minima. Our proposed hybrid algorithm attempts to overcome the disadvantages of CS and PSO. Computational experiments of nine benchmark systems of nonlinear equations and 28 benchmark functions of CEC 2013 with various dimensions are applied to test the performance of CSPSO. Computational results show that CSPSO outperforms other existing algorithms by obtaining the optimum solutions for most of the systems of nonlinear equations and 28 benchmark functions of CEC 2013, and reveals its efficacy in the comparison with other algorithms in the literature.

An Analytical Solution and Nonlinear Regression Analysis for Sandface Temperature Transient Data in the Presence of a Near-Wellbore Damaged Zone

This study presents an approximate analytical solution for predicting drawdown temperature transient behaviors of a fully penetrating vertical well in a two-zone radial composite reservoir system. The inner zone may represent a damaged (skin) zone, and the outer (non-skin) zone represents an infinitely extended reservoir. The analytical solution is obtained by solving the decoupled isothermal pressure diffusivity equation and temperature equation for the inner and outer zones with the Boltzmann transformation. The convection, transient adiabatic expansion and Joule–Thomson heating effects are all accounted for in the solution. The developed solution compares well with the results of a thermal numerical simulator. The analytical solution is used as a forward model for estimating the parameters of interest by nonlinear regression built on a gradient-based maximum likelihood estimation (MLE) method. A methodology, based on semilog analyses of pressure and temperature data as well as log–log diagnostic plots of pressure- and temperature-derivative data, is proposed to obtain good initial guesses of parameters, which derive the MLE objective function to have reliable optimized estimates. The statistical measures such as estimated standard deviation of noise in pressure and temperature data, confidence intervals for parameters and correlation coefficients between parameter pairs are used to evaluate the goodness of fit and reliability of the estimated parameters from history matching pressure and/or temperature data sets. The results show that the rock, fluid and thermal properties of the skin zone and non-skin zone can be reliably estimated by regressing on temperature transient data jointly with pressure transient data in the presence of noise.

Linear inverse problem for inferring eruption source parameters from sparse ash deposit data as viewed from an atmospheric dispersion modeling perspective

Determination of the volcanic eruption source parameters—total grain-size distribution and vertical ash mass distribution (VMD) within the source—is carried out on a collection of measured-area samples and granulometry data. For this, the geophysical inverse methods and Hybrid Particle and Concentration Transport Model (HYPACT) driven by wind and turbulence fields simulated with the Regional Atmospheric Modeling System (RAMS) are used. A two-step inversion procedure is proposed to obtain approximate but physically meaningful solution when the total number of ashfall samples is small and it is not possible to make a good initial guess of the source parameters. First, a spectrum of particle fall velocities is estimated by selecting a best-fit subset of aerodynamically distinct subpopulations of free and aggregate particles from the trial set used to simulate a polycomponent ashfall. The singular value decomposition (SVD) analysis is then employed to identify spatial components of the ash emissions’ vertical distribution, as resolvable by the observations. Model validation experiments are conducted for the January 12, 2011, short-duration vulcanian explosion at Kizimen and paroxysmal phase of the December 24, 2006, sub-Plinian eruption at Bezymianny. The derived VMDs exhibit high variability in fine ash content (~ 60–100 wt%) as well as strong secondary maxima in the lower troposphere, likely reflecting the contribution of ash particles fallen out of co-pyroclastic flow ash clouds and partially collapsing eruption columns.

A quantum eigensolver for symmetric tridiagonal matrices

The eigenproblem of a symmetric tridiagonal matrix widely appears in physical applications and scientific computing. The classical solvers are the symmetric QR iteration, divide-and-conquer, bisection and inverse iteration, etc. For a symmetric tridiagonal matrix of dimension N, the cost of these methods scales at least as $$O(N^2)$$ . In this work, we present an efficient quantum algorithm for solving this problem through quantum simulation of resonant transitions. The cost of the algorithm can scale as $$poly(\log \,N)$$ , provided a good initial guess on the eigenstates of the system is obtained. By coupling a probe qubit to a quantum register that represents a system, resonance dynamics can be observed on the probe qubit when its frequency matches a transition frequency in the system. Energy eigenvalues of the system can be obtained by locating resonant transition frequencies between the probe qubit and transitions from a reference state to eigenstates of the system through observing dynamics of the probe qubit for different frequencies. And the system can be evolved to one of its eigenstate with known eigenvalue by inducing an appropriate resonant transition between the probe qubit and the system.

Fast and efficient chemical process flowsheet simulation by pseudo-transient continuation on inertial manifolds

Process flowsheets are mathematical models that describe the steady-state operation of (petro)chemical processes and comprise large-scale systems of nonlinear algebraic equations. The solution of these systems (referred to as “flowsheet simulation”) is challenging for practically-relevant problems. Pseudo-transient continuation is a promising numerical approach for process flowsheet simulation, and has been shown to converge flowsheets reliably from a wider range of initial guesses compared to standard Newton-type methods. However, the pseudo-transient approach is more computationally demanding than flowsheet simulation with a Newton-type solver (assuming a good initial guess is available for the latter). Moreover, process flowsheet simulation with pseudo-transient continuation involves defining a set of dynamics in pseudo-time, and both convergence and computational efficiency can be highly dependent on the user-selected time constants that govern this dynamic behavior. In this work, we address these challenges with a novel algorithm for process flowsheet simulation based on a hierarchical, multiply-singularly perturbed formulation of pseudo-transient dynamics, and an associated cascade of quasi-steady-state assumptions. The algorithm replaces a subset of the differential equations of the reformulated model with algebraic quasi-steady-state conditions when the system approaches the inertial manifold defined by steady state of the respective dynamics. The proposed technique allows a seamless transition from pseudo-time integration to efficient algebraic solvers, and we prove that the method converges to the solution of the original algebraic system in a finite number of steps. Moreover, the time-scale decomposition of the pseudo-transient dynamics eliminates the need to explicitly set the time constants of the dynamics in each time scale. Using two prototype chemical process examples, we show that the proposed method significantly reduces the computational effort involved in pseudo-transient process flowsheet simulation.

Artificial neural network assisted two-phase flash calculations in isothermal and thermal compositional simulations

Flash calculations are widely employed in compositional simulations in the determination of a number of phases, phase amount and phase composition at each grid block. The efficiency and robustness of a flash algorithm is critical to saving simulation run time and producing reliable simulation results. Traditional flash calculation methods that consist of a stability test and phase splitting calculations often require tremendous computational time to iteratively solve nonlinear equations. In general, the stability test is the most time-consuming part as it aims at calculating the compositions of an unstable trial phase from various empirical initial estimates. For the phase splitting calculations, their convergence behavior highly depends on the quality of initial guesses of K-values. Good initial guesses can significantly reduce the number of iterations, avoid converging to an incorrect solution and, therefore, enhance the efficiency and robustness of a flash algorithm. Conventionally, initial guesses for the phase splitting calculations come from the values at the previous time step, the neighboring grid blocks or the stability test, but sometimes they do not work well.In this work, we use artificial neural networks (ANNs) to assist the traditional flash calculations in achieving their fast and robust convergence. ANN models are built standalone prior to a simulation and can be considered as a preparation stage. Once the ANN models are successfully built, they can benefit a large number of simulation runs during history matching and production optimization. There are two ANN models developed in this work: an ANN-STAB model and an ANN-SPLIT model. The ANN-STAB model is built to fit a saturation pressure diagram and the ANN-SPLIT model is generated to regress mole fractions and K-values. At given pressure, temperature and feed composition, the ANN-STAB model is first used to label the stability of a system; if it shows instability, the ANN-SPLIT model is then used to predict the mole fractions and K-values, which are to be used as initial guesses in the following phase splitting calculations. Compared with the traditional stability test algorithms, the ANN-STAB model gives almost the same accuracy but consumes much less computational time. The ANN-SPLIT model provides thoroughly reliable initial guesses for phase splitting calculations. As a result, it requires far fewer Newton iterations.