Gauss-Newton Optimization Method Research Articles

Deep Gauss-Newton for phase retrieval.

We propose the deep Gauss-Newton (DGN) algorithm. The DGN allows one to take into account the knowledge of the forward model in a deep neural network by unrolling a Gauss-Newton optimization method. No regularization or step size needs to be chosen; they are learned through convolutional neural networks. The proposed algorithm does not require an initial reconstruction and is able to retrieve simultaneously the phase and absorption from a single-distance diffraction pattern. The DGN method was applied to both simulated and experimental data and permitted large improvements of the reconstruction error and of the resolution compared with a state-of-the-art iterative method and another neural-network-based reconstruction algorithm.

Three-Dimensional Airborne Electromagnetic Data Inversion With Flight Altitude Correction

The flight altitude has a large effect on the airborne electromagnetic (AEM) responses. Due to the dynamic environment of the aircraft, the recorded sensor altitudes may contain errors. Research demonstrates that the AEM responses caused by a several meters altitude errors can be larger than caused by some anomalous body. Ignoring these errors will create erroneous results in AEM data interpretation. Considering that there is not yet a published 3D AEM inversion method that takes into account the flight altitude, we develop in this paper a 3D inversion algorithm for AEM with the flight height treated as an inversion parameter. For the forward modeling we use the finite element method, while for the inversion we use the Gauss-Newton optimization method. To make our inversion works for variable flight altitudes, we propose a scheme of 3D Jacobean matrix calculation for both the resistivities and flight altitudes without much increasing the computational cost. The numerical simulation result confirms that the flight altitude really has a large effect on the AEM responses. The inversions of synthetic data show that our 3D inversion method can both recover the resistivity distribution in the underground and decrease the altitude errors recorded, while the field data inversion demonstrates that our method can deliver a better inversion model with a smaller data misfit.

Two-dimensional Inversion of DC Resistivity Data on Unstructured Grids Using Fuzzy C-means Clustering Model Constraint

Direct current resistivity prospecting is a commonly geophysical method for environmental and engineering applications. In this paper, we propose a fuzzy C-means clustering model constrained inversion algorithm for two-dimensional DC resistivity. To fit arbitrary geological structure and surface of the earth, our inversion algorithm is developed based on unstructured model mesh. To be consistent with the geological structure, the fuzzy C-means clustering model constraint is added to the inversion cost function with the minimum structure model constraint, and the Gauss-Newton optimization method is used to seek solutions of the nonlinear inverse problem. Finally, we verify the performance of our algorithm by synthetic and field data sets. The results show that the resistivity and boundary can be better restored when the correct number and value of priori cluster centers were set. By testing the field data, the inversion algorithm can obtain obvious abnormal boundaries.

3-D Inversion of CSEM Data With Hexahedral Mesh in the Multinary Model Space

The controlled-source electromagnetic (CSEM) method is a crucial tool for near-surface investigations and hydrocarbon exploration because of its economic benefits. Limited resolution is one of the inherent defects of the CSEM method, and to obtain high contrast results a multinary transform function was introduced to CSEM 3-D inversion. The multinary transform function was constituted by superposing several error functions, and the transform function transformed the model parameters from continuously distributed space into a semistep distributed multinary space. To deal with complex geometries, the edge-based finite element (FE) method with an irregular hexahedral grid was applied to the modeling and inverse problem. We used the Gauss-Newton optimization method to minimize the Tikhonov parametric function. The complex chessboard model and the marine CSEM model with complex geometry were used to validate the ability of the new method in improving the resolution of the CSEM method. By comparing the results of conventional, focusing, and multinary inversion methods, the effectiveness of the multinary inversion method in depicting sharp boundaries of different physical properties was proved. Additionally, the comparisons proved that multinary inversion method can, to some extent, overcome the insensitivity to low conductors of the CSEM method.

Dynamic Characterization of Reservoirs Constrained by Time-Lapse Prestack Seismic Inversion

Dynamic changes of reservoir parameters within a reservoir are usually estimated by a history matching process based on the production data. However, it is difficult to accurately acquire the spatial distribution using production data as the constraints. We formulate a nonlinear inversion method to dynamically characterize the reservoir parameter variations from time-lapse prestack seismic data. Besides, we introduce the modified Hertz-Mindlin (H-M) model to simulate the changes in the elastic behavior of a turbidite sandstone during production by establishing a link between reservoir parameters and elastic parameters. Then, the exact Zoeppritz equation is exploited as a forward engine to convert these parameters into synthetic time-lapse seismic data. Numerical examples verify that the porosity parameter has a higher reflection sensitivity than the other two parameters, which is followed by the effective pressure. The water saturation parameter, however, is the least sensitive. Following a Bayesian approach, the baseline and monitor seismic data are used in the seismic inversion to establish the regularized augmented function. Combining the modified H-M rock physics model with the exact Zoeppritz equation as a forward operator, a regularized function is minimized with the Gauss-Newton optimization method to determine a model update. We further apply the proposed inversion algorithm to a real seismic data set, including baseline and monitor seismic data. The results demonstrate that the proposed inversion method can not only yield an accurate description of subsurface static reservoir properties but also improve the accuracy of dynamic reservoir parameter characterization.

3-D Magnetotelluric Inversion and Application Using the Edge-Based Finite Element With Hexahedral Mesh

Three-dimensional (3-D) inversion technique has become an important and practical approach for magnetotelluric (MT) data interpretation. In this article, we developed a 3-D parallelized MT inversion scheme using the edge-based finite element method and applied the developed method to the newly collected MT data in the Xinjiang Luntai area. The distorted hexahedral element is adopted to incorporate topography into the forward modeling and inversion for complicated scenarios. We use the Gauss–Newton optimization method to minimize the objective functional for MT inversion. The developed algorithm is parallelized using MPI over frequencies and parallel direct solvers when solving the forward and adjoint problems for each frequency. We compare the performance of the least-square QR (LSQR) factorization and preconditioned conjugate gradient (PCG) solvers for the model update within each Gauss–Newton iteration and found that the LSQR solver is more stable. The developed inversion algorithm is validated using several synthetic models. Finally, we applied the inversion algorithm to the subsurface resistivity imaging in the Luntai area. The recovered geoelectric model from full 3-D inversion fits well with the known geological and geophysical information. The recovered model shows a low resistivity layer which may be caused by the salt strata. Besides, the inversion results reveal the movement tectonic in this survey area within a depth of 9 km.

A new approach for porosity and permeability prediction from well logs using artificial neural network and curve fitting techniques: A case study of Niger Delta, Nigeria

A new approach based on artificial neural network and standardized non-linear regression analysis is proposed for the prediction of reservoir properties from limited well log data. Two separate multi-layer-perceptron neural networks using the Bayesian framework were employed and mathematical models were developed from the networks. These neural networks were used to predict reservoir properties from well log data. The porosity neural network (NN) utilized density log (RHOB) as input and core porosity as target, while the permeability NN employed RHOB and water saturation as input and core derived permeability as target. Mean square errors (MSEs) for training, validation and test for the porosity network were 1.20889e−6, 2.22574e−6, and 1.37204e−6 respectively while 5.62849e−1, 3.13709e−1, and 4.83728e−1 were obtained for the permeability network. These networks were later applied to four wells to predict porosity and permeability. Very good correlation coefficients (R-values) between the NN predicted data and the target data were obtained in each of these wells. For the porosity network R-values of 0.99987, 0.99099, 0.99474 and 0.83749 were obtained, whereas 0.97584, 0.83594, 0.97002 and 0.83512 respectively were observed from the permeability network. For the model development, standardized nonlinear regression curve fitting tool was utilized. The polynomial model that determines the coefficients of the fitted model through the Gauss-Newton optimization method was used. Several criteria such as robust fitting, normalization and scaling were employed to further reduce and randomize the errors so that it follows a Gaussian distribution with zero mean and constant variance. Goodness of the fit was tested with the observable statistical parameters. For the porosity model, the sum of squared error (SSE) of 2.2715 e−04, R-square of 0.9997, Adjusted R-square of 0.9997 and root mean square error (RSME) of 4.3562 e−04 was observed while the permeability model, SSE of 7.8315e01, R-square of 0.9753, Adjusted R-square of 0.9752 and RSME of 8.092 was observed. This approach provides a simple and cost effective way of estimating reservoir properties.

An integrated use of induced polarization and electrical resistivity imaging methods to delineate zones of potential gold mineralization in the Phitshane Molopo area, Southeast Botswana

We used 2D and pseudo-3D induced polarization (IP) in conjunction with electrical resistivity imaging (ERI) data inversion to identify and characterize potential zones of gold mineralization in Phitshane Molopo area. Reports on historic exploration suggest that gold mineralization is associated with quartz and/or carbonate veins that cut across the banded iron formations. The 2D IP/ERI measurements were acquired along ten 300 m long profiles trending northeast southwest, with 15 m line separation and 7.5 m electrode spacing. Thereafter, we inverted the data using the standard Gauss-Newton optimization method to produce 2D and 3D models. Based on a combination of IP and resistivity results, we identified distinct lithological units comprising the mineralized and non-mineralized zones. High chargeability (≥6.86 mV/V) and high resistivity (≥641 Ωm) anomalies were attributed to the presence of disseminated sulfides or gold-bearing quartz and/or carbonate veins that obliquely crosscut (east west trend) the banded iron formation (BIF) units and the intensely silicified BIF units, respectively. Zones exhibiting a combination of low to intermediate chargeability (1.0–6.86 mV/V) and low to intermediate resistivity (1.4–641 Ωm) are likely non-sulfidized zones with little/no gold mineralization in the study area. The integrated use of IP and ERI methods was successful in delineating zones that are likely to be hosting disseminated sulphides and/or potentially gold-rich zones in the Phitshane Molopo area, Kraaipan Greenstone Belt.

Performance enhancement of Gauss-Newton trust-region solver for distributed Gauss-Newton optimization method

Distributed Gauss-Newton (DGN) optimization method has been proved very efficient and robust for history matching and uncertainty quantification (HM&UQ). The major bottleneck for performance enhancement is the expensive computational cost of solving hundreds of Gauss-Newton trust-region (GNTR) subproblems in each iteration. The original GNTR solver applies the less efficient iterative Newton-Raphson (NR) method using a derivative which requires solving a large-scale linear system twice in each NR iteration. Instead of using a less accurate linear proxy as in the iterative NR method, the nonlinear GNTR equation is first approximated with an inverse-quadratic (IQ) or a cubic-spline (CS) model, by fitting points generated in previous iterations without using any derivative. Then, the analytical (or numerical) solution of the IQ (or CS) model is used as the new proposal for the next iteration. The performances of the two new GNTR solvers are benchmarked against different methods on different sets of test problems with different numbers of uncertain parameters (ranging from 500 to 100,000) and different numbers of observed data (ranging from 100 to 100,000). In terms of efficiency and robustness, the two new GNTR solvers have comparable performance, and they outperform other methods we tested, including the well-known direct and iterative trust-region solvers of the GALAHAD optimization library. Finally, the proposed GNTR solvers have been implemented in our in-house distributed HM&UQ system. Our numerical experiments indicate that the DGN optimizer using the newly proposed GNTR solver performs quite stable and is effective when applied to real-field history matching problems.

Integration of Support Vector Regression With Distributed Gauss-Newton Optimization Method and Its Applications to the Uncertainty Assessment of Unconventional Assets

Summary Reservoir model parameters generally have very large uncertainty ranges, and need to be calibrated by history matching (HM) available production data. Properly assessing the uncertainty of production forecasts (e.g., with an ensemble of calibrated models that are conditioned to production data) has a direct impact on business decision making. It requires performing numerous reservoir simulations on a distributed computing environment. Because of the current low-oil-price environment, it is demanding to reduce the computational cost of generating multiple realizations of history-matched models without compromising forecasting quality. To solve this challenge, a novel and more efficient optimization method (referred to as SVR-DGN) is proposed in this paper, by replacing the less accurate linear proxy of the distributed Gauss-Newton (DGN) optimization method (referred to as L-DGN) with a more accurate response-surface model of support vector regression (SVR). Resembling L-DGN, the proposed SVR-DGN optimization method can be applied to find multiple local minima of the objective function in parallel. In each iteration, SVR-DGN proposes an ensemble of search points or reservoir-simulation models, and the flow responses of these reservoir models are simulated on high-performance-computing (HPC) clusters concurrently. All successfully simulated cases are recorded in a training data set. Then, an SVR proxy is constructed for each simulated response using all training data points available in the training data set. Finally, the sensitivity matrix at any point can be calculated analytically by differentiating the SVR models. SVR-DGN computes more-accurate sensitivity matrices, proposes better search points, and converges faster than L-DGN. The quality of the SVR proxy is validated with a toy problem. The proposed method is applied to a real field HM example of a Permian liquid-rich shale reservoir. The uncertain parameters include reservoir static properties, hydraulic-fracture properties, and parameters defining relative permeability curves. The performance of the proposed SVR-DGN optimization method is compared with the L-DGN optimizer and the hybrid Gauss-Newton with a direct-pattern-search (GN-DPS) optimizer, using the same real field example. Our numerical tests indicate that the SVR-DGN optimizer can find better solutions with smaller values of the objective function and with a less computational cost (approximately one-third of L-DGN and 1/30 of GN-DPS). Finally, the proposed method is applied to generate multiple conditional realizations for the uncertainty quantification of production forecasts.

Enhancing the Performance of the Distributed Gauss-Newton Optimization Method by Reducing the Effect of Numerical Noise and Truncation Error With Support-Vector Regression

Summary Numerical optimization is an integral part of many history-matching (HM) workflows. However, the optimization performance can be affected negatively by the numerical noise existent in the forward models when the gradients are estimated numerically. As an unavoidable part of reservoir simulation, numerical noise refers to the error caused by the incomplete convergence of linear or nonlinear solvers or truncation errors caused by different timestep cuts. More precisely, the allowed solver tolerances and allowed changes of pressure and saturation imply that simulation results no longer smoothly change with changing model parameters. For HM with linear-distributed Gaussian-Newton (L-DGN), caused by the discontinuity of simulation results, the sensitivity matrix computed by linear interpolation might be less accurate, which might result in slow convergence or, even worse, failure of convergence. Recently, we have developed an HM workflow by integrating the support-vector regression (SVR) with the distributed-Gaussian-Newton (DGN) method optimization method referred to as SVR-DGN. Unlike L-DGN that computes the sensitivity matrix with a simple linear proxy, SVR-DGN computes the sensitivity matrix by taking the gradient of the SVR proxies. In this paper, we provide theoretical analysis and case studies to show that SVR-DGN can compute a more-accurate sensitivity matrix than L-DGN, and SVR-DGN is insensitive to the negative influence of numerical noise. We also propose a cost-saving training procedure by replacing bad-training points, which correspond to relatively large values of the objective function, with those training-data points (simulation data) that have smaller values of the objective function and are generated at most-recent iterations for training the SVR proxies. Both the L-DGN approach and the newly proposed SVR-DGN approach are tested first with a 2D toy problem to show the effect of numerical noise on their convergence performance. We find that their performance is comparable when the toy problem is free of numerical noise. As the numerical-noise level increases, the performance of the L-DGN degrades sharply. By contrast, the SVR-DGN performance is quite stable. Then, both methods are tested using a real-field HM example. The convergence performance of the SVR-DGN is quite robust for both the tight and loose numerical settings, whereas the performance of the L-DGN degrades significantly when loose numerical settings are applied.

Distributed Gauss-Newton optimization method for history matching problems with multiple best matches

Minimizing a sum of squared data mismatches is a key ingredient in many assisted history matching (AHM) workflows. A novel approach is developed to efficiently find multiple local minima of a data mismatch objective function, by performing Gauss-Newton (GN) minimizations concurrently while sharing information between dispersed regions in the reduced parameter space dynamically. To start, a large number of different initial parameter values (i.e., model realizations) are randomly generated and are used as initial search points and base-cases for each subsequent optimization. Predicted data for all realizations are obtained by simulating these search points concurrently, and relevant simulation results for all successful simulation jobs are recorded in a training data set. A local quadratic model around each base-case is constructed using the GN formulation, where the required sensitivity matrix is approximated by linear regression of nondegenerated points, collected in the training data set, that are closest to the given base-case. A new search point for each base-case is generated by minimizing the local quadratic approximate model within a trust region, and the training data set is updated accordingly once the simulation job corresponding to each search point is successfully completed. The base-cases are updated iteratively if their corresponding search points improve the data mismatch. Finally, each base-case will converge to a local minimum in the region of attraction of the initial base-case. The proposed approach is applied to different test problems with uncertain parameters being limited to hundreds or fewer. Most local minima of these test problems are found with both satisfactory accuracy and efficiency.

RBA: Reduced Bundle Adjustment for oblique aerial photogrammetry

This study proposes an efficient Bundle Adjustment (BA) model for oblique aerial photogrammetry to reduce the number of unknown parameters and the dimensions of a non-linear optimization problem. Instead of serving as independent exterior orientations, oblique camera poses are parameterized with nadir camera poses and constant relative poses between oblique and nadir cameras. New observation functions are created with image points as a function of the nadir pose and the relative pose parameters. With these observation functions, the problem of BA is defined as finding optimal unknown parameters by minimizing the total difference between estimated and observed image points. A Gauss-Newton optimization method is utilized to provide a solution for this least-square problem with a reduced normal equation, which plays a very critical role in the convergence and efficiency of BA. Compared with traditional BA methods, the number of unknown parameters and the dimension of the normal equations decrease, this approach dramatically reduces the computational complexity and memory cost especially for large-scale scenarios with a number of oblique images. Four synthetic datasets and a real dataset were used to check the validation and the accuracy of the proposed method. The accuracy of the proposed method is very close to that of the traditional BA method, but the efficiency can be significantly improved by the proposed method. For very large-scale scenarios, the proposed method can still address the limitation of memory and orientate all images captured by an oblique aerial multi-camera system.

Erroneous branch parameters detection and correction in real time simulation of power systems

Purpose The purpose of this paper is to propose an efficient method for detection and modification of erroneous branch parameters in real time power system simulators. The aim of the proposed method is to minimize the sum of squared errors (SSE) due to mismatches between simulation results and corresponding field measurements. Assuming that the network configuration is known, a limited number of erroneous branch parameters will be detected and corrected in an optimization procedure. Design/methodology/approach Proposing a novel formulation that utilizes network voltages and last modified admittance matrix of the simulation model, suspected branch parameters are identified. These parameters are more likely to be responsible for large values of SSE. Utilizing a Gauss-Newton (GN) optimization method, detected parameters will be modified in order to minimize the value of SSE. Required sensitivities in optimization procedure will be calculated numerically by the real time simulator. In addition, by implementing an efficient orthogonalization method, the more effective parameter will be selected among a set of correlated parameters to avoid singularity problems. Findings Unlike state estimation-based methods, the proposed method does not need the mathematical functions of measurements to simulation model parameters. The method can enhance other parameter estimation methods that are based on state estimation. Simulation results demonstrate the high efficiency of the proposed optimization method. Originality/value Incorrect branch parameter detection and correction procedures are investigated in real time simulators.

Gaussian Process Gauss–Newton for non-parametric simultaneous localization and mapping

In this paper, we present Gaussian Process Gauss–Newton (GPGN), an algorithm for non-parametric, continuous-time, nonlinear, batch state estimation. This work adapts the methods of Gaussian process (GP) regression to address the problem of batch simultaneous localization and mapping (SLAM) by using the Gauss–Newton optimization method. In particular, we formulate the estimation problem with a continuous-time state model, along with the more conventional discrete-time measurements. Two derivations are presented in this paper, reflecting both the weight-space and function-space approaches from the GP regression literature. Validation is conducted through simulations and a hardware experiment, which utilizes the well-understood problem of two-dimensional SLAM as an illustrative example. The performance is compared with the traditional discrete-time batch Gauss–Newton approach, and we also show that GPGN can be employed to estimate motion with only range/bearing measurements of landmarks (i.e. no odometry), even when there are not enough measurements to constrain the pose at a given timestep.

Full-Wave Three-Dimensional Microwave Imaging With a Regularized Gauss–Newton Method— Theory and Experiment

A reconstruction algorithm is detailed for three-dimensional full-vectorial microwave imaging based on Newton-type optimization. The goal is to reconstruct the three-dimensional complex permittivity of a scatterer in a homogeneous background from a number of time-harmonic scattered field measurements. The algorithm combines a modified Gauss-Newton optimization method with a computationally efficient forward solver, based on the fast Fourier transform method and the marching-on-in-source-position extrapolation procedure. A regularized cost function is proposed by applying a multiplicative-additive regularization to the least squares datafit. This approach mitigates the effect of measurement noise on the reconstruction and effectively deals with the non-linearity of the optimization problem. It is furthermore shown that the modified Gauss-Newton method converges much faster than the Broyden-Fletcher-Goldfarb-Shanno quasi-Newton method. Promising quantitative reconstructions from both simulated and experimental data are presented. The latter data are bi-static polarimetric free-space measurements provided by Institut Fresnel, Marseille, France.

FAULT DETECTION OF A NONLINEAR SWITCHING SYSTEM USING FINITE MEMORY OBSERVERS

This paper is devoted to the design of a fault detection strategy for nonlinear switching discrete-time systems using Finite Memory Observers (FMO). Using both the past input/output measurements and the model equations, state estimation is expressed as a minimization problem of a cost function. Gauss-Newton's optimization method is investigated in a diagnosis purpose. Performances of FMO observers for residual generation on hybrid systems will be illustrated through numerical simulations on a benchmark two tank system.

Effects of Model Sensitivity and Nonlinearity on Nonlinear Regression of Ground Water Flow

Nonlinear regression is increasingly applied to the calibration of hydrologic models through the use of perturbation methods to compute the Jacobian or sensitivity matrix required by the Gauss-Newton optimization method. Sensitivities obtained by perturbation methods can be less accurate than those obtained by direct differentiation, however, and concern has arisen that the optimal parameter values and the associated parameter covariance matrix computed by perturbation could also be less accurate. Sensitivities computed by both perturbation and direct differentiation were applied in nonlinear regression calibration of seven ground water flow models. The two methods gave virtually identical optimum parameter values and covariances for the three models that were relatively linear and two of the models that were relatively nonlinear, but gave widely differing results for two other nonlinear models. The perturbation method performed better than direct differentiation in some regressions with the nonlinear models, apparently because approximate sensitivities computed for an interval yielded better search directions than did more accurately computed sensitivities for a point. The method selected to avoid overshooting minima on the error surface when updating parameter values with the Gauss-Newton procedure appears for nonlinear models to be more important than the method of sensitivity calculation in controlling regression convergence.

Identification of moderately nonlinear flight mechanics systems withadditive process and measurement noise

The parameter estimation problem for dynamic systems with both process and measurement noise from nonlinear model postulates is addressed in this paper. A two-step estimation procedure explicitly computes the covariance matrix of residuals and updates the system parameters, the initial conditions, and the state noise matrix using the Gauss-Newton optimization method. For the purpose of state estimation in nonlinear systems with process noise, an approximate steady-state filter is used. In each iteration, the filter-gain matrix is obtained from the postulated system model linearized at the updated initial conditions. The gradients of the output variables and of the system functions are approximated by finite differences. The proposed approach for nonlinear systems with unknown process and measurement noise covariances is first validated on simulated aircraft response data. It is then applied to estimate the aircraft longitudinal derivatives from flight test data using two models with different degrees of nonlinearities. Advantages and possible limitations of the method are discussed.

Esterification kinetics of phenol with phosphoryl trichloride

The liquid-phase esterification kinetics of phenol with POCl3 has been studied at isothermal conditions within the temperature interval from 90 to 110 °C. The esterification rate constants of the first, second, and third esterification steps and the activation energies of these steps have been calculated. The given system of competitive consecutive reactions can be described by a system of differential equations which has been solved by the Gauss-Newton optimization method of non-linear regression in the Marquardt modification. In the sequence of the first, second, and third step the esterification rate constants have been found to gradually decrease, whereas the activation energies uncrease in the same sequence.