Nonlinear Conjugate Gradient Research Articles

Accelerating the Frequency Domain Controlled-Source Electromagnetic Data Inversion Using Rational Krylov Subspace Algorithm

Controlled-source electromagnetic (CSEM) method is crucial for detecting and locating underground anomalies and structures. However, it is challenging to interpret the field data with multi-frequency via 3D CSEM inversion. To fully excavate and utilize the valuable information of CSEM data at different frequencies, we propose an efficient algorithm for 3D multi-frequency CSEM (MFCSEM) inversion based on rational Krylov (RK) subspace. Within the framework of our algorithm, we first use the three-term Lanczos recursion to construct the RK basis matrix quickly; thus, the fast MFCSEM forward modeling can be realized via the RK approximation. Subsequently, we present a novel cyclic projection and correction (CPC) algorithm to solve the MFCSEM adjoint forward problems. Finally, the nonlinear conjugate gradient (NLCG) method is adopted to seek a solution to this nonlinear inverse problem. We demonstrate the excellent performance of our algorithm by synthetic and field data sets. The inversion results show that our algorithm is computationally efficient, resulting in considerable speedup compared with the conventional method. Our algorithm provides a new idea that would significantly improve the efficiency of MFCSEM inversion.

Nonlinear Conjugate Gradient Coefficients with Exact and Strong Wolfe Line Searches Techniques

Nonlinear conjugate gradient (CG) methods are very important for solving unconstrained optimization problems. These methods have been subjected to extensive researches in terms of enhancing them. Exact and strong Wolfe line search techniques are usually used in practice for the analysis and implementation of conjugate gradient methods. For better results, several studies have been carried out to modify classical CG methods. The method of Fletcher and Reeves (FR) is one of the most well‐known CG methods. It has strong convergence properties, but it gives poor numerical results in practice. The main goal of this paper is to enhance this method in terms of numerical performance via a convexity type of modification on its coefficient βk. We ensure that with this modification, the method is still achieving the sufficient descent condition and global convergence via both exact and strong Wolfe line searches. The numerical results show that this modified FR is more robust and effective.

An Accelerated Algorithm for 3-D Multifrequency CSEM Imaging With Undulating Topography

Three-dimensional frequency-domain controlled-source electromagnetic (CSEM) inversion is an essential technology for subsurface conductivity imaging. In this letter, we develop an efficient 3D inversion scheme for multi-frequency CSEM (MFCSEM) data measured on a topographic earth. Firstly, the model is discretized using the unstructured mesh, which has the ability to simulate the undulating topography. Then, we divide the frequency range into two independent frequency intervals and use the rational Krylov (RK) subspace algorithm with the OpenMP/MPI hybrid parallelization scheme to accelerate the calculations of MFCSEM forward and adjoint forward. Finally, the nonlinear conjugate gradient (NLCG) method is utilized to solve this optimization problem. We invert a synthetic data set to verify the computational performance, and the results show that our algorithm is efficient and can obtain reliable inversion results. Furthermore, it also indicates that ignoring the effect of topography can cause severe distortion to the inversion results.

Sufficient Descent Condition of the Polak-Ribière-Polyak (PRP) Conjugate Gradient Method without Line Search

Nonlinear conjugate gradient methods for unconstrained optimization problems are used in many aspects of theoretical and applied sciences. They are iterative methods, so at any iteration a step length is computed using a method called line search. In most cases, the sufficient descent condition plays an important role to prove the global convergence of a conjugate gradient method. Due to its outperformance in practical computation, the Polak-Ribière-Polyak (PRP) conjugate gradient method is widely used for solving nonlinear unconstrained optimization problems. However, the sufficient descent condition of PRP has not established without line search yet. In this study, we established the sufficient descent condition without line search based on the conditions 0 < bkPRP £ x bkFR and 1 < bkPRP £mbkFR, where 0 < x < 1 and m > 1. As a result, we found that under certain conditions, the sufficient descent condition is satisfied when the PRP implemented without line search

Improvement of conjugate gradient methods for removing impulse noise images

Optimization problems occur in most disciplines like engineering, physics, mathematics, economics, administration, commerce, social sciences, and even politics. The conjugate coefficient is the cornerstone of conjugate gradient algorithms with the desired conjugate property. In this study, we discovered fresh second order information for the Hessian from the target function, which might lead to a new search direction. Based on a unique search direction, we proposed the update formula and nonlinear conjugate gradient technique. Under Wolfe line search and moderate objective function assumptions, the strategy has acceptable descent property and is always globally convergent. According to numerical results, the technique is successful and competitive in recovering the original picture from an image corrupted by impulsive noise.

Paleozoic suture and Mesozoic tectonic evolution of the lithosphere between the northern section of the Xing'an Block and the Songnen Block: Evidence from three-dimensional magnetotelluric detection

The Xing'an and Songnen Blocks are microcontinents in the eastern Central Asian Orogenic Belt (CAOB). The Heihe–Hegenshan suture has been interpreted to represent the boundary between the blocks. However, the lack of ophiolite exposure and deformation towards the northern extent of the suture mean that its precise nature and location remain controversial. To better understand the position and structure of the suture, magnetotelluric data were acquired at 104 stations along five profiles in the northern Songnen and Xing'an Blocks. A three-dimensional nonlinear conjugate gradient algorithm was used to image the resistivity structure of the lithosphere. Our resistivity models indicate that the lithospheres of both the Xing'an and Songnen Blocks are characterized by overall high resistivity and have heterogeneous electrical structures. We define a highly conductive anomaly (C2) to the east of the Duobaoshan Island Arc that divides lithosphere of high resistivity and controls the distribution of Carboniferous magmatic rocks related to collision. We suggest that C2 is the northern extension of the Heihe–Hegenshan suture. The highly conductive anomaly (C1) is located in the Xing'an Block on the west side of the island arc. While the exposed lithology on the surface is Jurassic intrusions, usually with high resistivity. We suggest that C1 may be an early structure, which was transformed by the closure of the Mongolia–Okhotsk Ocean. No large-scale Mesozoic intrusive bodies occur above C3, a highly conductive anomalous region in the Songnen Block. And the seismic reflection characteristics have changed significantly in the region C3. Therefore, we interpret C3 as a shear zone in the crust related to a change in the subduction direction of the paleo-Pacific plate. The resistivity model provides new geophysical evidence for determining the suture and the evolutionary traces of the Mongolia-Okhotsk Ocean and the Paleo-Pacific Ocean.

Improved Fletcher-Reeves Methods Based on New Scaling Techniques

This paper introduces a scaling parameter to the Fletcher-Reeves (FR) nonlinear conjugate gradient method. The main aim is to improve its theoretical and numerical properties when applied with inexact line searches to unconstrained optimization problems. We show that the sufficient descent and global convergence properties of Al-Baali for the FR method with a fairly accurate line search are maintained. We also consider the possibility of extending this result to less accurate line search for appropriate values of the scaling parameter. The reported numerical results show that several values for the proposed scaling parameter improve the performance of the FR method significantly.

On the convergence rate of Fletcher‐Reeves nonlinear conjugate gradient methods satisfying strong Wolfe conditions: Application to parameter identification in problems governed by general dynamics

Over the last decades, many efforts have been made toward the understanding of the convergence rate of the gradient‐based method for both constrained and unconstrained optimization. The cases of the strongly convex and weakly convex payoff function have been extensively studied and are nowadays fully understood. Despite the impressive advances made in the convex optimization context, the nonlinear non‐convex optimization problems are still not fully exploited. In this paper, we are concerned with the nonlinear, non‐convex optimization problem under system dynamic constraints. We apply our analysis to parameter identification of systems governed by general nonlinear differential equations. The considered inverse problem is presented using optimal control tools. We tackle the optimization through the use of Fletcher‐Reeves nonlinear conjugate gradient method satisfying strong Wolfe conditions with inexact line search. We rigorously establish a convergence analysis of the method and report a new linear convergence rate which forms the main contribution of this work. The theoretical result reported in our analysis requires that the second derivative of the payoff functional be continuous and bounded. Numerical evidence on a selection of popular nonlinear models is presented as a direct application of parameter identification to support the theoretical findings.

A new conjugate gradient algorithms using conjugacy condition for solving unconstrained optimization

The primarily objective of this paper which is indicated in the field of conjugate gradient algorithms for unconstrained optimization problems and algorithms is to show the advantage of the new proposed algorithm in comparison with the standard method which is denoted as. Hestenes Stiefel method, as we know the coefficient conjugate parameter is very crucial for this reason, we proposed a simple modification of the coefficient conjugate gradient which is used to derived the new formula for the conjugate gradient update parameter described in this paper. Our new modification is based on the conjugacy situation for nonlinear conjugate gradient methods which is given by the conjugacy condition for nonlinear conjugate gradient methods and added a nonnegative parameter to suggest the new extension of the method. Under mild Wolfe conditions, the global convergence theorem and lemmas are also defined and proved. The proposed method's efficiency is programming and demonstrated by the numerical instances, which were very encouraging.

New scaled algorithm for non-linear conjugate gradients in unconstrained optimization

&lt;p&gt;A new scaled conjugate gradient (SCG) method is proposed throughout this paper, the SCG technique may be a special important generalization conjugate gradient (CG) method, and it is an efficient numerical method for solving nonlinear large scale unconstrained optimization. As a result, we proposed the new SCG method with a strong Wolfe condition (SWC) line search is proposed. The proposed technique's descent property, as well as its global convergence property, are satisfied without the use of any line searches under some suitable assumptions. The proposed technique's efficiency and feasibility are backed up by numerical experiments comparing them to traditional CG techniques.&lt;/p&gt;

A comparative study for selecting and using simulation methods of Gaussian random surfaces

Conventional methods for generating Gaussian random surfaces, including the moving average (MA) time series model with nonlinear conjugate gradient method (NCGM), two-dimensional (2-D) digital filter method, and spectral representation method (SRM), are implemented with a wide range of autocorrelation length and truncation length values of the autocorrelation function (ACF). The ACF, power spectral density function (PSDF), and essential roughness parameters of the simulated surfaces are calculated and compared. Based on the simulation results, the mechanism of the truncation length of ACF affecting the simulated surfaces can be summarized as that the step formed by truncating ACF is not sufficiently small, thus resulting in non-negligible errors in the corresponding PSDF. Such errors will be propagated to the simulated surfaces. A conservative criterion is proposed to avoid the adverse effects of truncating ACF: make the autocorrelation length less than 8% of the surface dimensions and the truncation length at least seven times autocorrelation length. The results show that the MA model with NCGM overestimates the PSDF values of simulated surfaces in the high-frequency region, meaning significant high-frequency noise in the simulated surfaces. The 2-D digital filter method and the SRM have almost the same performance, and both methods are better than the MA model with NCGM when the criterion of truncating ACF is fulfilled. The SRM generates rough surfaces with the smallest standard deviation in terms of the roughness parameters, ACF, and PSDF in most cases, meaning that it can generate accurate surfaces at every single simulation and is more stable and efficient. Therefore, the SRM is the most recommended method among the three methods studied.

Applying three-dimensional inversion to the frequency-domain response converted from transient electromagnetic data for a rectangular fixed loop

The computationally expensive cost and complexity are practical challenges in solving the three-dimensional (3-D) inverse problem in the time domain, and many real-world surveys are still inverted in one dimension (1-D). We present a new way to invert transient electromagnetic (TEM) data from a rectangular fixed loop to image subsurface electrical conductivity, which overcomes this limitation and permits efficient implementation of the inversion process on a personal computer. A TEM signal was transformed to the real and imaginary components of a frequency-domain (FD) electromagnetic (EM) response. The transformation was performed by a regularization inversion method. The recovered FD EM response was used as the data for the 3-D inversion. The forward problem and sensitivity were solved by using a staggered-grid finite-difference technique in the FD, and the background Green's function was carried out using the virtual interface method. The non-linear conjugate gradient method (NLCG) was employed to solve the 3-D inverse problem. A synthetic example revealed the basic functionality of the approach, and the FD inversion of a TEM field dataset acquired in the northeastern Qinshui Basin, Shanxi Province, China demonstrated a real-world application. Our inversion approach is general and suitable for different-type sources, such as a vertical magnetic dipole or horizontal electric dipole.

Truncated trust region method for nonlinear inverse problems and application in full-waveform inversion

We present a general truncated trust region method to solve large-scale nonlinear inverse problems. The truncated trust region method can serve as an implicit regularization method, and it can take advantage of the second-order derivative information of the misfit functional. The convergence of the truncated trust region method is provided under some smoothness assumptions. To improve the computational efficiency and reduce the memory requirement, we develop a second-order adjoint-state method to efficiently estimate matrix–vector products, and solve the trust region subproblem using the truncated conjugate gradient method with the matrix-free strategy. The full-waveform inversion problem is used to test the numerical performance of the proposed method. Numerical results show that the truncated trust region method can perform better than conventional methods (e.g. nonlinear conjugate gradient method and L-BFGS) for highly nonlinear inverse problems, in terms of inverting resolution.

Global convergence of a descent PRP type conjugate gradient method for nonconvex optimization

Nonlinear conjugate gradient methods are often used to solve large-scale unconstrained optimization problems owing to their lower memory requirement and less computation cost. In this paper, we propose a new Polak-Ribiére-Polyak type conjugate gradient method, which satisfies the sufficient descent condition independent of any line search. A remarkable property about this method is its strong global convergence with the standard Wolfe line search conditions as well as the standard Armijo line search strategy without convexity assumption of the objective function. The numerical results demonstrating the efficiency of the proposed method are reported.

3D nonlinear conjugate gradient inversion for frequency-domain airborne EM based on vector finite element method

The 3D inversion is a challenging task in the research of the frequency-domain airborne electromagnetic (AEM) method due to the massive amounts of data and limited computational efficiency. In this paper, we introduce the 3D FAEM inversion based on the vector finite element method (FEM). Compare with the traditional nodal FEM, the vector FEM can improve the computational efficiency by reducing the number of variables and the scale of linear equations. To tackle the computational demands, in the calculation process we apply the PARDISO solver with shared-memory distributed-memory multi-processors to solve the multi-source forward linear equations and calculate the gradient of the objective function by adjoint equations. Then typical resistivity model with the anomalous body is carried out to verify the validity of the proposed method. Meanwhile, in allusion to the multi-solution problem caused by the inversion problem, we study the influence of typical initial models on inversion results. Finally, we applied our algorithm to invert field data. The inversion results demonstrate that the proposed method can obtain reliable AEM resistivity inverted results.

Two Improved Nonlinear Conjugate Gradient Methods with the Strong Wolfe Line Search

Two Improved Nonlinear Conjugate Gradient Methods with the Strong Wolfe Line Search

A new class of nonlinear conjugate gradient coefficients for unconstrained optimization

The nonlinear Conjugate gradient method (CGM) is a very effective way in solving large-scale optimization problems. Zhang et al. proposed a new CG coefficient which is defined by [Formula: see text]. They proved the sufficient descent condition and the global convergence for nonconvex minimization in strong Wolfe line search. In this paper, we prove that this CG coefficient possesses sufficient descent conditions and global convergence properties under the exact line search.

A Mathematical and Computational Proposal for the Development of Tight-Binding Order-N Density Matrix Methodology

The aim of this paper is to present, in a broad context, a proposal of mathematical foundation that serves as framework for the development of the electronic structure calculation methodology called “DMTB-Densit Matrix Tight Binding method.” ), which was originally presented in the literature by the group led by physicist David Vanderbilt in 1993 (\cite{vanderbilt}), as well as its relationship to the computational modeling that can be chosen for its implementation. The approach adopted makes it clear that the final mathematical formulation of this methodology is completely dependent on the computational strategies chosen for its effective implementation. Thus, we put the DMTB as a mathematical-computational model of variable final formulation. Finally, we propose an implementation based on the nonlinear conjugate gradient method. The final model obtained is slightly different from the DMTB that was originally presented in 1993, in agreement with the version presented by Millam and Scuseria in 1997 (\cite{scuseria}). The approach used develops the mathematical aspects, aiming at the effective computational implementation of the methodology.

Optimal Control through Leadership of the Cucker and Smale Flocking Model with Time Delays

The Cucker and Smale model is a well-known flocking model that describes the emergence of flocks based on alignment. The first part focuses on investigating this model, including the effect of time delay and the presence of a leader. Furthermore, the control function is inserted into the dynamics of a leader to drive a group of agents to target. In the second part of this work, leadership-based optimal control is investigated. Moreover, the existence of the first-order optimality conditions for a delayed optimal control problem is discussed. Furthermore, the Runge–Kutta discretization method and the nonlinear conjugate gradient method are employed to solve the discrete optimality system. Finally, the capacity of the proposed control approach to drive a group of agents to reach the desired places or track the trajectory is demonstrated by numerical experiment results.

Dai–Liao extensions of a descent hybrid nonlinear conjugate gradient method with application in signal processing

Dai–Liao extensions of a descent hybrid nonlinear conjugate gradient method with application in signal processing