Line Search Strategy Research Articles

Multi-Material Decomposition for Single Energy CT Using Material Sparsity Constraint.

Multi-material decomposition (MMD) decomposes CT images into basis material images, and is a promising technique in clinical diagnostic CT to identify material compositions within the human body. MMD could be implemented on measurements obtained from spectral CT protocol, although spectral CT data acquisition is not readily available in most clinical environments. MMD methods using single energy CT (SECT), broadly applied in radiological departments of most hospitals, have been proposed in the literature while challenged by the inferior decomposition accuracy and the limited number of material bases due to the constrained material information in the SECT measurement. In this paper, we propose an image-domain SECT MMD method using material sparsity as an assistance under the condition that each voxel of the CT image contains at most two different elemental materials. L0 norm represents the material sparsity constraint (MSC) and is integrated into the decomposition objective function with a least-square data fidelity term, total variation term, and a sum-to-one constraint of material volume fractions. An accelerated primal-dual (APD) algorithm with line-search scheme is applied to solve the problem. The pixelwise direct inversion method with the two-material assumption (TMA) is applied to estimate the initials. We validate the proposed method on phantom and patient data. Compared with the TMA method, the proposed MSC method increases the volume fraction accuracy (VFA) from 92.0% to 98.5% in the phantom study. In the patient study, the calcification area can be clearly visualized in the virtual non-contrast image generated by the proposed method, and has a similar shape to that in the ground-truth contrast-free CT image. The high decomposition image quality from the proposed method substantially facilitates the SECT-based MMD clinical applications.

Multigrid Method for Optimal Control Problem Constrained by Stochastic Stokes Equations with Noise

Optimal control problems governed by stochastic partial differential equations have become an important field in applied mathematics. In this article, we investigate one such important optimization problem, that is, the stochastic Stokes control problem with forcing term perturbed by noise. A multigrid scheme with three-factor coarsening to solve the corresponding discretized control problem is presented. On staggered grids, a three-factor coarsening strategy helps in simplifying the inter-grid transfer operators and reduction in computation (CPU time). For smoothing, a distributive Gauss–Seidel scheme with a line search strategy is employed. To validate the proposed multigrid staggered grid framework, numerical results are presented with white noise at the end.

Elastic Full Waveform Inversion Based on the Trust Region Strategy

In this paper, we investigate the elastic wave full-waveform inversion (FWI) based on the trust region method. The FWI is an optimization problem of minimizing the misfit between the observed data and simulated data. Usually, the line search method is used to update the model parameters iteratively. The line search method generates a search direction first and then finds a suitable step length along the direction. In the trust region method, it defines a trial step length within a certain neighborhood of the current iterate point and then solves a trust region subproblem. The theoretical methods for the trust region FWI with the Newton type method are described. The algorithms for the truncated Newton method with the line search strategy and for the Gauss-Newton method with the trust region strategy are presented. Numerical computations of FWI for the Marmousi model by the L-BFGS method, the Gauss-Newton method and the truncated Newton method are completed. The comparisons between the line search strategy and the trust region strategy are given and show that the trust region method is more efficient than the line search method and both the Gauss-Newton and truncated Newton methods are more accurate than the L-BFGS method.

Modified projected Newton scheme for non-convex function with simple constraints

In this paper, a descent line search scheme is proposed to find a local minimum point of a non-convex optimization problem with simple constraints. The idea ensures that the scheme escapes the saddle points and finally settles for a local minimum point of the non-convex optimization problem. A positive definite scaling matrix for the proposed scheme is formed through symmetric indefinite matrix factorization of the Hessian matrix of the objective function at each iteration. A numerical illustration is provided, and the global convergence of the scheme is also justified.

On the Barzilai–Borwein gradient methods with structured secant equation for nonlinear least squares problems

ABSTRACT We propose structured spectral gradient algorithms for solving nonlinear least squares problems based on a modified structured secant equation. The idea was to integrate more details of the Hessian of the objective function into the standardized spectral parameters with the goal of improving numerical efficiency. We safeguard the structured spectral parameters to avoid negative curvature search direction. The sequence of the search direction generated by the respective proposed algorithm satisfies the sufficient descent condition. Using a nonmonotone line search strategy, we establish the global convergence of the proposed algorithms under some standard assumptions. Numerical experiments on some benchmark test problems show that the proposed algorithms are efficient and outperform some existing competitors.

Accelerating full waveform inversion through Adaptive Gradient Optimization methods and Dynamic Simultaneous Sources

Summary Full-Waveform Inversion (FWI) is a procedure based on the minimization of a misfit (or cost) function applied to the difference between synthetic waveforms and real seismic traces that derives high-resolution velocity models. This is achieved through the iterative adjustment of the velocity model and/or some other physical parameters of the Earth’s subsurface, which generally implies large computational effort. In order to minimize this cost function we explore the use of Adaptive Gradient Optimization (AGO), a variant of Stochastic Gradient Descent (SGD) methods, combining them with a dynamic simultaneous sources strategy that allow us to reduce the computational cost involved in this process. AGO methods are computationally efficient, have little memory requirements and have the capability of adapting the step-length according to the optimization process’ evolution. Since a precise calibration of the step-length is needed to ensure efficiency, the AGOs are well-suited for this task because they are able to adapt the step-length according to the optimization’s development. In this work, we propose a simple non-linear relationship that allows an adjustment of the step-length with respect to the frequencies used in the multiscale FWI, avoiding the line-search strategy’s high computational burden. Additionally, the application of this new step-length rule into the AGO methods with a dynamic simultaneous sources strategy, allow us to concurrently accelerate and significantly improve the FWI’s numerical performance and results. We compare the performance and final results of seven AGO methods, using two different FWI misfit functionals (based on L1 and L2 norms) applied to estimate the final velocity models of two benchmark acoustic models: the Marmousi and the Canadian overthrust BP velocity models.

A New Inexact Line Search Method for Convex Optimization Problems

In general one can say that line search procedure for the steplength and search direction are two important elements of a line search algorithm. The line search procedure requires much attention because of its far implications on the robustness and efficiency of the algorithm. The purpose of this paper is to propose a simple yet effective line search strategy in solving unconstrained convex optimization problems. This line search procedure does not require the evaluation of the objective function. Instead, it forces reduction in gradient norm on each direction. Hence it is suitable for problems when function evaluation is very costly. To illustrate the effectiveness of our line search procedure, we employ this procedure together with the symmetric rank one quasi-Newton update and test it against the same quasi-Newton method with the well-known Armijo line search. Numerical results on a set of standard unconstrained optimization problems showed that the proposed procedure is superior to the Armijo line search.

On the global convergence of a new spectral residual algorithm for nonlinear systems of equations

We present a derivative-free method for solving systems of nonlinear equations that belongs to the class of spectral residual methods. We will show that by endowing a previous version of the algorithm with a suitable new linesearch strategy, standard global convergence results can be attained under mild general assumptions. The robustness of the new method is therefore potentially improved with respect to the previous version as shown by the reported numerical experiments.

Improved convergence analysis of a smoothing Newton method for the circular cone programming

ABSTRACT In this paper, we propose a new smoothing Newton method to solve the circular cone programming (denoted by CCP). The proposed method is designed based on a non-monotone derivative-free line search scheme. We show that any accumulation point of the iteration sequence generated by this method is a solution of the CCP. Moreover, we prove that the proposed method is locally quadratically convergent without requiring strict complementarity conditions. Compared with existing smoothing Newton methods for solving the CCP, our method has three new features: (i) the generated iteration sequence is bounded; (ii) the value of the merit function converges to zero; (iii) the whole iteration sequence converges to an accumulation point if this point is isolated. Some numerical results are reported.

Topology optimization considering the Drucker–Prager criterion with a surrogate nonlinear elastic constitutive model

We address material nonlinear topology optimization problems considering the Drucker–Prager strength criterion by means of a surrogate nonlinear elastic model. The nonlinear material model is based on a generalized J2 deformation theory of plasticity. From an algorithmic viewpoint, we consider the topology optimization problem subjected to prescribed energy, which leads to robust convergence in nonlinear problems. The objective function of the optimization problem consists of maximizing the strain energy of the system in equilibrium subjected to a volume constraint. The sensitivity analysis is quite effective and efficient in the sense that there is no extra adjoint equation. In addition, the nonlinear structural equilibrium problem is solved through direct minimization of the structural strain energy using Newton’s method with an inexact line search strategy. Four numerical examples demonstrate features of the proposed nonlinear topology optimization framework considering the Drucker–Prager strength criterion.

A Second Order Algorithm for MCP Regularized Optimization

In this article, we provide two optimal property of MCP regularization optimization. One shows that the support set of a local minimizer corresponds to linearly independent columns of <inline-formula> <tex-math notation="LaTeX">$A$ </tex-math></inline-formula>, the other provides two sufficient conditions for a stationary point to be a local minimizer point. An active set subspace second-order algorithm for MCP regularized optimization is proposed. The active sets are estimated by an identification technique that can accurately identify the zero components in a neighbourhood of a stationary point. The search direction consists of two parts: some of the components are simply defined; the other components are determined by a second-order algorithm. A nonmonotone line search strategy that guarantees global convergence is used. The numerical comparisons with several state-of-art methods demonstrate the efficiency of the proposed method.

Inertial proximal gradient methods with Bregman regularization for a class of nonconvex optimization problems

This paper proposes an inertial Bregman proximal gradient method for minimizing the sum of two possibly nonconvex functions. This method includes two different inertial steps and adopts the Bregman regularization in solving the subproblem. Under some general parameter constraints, we prove the subsequential convergence that each generated sequence converges to the stationary point of the considered problem. To overcome the parameter constraints, we further propose a nonmonotone line search strategy to make the parameter selections more flexible. The subsequential convergence of the proposed method with line search is established. When the line search is monotone, we prove the stronger global convergence and linear convergence rate under Kurdyka–Łojasiewicz framework. Moreover, numerical results on SCAD and MCP nonconvex penalty problems are reported to demonstrate the effectiveness and superiority of the proposed methods and line search strategy.

A New Conjugate Gradient Projection Method for Convex Constrained Nonlinear Equations

The conjugate gradient projection method is one of the most effective methods for solving large-scale monotone nonlinear equations with convex constraints. In this paper, a new conjugate parameter is designed to generate the search direction, and an adaptive line search strategy is improved to yield the step size, and then, a new conjugate gradient projection method is proposed for large-scale monotone nonlinear equations with convex constraints. Under mild conditions, the proposed method is proved to be globally convergent. A large number of numerical experiments for the presented method and its comparisons are executed, which indicates that the presented method is very promising. Finally, the proposed method is applied to deal with the recovery of sparse signals.

A new smoothing-type algorithm for nonlinear weighted complementarity problem

In this paper we study the nonlinear weighted complementarity problem (denoted by NWCP). We first introduce a smoothing function which can be used to reformulate the NWCP as a system of smooth nonlinear equations. Then we propose a new smoothing-type algorithm to solve the NWCP which adopts a nonmonotone line search scheme. In each iteration, our algorithm solves one linear system of equations and performs one line search. Under suitable assumptions, we prove that the proposed algorithm is globally and locally quadratically convergent. Some numerical results are reported.

A Projection Hestenes–Stiefel Method with Spectral Parameter for Nonlinear Monotone Equations and Signal Processing

A number of practical problems in science and engineering can be converted into a system of nonlinear equations and therefore, it is imperative to develop efficient methods for solving such equations. Due to their nice convergence properties and low storage requirements, conjugate gradient methods are considered among the most efficient for solving large-scale nonlinear equations. In this paper, a modified conjugate gradient method is proposed based on a projection technique and a suitable line search strategy. The proposed method is matrix-free and its sequence of search directions satisfies sufficient descent condition. Under the assumption that the underlying function is monotone and Lipschitzian continuous, the global convergence of the proposed method is established. The method is applied to solve some benchmark monotone nonlinear equations and also extended to solve ℓ 1 -norm regularized problems to reconstruct a sparse signal in compressive sensing. Numerical comparison with some existing methods shows that the proposed method is competitive, efficient and promising.

The spectral conjugate gradient method in variational adjoint assimilation for model terrain correction II: Numerical test

The performance of the spectral conjugate gradient (SCG) method proposed in part I of this paper is evaluated by comparison with that of the limited-memory Broyden-Fletcher-Goldfarb-Shanno (LBFGS) method and Hager and Zhang’s CG_DESCENT method. The step length is determined by solving the tangent linear model and a line search algorithm with cubic interpolation is introduced for comparison. Numerical tests indicate that the SCG method is effective at correcting the bottom terrain. It is robust, providing small root mean square (RMS) errors and smooth profiles for the corrected bottom terrain for all tests conducted. It has much higher optimizing efficiency than Hager and Zhang’s CG_DESCENT and Liu’s LBFGS methods. The SCG method reduces the norm of the gradient sufficiently and the value of the cost function very quickly. It always provides linear convergence rates, especially when including a regularization term. This study suggests the combined utilization of the SCG method and line search strategy to ensure high efficiency when solving the large-scale unconstrained problems.

On the bang-bang control approach via a component-wise line search strategy for unconstrained optimization

&lt;p style='text-indent:20px;'&gt;A bang-bang iteration method equipped with a component-wise line search strategy is introduced to solve unconstrained optimization problems. The main idea of this method is to formulate an unconstrained optimization problem as an optimal control problem to obtain an optimal trajectory. However, the optimal trajectory can only be generated by impulsive control variables and it is a straight line joining a guessed initial point to a minimum point. Thus, &lt;i&gt;a priori&lt;/i&gt; bounds are imposed on the control variables in order to obtain a feasible solution. As a result, the optimal trajectory is made up of bang-bang control sub-arcs, which form an iterative model based on the Lyapunov function's theorem. This is to ensure monotonic decrease of the objective function value and convergence to a desirable minimum point. However, a chattering behavior may occur near the solution. To avoid this behavior, the Newton iterations are then applied to the proposed method via a two-phase approach to achieve fast convergence. Numerical experiments show that this new approach is efficient and cost-effective to solve the unconstrained optimization problems.&lt;/p&gt;

Gauss–Newton methods with approximate projections for solving constrained nonlinear least squares problems

This paper is concerned with algorithms for solving constrained nonlinear least squares problems. We first propose a local Gauss–Newton method with approximate projections for solving the aforementioned problems and study, by using a general majorant condition, its convergence results, including results on its rate. By combining the latter method and a nonmonotone line search strategy, we then propose a global algorithm and analyze its convergence results. Finally, some preliminary numerical experiments are reported in order to illustrate the advantages of the new schemes.

An exact line search scheme to accelerate the EM algorithm: Application to Gaussian mixture models identification

This paper tackles the slowness issue of the well-known expectation-maximization (EM) algorithm in the context of Gaussian mixture models. To cope with this slowness problem, an Exact Line Search scheme is proposed. It is based on exact computation of the step size required to jump, for a given search direction, towards the final solution. Computing this exact step size is easily done by only rooting a second-order polynomial computed from the initial log-likelihood maximization problem. Numerical results using both simulated and real dataset showed the efficiency of the proposed exact line search scheme when applied to the conventional EM algorithm as well as the anti-annealing based acceleration techniques based on either the EM or the expectation conjugate gradient algorithm.

A Family of Derivative-Free Conjugate Gradient Methods for Constrained Nonlinear Equations and Image Restoration

In this paper, a derivative-free conjugate gradient method for solving nonlinear equations with convex constraints is proposed. The proposed method can be viewed as an extension of the three-term modified Polak-Ribiere-Polyak method (TTPRP) and the three-term Hestenes-Stiefel conjugate gradient method (TTHS) using the projection technique of Solodov and Svaiter [Reformulation: Nonsmooth, Piecewise Smooth, Semismooth and Smoothing Methods, 1998, 355-369]. The proposed method adopts the adaptive line search scheme proposed by Ou and Li [Journal of Applied Mathematics and Computing 56.1-2 (2018): 195-216] which reduces the computational cost of the method. Under the assumption that the underlying operator is Lipschitz continuous and satisfies a weaker condition of monotonicity, the global convergence of the proposed method is established. Furthermore, the proposed method is extended to solve image restoration problem arising in compressive sensing. Numerical results are presented to demonstrate the effectiveness of the proposed method.