Inexact Line Search Research Articles

Steepest descent method for uncertain multiobjective optimization problems under finite uncertainty set

In this paper, a steepest descent method is developed for the robust counterpart of uncertain multiobjective optimization problems under finite uncertainty. The robust counterpart under consideration is the minimum of objective wise worst case, which is a nonsmooth deterministic multiobjective optimization problem. To solve this robust counterpart with the help of the steepest descent method, a subproblem is constructed and solved to find a descent direction. An Armijo-type inexact line search technique is employed to find a suitable step length. With the help of the descent direction and step length, a sequence is generated. The convergence of the proposed method is justified under some common assumptions. Finally, the algorithm is verified and compared with one existing method with the help of some numerical problems.

The Dai–Liao-type conjugate gradient methods for solving vector optimization problems

This paper attempts to propose Dai–Liao (DL)-type nonlinear conjugate gradient (CG) methods for solving vector optimization problems. Four variants of the DL method are extended and analysed from the scalar case to the vector setting. We first give a direct vector extended version of the modified Dai–Liao (DL+) method. Then the second algorithm is presented by combining a new extension form of the well-known Hager–Zhang (HZ) parameter. This new scheme equips a good descent property and its parameter also reduces to the classical one in the scalar case. The last two algorithms extend two sufficient descent DL-type methods. Particularly, two different vector forms of their parameters are considered and compared. As a result, we reveal some differences between scalar and vector cases to a certain extent. Without any convex assumption, the sufficient descent property and global convergence of all algorithms are established under inexact line searches. Finally, preliminary numerical experiments illustrate the practical behaviour of our algorithms by using Dolan and Moré performance profile.

A modified quasi-Newton method for uncertain multiobjective optimization problems under a finite uncertainty set

ABSTRACT In this article, a modified quasi-Newton method is developed for the robust counterpart (RC) of an uncertain multiobjective optimization problem (UMOP). Specifically, the RC of a UMOP is the minimum of an objective-wise worst-case (OWWC) type RC. To determine the descent direction for the RC, a subproblem using a common Hessian approximation is constructed. An inexact line search technique based on the Armijo technique is employed to find an appropriate step length. Furthermore, a modified version of the Davidon–Fletcher–Powell (DFP) update formula is developed for the RC. A modified quasi-Newton algorithm is built using descent direction step length, and DFP Hessian update formula. Furthermore, the convergence of the modified quasi-Newton algorithm is also discussed. Finally, the modified quasi-Newton algorithm is illustrated with a set of test problems and compared with the weighted sum method. Moreover, it is demonstrated that the algorithm is effective for both convex and non-convex problems.

A Descent Matrix-free Nonlinear Conjugate Gradient Algorithm for Impulse Noise Removal

The convergence of the Polak, Ribie‘re-Polyak (PRP) conjugate gradient (CG) method requires some modifications for improved theoretical properties. In this article, we explore an optimal choice of the Perry conjugacy condition to propose a hybrid CG parameter for solving optimization and inverse problems, particularly in an image reconstruction model. This parameter is selected to satisfy a combination of revised version of the PRP and Dai-Yuan (DY) CG methods. The numerical implementation includes inexact line search, showcasing the scheme's robustness (highest number of solved functions) compared to other known CG algorithms. The efficiency is shown in terms of Real error (RelErr), peak signal noise ratio (PNSR), and CPU time in seconds for impulse noise removal while for unconstrained minimization problems, the study evaluated the efficiency based on number of iterations, function evaluation, and CPU time in seconds. An interesting feature of the proposed method is its ability to converges to the minimizer regardless of the initial guess, relying on certain established assumptions.

A Dai-Liao Hybrid PRP and DY Schemes for Unconstrained Optimization

This article presents a new conjugate gradient (CG) method that requires first-order derivatives but overcomes the slow convergence issue associated with the steepest descent method and does not require the computation of second-order derivatives, as needed in the Newton method. The CG update parameter is suggested from the extended conjugacy condition as a convex combination of Polak, Ribiére, and Polyak (PRP) and Dai and Yuan (DY) algorithms by employing the optimal choice of the modulating parameter 't'. Numerical computations show that the algorithm is robust and efficient based on the number of iterations and CPU time. The scheme converges globally under Wolfe line search and adopts an inexact line search to obtain the step-size that generates a descent property, without requiring exalt computation of the step size. Conjugate gradient method, Descent property, Dai-Liao conjugacy condition, Global convergence, Numerical methods

The modification conjugate coefficient for conjugate gradient method to solving unconstrained optimization problems

In this manuscript, a novel conjugate coefficient is derived through the utilization of equality conjugate gradient direction in conjunction with a modified quasi-Newton (QN) direction. The demonstrated results establish that this innovative approach ensures both sufficient descent and global convergence, meeting the criteria of the Wolf line search condition and inexact line search. Through numerical experiments, we employ the proposed method on a diverse set of established test functions, clearly demonstrating its enhanced efficiency. A comparative analysis is conducted against the FR CG method using the Dolan-More performance profile based on the number of function evaluations (NOF), number of iterations (NOI), and CPU time.

Hybrid derivative-free methods for solving system of nonlinear equations

This work presents a predictor-corrector iterative approach for solving systems of nonlinear equations. The methods are derivative-free with correction and acceleration parameters obtained via approximating the Jacobian matrix. Using an inexact line search procedure and under appropriate conditions, we proved that the proposed method is globally convergent. We, additionally, present some numerical results to show the efficiency and effectiveness of the proposed method.

MTAINV3D: A Three-dimensional Adaptive Magnetotelluric Inversion Tool Using Unstructured Grids

Currently, 3D magnetotelluric inversion often uses a fixed inversion grid. Too sparse inversion grids cannot approximate the complex subsurface conductivity distribution, and may also lead to inversion results failing to converge. Too dense inversion grids will increase not only the computational cost, but also the non-uniqueness of the inversion. Aiming at the above problems, we have developed a three-dimensional magnetotelluric adaptive inversion strategy with unstructured finite element and limited memory BFGS (L-BFGS) algorithms. Firstly, based on the unstructured finite element method, the high accuracy forward responses of the 3D geo-electric model with arbitrary complex terrain and conductivity distributions can be obtained. Then, the forward and inversion meshes are decoupled by using the nested forward and inversion tetrahedral meshes. The Thikhonov regularization objective function with smooth constraints is established and minimized by L-BFGS optimization algorithm with inexact line search procedure, where the gradient of the objective function is solved by using the adjoint principle. Most importantly, an inversion grid adjustment strategy based on the dual constraints of sensitivity matrix and model gradient is proposed, which are used to guide the automatic refinement of inversion grid in the optimization process. Finally, a topographic model is used to validate and test the performance of the proposed adaptive inversion algorithm.

Modifications for Quasi-Newton Method and Its Spectral Algorithm for Solving Unconstrained Optimization Problems

In this paper, two modifications for spectral quasi-Newton algorithm of type BFGS are imposed. In the first algorithm, named SQNEI, a certain spectral parameter is used in such a step for BFGS algorithm differs from other presented algorithms. The second algorithm, SQNEv-Iv, has both new parameter position and value suggestion. In SQNEI and SQNEv-Iv methods, the parameters are involved in a search direction after an approximated Hessian matrix is updated. It is provided that two methods are effective under some assumptions. Moreover, the sufficient descent property is proved as well as the global and superlinear convergence for SQNEv-Iv and SQNEI. Both of them are superior the standard BFGS (QNBFGS) and previous spectral quasi-Newton (SQNLC). However, SQNEv-Iv is outstanding SQNEI if it is convergent to the solution. This means that, two modified methods are in the race for the more efficiency method in terms less iteration numbers and consuming time in running CPU. Finally, numerical results are presented for the four algorithms by running list of test problems with inexact line search satisfying Armijo condition.

Optimization of unconstrained problems using a developed algorithm of spectral conjugate gradient method calculation

This paper presents a numerical investigation of the spectral conjugate directions formulation for optimizing unconstrained problems. A novel modified algorithm is proposed based on the conjugate gradient coefficient method. The algorithm employs the Wolfe inexact line search conditions to determine the optimum step length at each iteration and selects the appropriate conjugate gradient coefficient accordingly. The algorithm is evaluated through several numerical experiments using various unconstrained functions. The results indicate that the algorithm is highly stable, regardless of the starting point, and has better convergence rates and efficiency compared to classical methods in certain cases. Overall, this research provides a promising approach to solving unconstrained optimization problems.

Gradient-based descent linesearch to solve interval-valued optimization problems under gH-differentiability with application to finance

In this article, gradient based descent line search scheme is proposed to solve interval optimization problems under generalized Hukuhara differentiability. The innovation and importance of these concepts are presented from practical and computational point of view. The necessary condition for existence of critical point is presented in inclusion form of interval-valued gradient, without transforming it into real valued function. Suitable efficient descent direction is chosen based on the monotonic property of the interval-valued function and specific interval ordering. Mathematical convergence of the scheme is proved under the assumption of Inexact line search for interval optimization problem. The theoretical developments are implemented with a set of interval test problems in different dimensions. A possible application in finance is provided and solved by our proposed scheme.

Fuzzy Adaptive Parameter in the Dai–Liao Optimization Method Based on Neutrosophy

The impact of neutrosophy has increased rapidly in many areas of science and technology in recent years. Furthermore, numerous applications of the neutrosophic theory have become more usual. We aim to use neutrosophy to enhance Dai–Liao conjugate gradient (CG) iterative method. In particular, we suggest and explore a new neutrosophic logic system intended to compute the essential parameter t required in Dai–Liao CG iterations. Theoretical examination and numerical experiments signify the effectiveness of the introduced method for controlling t. By incorporation of the neutrosophy in the Dai–Liao conjugate gradient principle, we established novel Dai–Liao CG iterations for solving large-scale unconstrained optimization problems. Global convergence is proved under standard assumptions and with the use of the inexact line search. Finally, computational evidence shows the computational effectiveness of the proposed fuzzy neutrosophic Dai–Liao CG method.

A Levenberg-Marquardt Method for Tensor Approximation

This paper presents a tensor approximation algorithm, based on the Levenberg–Marquardt method for the nonlinear least square problem, to decompose large-scale tensors into the sum of the products of vector groups of a given scale, or to obtain a low-rank tensor approximation without losing too much accuracy. An Armijo-like rule of inexact line search is also introduced into this algorithm. The result of the tensor decomposition is adjustable, which implies that the decomposition can be specified according to the users’ requirements. The convergence is proved, and numerical experiments show that it has some advantages over the classical Levenberg–Marquardt method. This algorithm is applicable to both symmetric and asymmetric tensors, and it is expected to play a role in the field of large-scale data compression storage and large-scale tensor approximation operations.

Extending the inverse sequential quasi-Newton method for on-line monitoring and controlling of process conditions in the solidification of alloys

This work introduces the sequential quasi-Newton method (SQNM) to solve the inverse design problem of estimating the transient boundary heat fluxes that produce desired evolutions of the solidus and liquidus isotherms in alloys’ solidification processes. This is an unprecedented application for sequential gradient-based methods. The final goal is monitoring and controlling the evolution of the mushy zone in a near real-time fashion, which is of high technological interest to the foundry industry. We assess the performance of the SQNM in solving several inverse design problems in one- and two-dimensional domains. The method satisfactorily estimates the boundary heat fluxes using relatively few future measurements. However, some instabilities have been detected in the SQNM-predicted heat fluxes. To alleviate this issue, we proposed a modified version of SQNM, say mSQNM, that employs an inexact line search strategy to find the optimal step length during the iterative estimation procedure. We finally show how mSQNM improves the estimation of the boundary heat fluxes.

An Efficient Hybrid Conjugate Gradient Algorithm for Solving Intuitionistic Fuzzy Nonlinear Equations

This paper presents an iterative algorithm for solving intuitionistic fuzzy nonlinear equations (IFNEs). The proposed method is based on the classical conjugate gradient (CG) search direction. An interesting feature of the new algorithm is that it considers problems based on the special triangular intuitionistic fuzzy number. For this purpose, intuitionistic fuzzy quantities are transformed into membership and non-membership parametric forms, and a line search procedure is employed to compute the step length. Preliminary results from numerical experiments are presented to demonstrate the performance of the method. It is observed that the proposed hybrid CG method is highly effective and promising. KEYWORDS Hybrid CG, intuitionistic fuzzy nonlinear equation, parametric form, step length, inexact line search

A Gauss–Newton Online Ranging Method Based on Saturated Waveform Compensation of LiDAR

The saturated waveform is a main limiting factor of dynamic detection range of LiDAR, thus extracting range from saturated waveform accurately can extend the detection range of full-waveform LiDAR. A Gauss-Newton online ranging method based on saturated waveform compensation (GNC) was proposed, including rough estimation and optimized estimation of waveform parameters. To realize the rough estimation, the saturated echo waveform needs to be compensated using the Gaussian mixture model by scaling in amplitude and delaying in time of emitted waveform. A scale factor was estimated by piecewise fitting of the corrected saturation length and a delay factor can be quickly estimated by inexact line search. To extract the optimized parameters, a Gauss-Newton method was used to optimize rough parameters based on the compensated waveform. To measure the distance of the target in real time, the GNC method was implemented on FPGA (ZYNQ-7000) using a pipeline structure, taking the execution time of 7.87 μs for one saturated echo. A series of ranging experiments were conducted to evaluate the ranging performance of GNC for saturated waveforms. The experimental results showed that the ranging standard deviation (RStD) of GNC implemented on FPGA was less than 2.5 cm, and the absolute of mean ringing error (MRE) was less than 1.9 cm for 0~165% saturation. The proposed method provides an effective way to enhance the dynamic detection range with high precision.

Globally convergent conjugate gradient algorithms without the Lipschitz condition for nonconvex optimization

It is well known that under the Wolfe–Powell inexact line search, the global convergence of the nonlinear conjugate gradient method always requires the Lipschitz continuous condition for nonconvex functions. In this paper, we find that the Lipschitz continuous condition is unnecessary for proving the global convergence for particular algorithms if its searching direction has the well-known sufficient descent property and the trust region feature. Thus, the global convergence of the family conjugate gradient algorithms proposed by Yuan et al. (Numer. Algorithms, 84(2020)) is established without the Lipschitz condition for nonconvex functions since they have these two properties. Furthermore, a new algorithm without a line search technique is presented, and its global convergence is also analysed under suitable assumptions. The numerical results show that the performance of the proposed algorithm is competitive with some particular problems.

A New Quasi-Newton Method with PCG Method for Nonlinear Optimization Problems

The major stationary iterative method used to solve nonlinear optimization problems is the quasi-Newton (QN) method. Symmetric Rank-One (SR1) is a method in the quasi-Newton family. This algorithm converges towards the true Hessian fast and has computational advantages for sparse or partially separable problems [1]. Thus, investigating the efficiency of the SR1 algorithm is significant. It's possible that the matrix generated by SR1 update won't always be positive. The denominator may also vanish or become zero. To overcome the drawbacks of the SR1 method, resulting in better performance than the standard SR1 method, in this work, we derive a new vector <img src=image/13429423_01.gif> depending on the Barzilai-Borwein step size to obtain a new SR1 method. Then using this updating formula with preconditioning conjugate gradient (PCG) method is presented. With the aid of inexact line search procedure by strong Wolfe conditions, the new SR1 method is proposed and its performance is evaluated in comparison to the conventional SR1 method. It is proven that the updated matrix of the new SR1 method, <img src=image/13429423_02.gif>, is symmetric matrix and positive definite matrix, given <img src=image/13429423_03.gif> is initialized to identity matrix. In this study, the proposed method solved 13 problems effectively in terms of the number of iterations (NI) and the number of function evaluations (NF). Regarding NF, the new SR1 method also outperformed the classic SR1 method. The proposed method is shown to be more efficient in solving relatively large-scale problems (5,000 variables) compared to the original method. From the numerical results, the proposed method turned out to be significantly faster, effective and suitable for solving large dimension nonlinear equations.

A family of hybrid derivative-free methods via acceleration parameter for solving system of nonlinear equations

In this paper, we present some derivative-free methods for solving system of nonlinear equations based on approximating the Jacobian matrix via acceleration and correction parameters. Furthermore, we compute the step length using inexact line search procedure. Under appropriate conditions, we proved that the proposed methods are globally. We also present some numerical results to show the efficiency of the proposed methods by comparing them with some existing derivative-free methods in the recent literature.

Global convergence of three-term conjugate gradient methods on general functions under a new inexact line search strategy

This article deals with the global convergence of 3-Term Conjugate Gradient (3TCG) methods on general functions under a novel Inexact Line Search (ILS) strategy. The proposed algorithm satisfies the sufficient descent condition, and its global convergence can be easily proved. To verify the efficiency of the authors' theoretical approach, some numerical experiments and comparisons are presented. These results show that the suggested method is sufficiently efficient and comparable to new similar strategies for solving unconstrained optimization problems.