Large-scale Systems Of Nonlinear Equations Research Articles

Two-step inertial derivative-free projection method for solving nonlinear equations with application

Large-scale systems of nonlinear equations play a fundamental role in various applications, spanning from differential equations to economics, engineering, management science, probability theory, and various other applied sciences. This paper proposes a derivative-free iterative method to find ϵ-approximate solutions of large-scale nonlinear equations. The proposed scheme combines the hyperplane projection technique with a two-step inertial extrapolation, presenting a new approach. The proposed method does not require Jacobian information and thus can be applied to solve nonsmooth equations. Under standard assumptions, we prove the proposed method’s global convergence and nonasymptotic O(1/k) convergence rate. Furthermore, we include numerical results and comparisons to demonstrate the efficiency of the proposed method. Finally, we demonstrate the applicability of the proposed method in solving regularized decentralized logistic regression, a popular problem in machine learning applications.

Projection Method for Solving Large-scale System of Nonlinear Equations

Derivative-free projection methods have proven to be highly effective and valuable in solving large-scale systems of nonlinear equations (SNE). Extensive research is continuously being conducted to enhance existing methods and develop new projection methods. In this paper, we modified the conjugate gradient parameter proposed by Zhu et al. and extend it to solve SNE with convex constrain. The advantage of the proposed method is that it does not rely on Jacobian information and does not require the storage of any matrices at each iteration. This characteristic makes it well-suited for tackling large-scale non-smooth problems. Under appropriate conditions, we show that the proposed method is globally convergent. Numerical experiments were conducted to evaluate the effectiveness of the proposed method and compare it with other approaches.

Inexact free derivative quasi-Newton method for large-scale nonlinear system of equations

In this work, we propose a free derivative quasi-Newton method for solving large-scale nonlinear systems of equation. We introduce a two-stage linear search direction and develop its global convergence theory. Besides, we prove that the method enjoys superlinear convergence rate. Finally, numerical experiments illustrate that the proposed method is competitive with respect to Newton-Krylov methods and other well-known methods for solving large-scale nonlinear systems of equations.

A globally convergent method to accelerate large-scale optimization using on-the-fly model hyperreduction: Application to shape optimization

We present a numerical method to efficiently solve optimization problems governed by large-scale nonlinear systems of equations, including discretized partial differential equations, using projection-based reduced-order models accelerated with hyperreduction (empirical quadrature) and embedded in a trust-region framework that guarantees global convergence. The proposed framework constructs a hyperreduced model on-the-fly during the solution of the optimization problem, which completely avoids an offline training phase. This ensures all snapshot information is collected along the optimization trajectory, which avoids wasting samples in remote regions of the parameters space that are never visited, and inherently avoids the curse of dimensionality of sampling in a high-dimensional parameter space. At each iteration of the proposed algorithm, a reduced basis and empirical quadrature weights are constructed precisely to ensure the global convergence criteria of the trust-region method are satisfied, ensuring global convergence to a local minimum of the original (unreduced) problem. Numerical experiments are performed on two fluid shape optimization problems to verify the global convergence of the method and demonstrate its computational efficiency; speedups over 18× (accounting for all computational cost, even cost that is traditionally considered “offline” such as snapshot collection and data compression) relative to standard optimization approaches that do not leverage model reduction are shown.

On maximum residual nonlinear Kaczmarz-type algorithms for large nonlinear systems of equations

Recently, a class of nonlinear Kaczmarz (NK) algorithms has been proposed to solve large-scale nonlinear systems of equations. The NK algorithm is a generalization of the Newton–Raphson (NR) method and does not need to compute the entire Jacobian matrix. In this paper, we present a maximum residual nonlinear Kaczmarz (MRNK) algorithm for solving large-scale nonlinear systems of equations, which employs a maximum violation row selection and acts only on single rows of the entire Jacobian matrix at a time. Furthermore, we also establish the convergence theory of MRNK. In addition, inspired by the effectiveness of block Kaczmarz algorithms for solving linear systems, we further present a block MRNK (MRBNK) algorithm based on an approximate maximum residual criterion. Based on sketch-and-project technique and sketched Newton–Raphson method, we propose the deterministic sketched Newton–Raphson (DSNR) method which is equivalent to MRNBK, and then the global convergence theory of DSNR is established based on some assumptions and μ-strongly quasi-convex condition. Furthermore, the convergence theory of DSNR is provided under star-convex assumption. Finally, some numerical examples are tested to show the effectiveness of our new technique.

A Hybrid Conjugate Gradient Algorithm for Nonlinear System of Equations through Conjugacy Condition

the purpose of solving a large-scale system of nonlinear equations, a hybrid conjugate gradient algorithm is introduced in thispaper, based on the convex combination ofβFRkandβPRPkparameters. It is made possible by incorporating the conjugacy condition togetherwith the proposed conjugate gradient search direction. Furthermore, a significant property of the method is that through a non-monotone typeline search it gives a descent search direction. Under appropriate conditions, the algorithm establishes its global convergence. Finally, resultsfrom numerical tests on a set of benchmark test problems indicate that the method is more effective and robust compared to some existingmethods.

Secant Acceleration of Sequential Residual Methods for Solving Large-Scale Nonlinear Systems of Equations

Sequential Residual Methods try to solve nonlinear systems of equations $F(x)=0$ by iteratively updating the current approximate solution along a residual-related direction. Therefore, memory requirements are minimal and, consequently, these methods are attractive for solving large-scale nonlinear systems. However, the convergence of these algorithms may be slow in critical cases; therefore, acceleration procedures are welcome. In this paper, we suggest to employ a variation of the Sequential Secant Method in order to accelerate Sequential Residual Methods. The performance of the resulting algorithm is illustrated by applying it to the solution of very large problems coming from the discretization of partial differential equations.

On efficient matrix-free method via quasi-Newton approach for solving system of nonlinear equations

In this paper. a matrix-free method for solving large-scale system of nonlinear equations is presented. The method is derived via quasi-Newton approach, where the approximation to the Broyden's update is sufficiently done by constructing diagonal matrix using acceleration parameter. A fascinating feature of the method is that it is a matrix-free, so is suitable for solving large-scale problems. Furthermore, the convergence analysis and preliminary numerical results that is reported using a benchmark test problems, shows that the method is promising.

On the derivative-free quasi-Newton-type algorithm for separable systems of nonlinear equations

A derivative-free quasi-Newton-type algorithm in which its search direction is a product of a positive definite diagonal matrix and a residual vector is presented. The algorithm is simple to implement and has the ability to solve large-scale nonlinear systems of equations with separable functions. The diagonal matrix is simply obtained in a quasi-Newton manner at each iteration. Under some suitable conditions, the global and R-linear convergence result of the algorithm are presented. Numerical test on some benchmark separable nonlinear equations problems reveal the robustness and efficiency of the algorithm.

Solving Systems of Nonlinear Monotone Equations by Using a New Projection Approach

The projection technique is one of the famous method and highly useful to solve the optimization problems and nonlinear systems of equations. In this work, a new projection approach for solving systems of nonlinear monotone equation is proposed combining with the conjugate gradient direction because of their low storage. The new algorithm can be used to solve the large-scale nonlinear systems of equations and satisfy the sufficient descent condition. The new algorithm generates appropriate direction then employs a good line search along this direction to reach a new point. If this point solves the problem then the algorithm stops, otherwise, it constructs a suitable hyperplane that strictly separate the current point from the solution set. The next iteration is obtained by projection the new point onto the separating hyperplane. We proved that the line search of the new projection algorithm is well defined. Furthermore, we established the global convergence under some mild conditions. The numerical experiment indicates that the new method is effective and very well.

A Simple Conjugate Gradient Type Method for Solving Large-scale Systems of Nonlinear Equations

Abstract Quasi Newton’s method is among the most promising algorithm for solving systems of nonlinear equations. In this family, Broyden’s method update is the famous. This paper presents a simple conjugate gradient (CG) method for solving large-scale systems of nonlinear equations via memoryless Broyden’s approach. The attractive attribute of this method is due to its low memory requirements, global convergence properties and simple implementation. Under suitable conditions, the proposed method converges globally. Numerical performance of the proposed method are compared with the well-known conjugate gradients (CG) methods using benchmarks problems. The report demonstrate that the proposed method is reliable, efficient and competitive.

A Derivative-Free Liu–Storey Method for Solving Large-Scale Nonlinear Systems of Equations

In this paper, a descent Liu–Storey conjugate gradient method is extended to solve large-scale nonlinear systems of equations. Based on certain assumptions, the global convergence property is obtained with a nonmonotone line search. The proposed method is suitable to solve large-scale problems for the low-storage requirement. Numerical experiment results show that the new method is practically effective.

A gradient descent method for solving a system of nonlinear equations

This paper develops a gradient descent (GD) method for solving a system of nonlinear equations with an explicit formulation. We theoretically prove that the GD method has linear convergence in general and, under certain conditions, is equivalent to Newton’s method locally with quadratic convergence. A stochastic version of the gradient descent is also proposed for solving large-scale systems of nonlinear equations. Finally, several benchmark numerical examples are used to demonstrate the feasibility and efficiency compared to Newton’s method.

Efficient matrix-free direction method with line search for solving large-scale system of nonlinear equations

We proposed a matrix-free direction with an inexact line search technique to solve system of nonlinear equations by using double direction approach. In this article, we approximated the Jacobian matrix by appropriately constructed matrix-free method via acceleration parameter. The global convergence of our method is established under mild conditions. Numerical comparisons reported in this paper are based on a set of large-scale test problems and show that the proposed method is efficient for large-scale problems.

A Derivative-Free Decent Method Via Acceleration Parameter for Solving Systems of Nonlinear Equations

An algorithm for solving large-scale systems of nonlinear equations based on the transformation of the Newton method with the line search into a derivative-free descent method is introduced. Main idea used in the algorithm construction is to approximate the Jacobian by an appropriate diagonal matrix. Furthermore, the step length is calculated using inexact line search procedure. Under appropriate conditions, the proposed method is proved to be globally convergent under mild conditions. The numerical results presented show the efficiency of the proposed method.

A Dai\u2013Liao conjugate gradient method via modified secant equation for system of nonlinear equations

In this paper, we propose a Dai–Liao (DL) conjugate gradient method for solving large-scale system of nonlinear equations. The method incorporates an extended secant equation developed from modified secant equations proposed by Zhang et al. (J Optim Theory Appl 102(1):147–157, 1999) and Wei et al. (Appl Math Comput 175(2):1156–1188, 2006) in the DL approach. It is shown that the proposed scheme satisfies the sufficient descent condition. The global convergence of the method is established under mild conditions, and computational experiments on some benchmark test problems show that the method is efficient and robust.

A Projected Hybrid Conjugate Gradient Method for Solving Large-scale System of Nonlinear Equations

In this article, a fully derivative-free projected hybrid conjugate gradient method for solving large-scale systems of nonlinear equations is proposed. The proposed method is the convex combination of FR and PRP conjugate gradient methods with the projection method. However, the global convergence of the given method is established under suitable conditions with nonmonotone line search. Numerical results show that the method is efficient for large-scale problems.

Weighted Frobenius Norm Diagonal Quasi-Newton Method For Solving Large-Scale System of Nonlinear Equations

One of the major shortcomings of the quasi-Newton methods and some of theirs variants in solving systems of nonlinear equations is computing and storing the approximation of the Jacobian matrix or its inverse at each iteration. This paper, presents a new method based on weighted Frobenius norm, with choice of the weighting matrix to be the identity matrix. The proposed method are matrix and derivative free. The basic idea is to incorporate Leong et al., [13] update via diagonal updating by least change secant update strategy. This can be achieved by using weighted Frobenius norm. The local convergence analysis of the scheme is presented. Numerical results carried out on some benchmark test problems show that the proposed method is promising compared to some existing conjugate gradient methods.

Identification of hydraulic resistance parameters in hydraulic network model

The main purpose of the work is to develop and analyze new identification algorithm for multivariate multi-connected implicit system. Computational experiments and nonparametric statistics are implemented. The result of the work is a qualitatively new identification and output forecasting algorithm for multivariate systems. Based on the proposed algorithm a new class of generalized multivariate implicit nonparametric models is suggested. The significance of the research is subject to the fact that the majority of complex technical objects is usually described by large-scale nonlinear systems of equations up to input and output variables. Even if random effects in such models could be neglected a procedure of numerical solving the equation systems were either unstable or it would last too long to be implemented in forecasting of continuous process in the controlled object. Any delay in model calculation can also impact negatively on real-time control procedure. Using the proposed approach one can substitute solving the equation by estimation of the solution based on sample including measurements of input and output variables. Moreover, compared to counterparts, using the proposed approach it is possible to develop a single model for a range of input effects as it is presented in the research. Modelling and identification of multivariate systems is one of the most urgent tasks. Nowadays the development modelling methods does not keep up with the continuous and unlimited increase in the volume of information. To solve this problem, the authors proposed a new approach to constructing models of multidimensional systems. The scientific novelty of the article is as follows. A new modification of the nonparametric algorithm for identification of multidimensional systems has been proposed and tested. It makes possible to improve the accuracy and speed of computations.

An Accelerated Three-Term Conjugate Gradient Algorithm for Solving Large-Scale Systems of Nonlinear Equations

An Accelerated Three-Term Conjugate Gradient Algorithm for Solving Large-Scale Systems of Nonlinear Equations