Large-scale Nonlinear Equations Research Articles

A new approximate descent derivative-free algorithm for large-scale nonlinear symmetric equations

A new approximate descent derivative-free algorithm for large-scale nonlinear symmetric equations

Two-step inertial derivative-free projection method for solving nonlinear equations with application

Large-scale systems of nonlinear equations play a fundamental role in various applications, spanning from differential equations to economics, engineering, management science, probability theory, and various other applied sciences. This paper proposes a derivative-free iterative method to find ϵ-approximate solutions of large-scale nonlinear equations. The proposed scheme combines the hyperplane projection technique with a two-step inertial extrapolation, presenting a new approach. The proposed method does not require Jacobian information and thus can be applied to solve nonsmooth equations. Under standard assumptions, we prove the proposed method’s global convergence and nonasymptotic O(1/k) convergence rate. Furthermore, we include numerical results and comparisons to demonstrate the efficiency of the proposed method. Finally, we demonstrate the applicability of the proposed method in solving regularized decentralized logistic regression, a popular problem in machine learning applications.

A Modified Three-Term Conjugate Descent Derivative-Free Method for Constrained Nonlinear Monotone Equations and Signal Reconstruction Problems

Iterative methods for solving constraint nonlinear monotone equations have been developed and improved by many researchers. The aim of this research is to present a modified three-term conjugate descent (TTCD) derivative-free method for constrained nonlinear monotone equations. The proposed algorithm requires low storage memory; therefore, it has the capability to solve large-scale nonlinear equations. The algorithm generates a descent and bounded search direction dk at every iteration independent of the line search. The method is shown to be globally convergent under monotonicity and Lipschitz continuity conditions. Numerical results show that the suggested method can serve as an alternative to find the approximate solutions of nonlinear monotone equations. Furthermore, the method is promising for the reconstruction of sparse signal problems.

Extended modified three-term conjugate gradient method for large-scale nonlinear equations

<p>In this research paper, we introduce a novel gradient-free modified three-term conjugate gradient method designed to solve nonlinear equations subject to convex constraints. Our approach incorporates the projection scheme, which enhances the effectiveness of the proposed method. Building upon the modified three-term conjugate gradient method for solving M-tensor systems and ℓ1- norm-based nonsmooth optimization problems, our method can be regarded as an extension of their technique. By making mild assumptions, we establish the theoretical convergence properties of our iterative method. Through extensive numerical experiments, we demonstrate that our proposed approach is not only highly efficient but also outperforms other existing methods in terms of performance and accuracy.</p>

A Reduced Order Model Based on ANN-POD Algorithm for Steady-State Neutronics and Thermal-Hydraulics Coupling Problem

The neutronics and thermal-hydraulics (N/TH) coupling behavior analysis is a key issue for nuclear power plant design and safety analysis. Due to the high-dimensional partial differential equations (PDEs) derived from the N/TH system, it is usually time consuming to solve such a large-scale nonlinear equation by the traditional numerical solution method of PDEs. To solve this problem, this work develops a reduced order model based on the proper orthogonal decomposition (POD) and artificial neural networks (ANNs) to simulate the N/TH coupling system. In detail, the POD method is used to extract the POD modes and corresponding coefficients from a set of full-order model results under different boundary conditions. Then, the backpropagation neural network (BPNN) is utilized to map the relationship between the boundary conditions and POD coefficients. Therefore, the physical fields under the new boundary conditions could be calculated by the predicated POD coefficients from ANN and POD modes from snapshot. In order to assess the performance of an ANN-POD-based reduced order method, a simplified pressurized water reactor model under different inlet coolant temperatures and inlet coolant velocities is utilized. The results show that the new reduced order model can accurately predict the distribution of the physical fields, as well as the effective multiplication factor in the N/TH coupling nuclear system, whose relative errors are within 1%.

A modified spectral gradient projection-based algorithm for large-scale constrained nonlinear equations with applications in compressive sensing

To solve large-scale nonlinear equations with convex constraints efficiently and overcome the shortcomings of other algorithms, such as large storage and high complexity, we develop a descent and derivative-free algorithm based on a classical spectral gradient method, the projection technique and a new monotone line search method. The new algorithm has the following features: (1) There is lower storage and derivative-free information; (2) It is suitable to solve large-scale nonlinear equations; (3) Its direction possesses sufficient descent and trust region properties; (4) The spectral parameter is bounded. The global convergence of the new algorithm is proven under some appropriate assumptions. If the local error bound condition is satisfied, the proposed algorithm is linearly convergent. Preliminary numerical results demonstrate that the new algorithm is effective and robust. Furthermore, we also extend this algorithm to solve the sparse signal reconstruction problem and the blurred image recovery problem in compressed sensing.

An efficient conjugate gradient-based algorithm for unconstrained optimization and its projection extension to large-scale constrained nonlinear equations with applications in signal recovery and image denoising problems

In this paper, a hybrid version of the Rivaie–Mustafa–Ismail–Leong, RMIL for short, conjugate gradient method is proposed for unconstrained optimization problem. The search direction fulfills the sufficient descent condition and trust region property at each iteration, independent of any line search. We establish the global convergence of the proposed algorithm under normal assumptions and Wolfe line search. To move forward, incorporating with derivative-free projection technique, the proposed conjugate coefficient is extended to deal with nonlinear monotone equations. Without Lipschitz continuity assumption, the global convergence is proved with mild conditions. The numerical results for both unconstrained optimization instances and large-scale nonlinear equations prove the efficiency of the proposed methods with less computation cost. In particular, we also explore the practical applications on signal recovery and image denoising problems.

Sketched Newton--Raphson

We propose a new globally convergent stochastic second-order method. Our starting point is the development of a new sketched Newton--Raphson (SNR) method for solving large scale nonlinear equations of the form $F(x)=0$ with $F:\mathbb{R}^p \rightarrow \mathbb{R}^m$. We then show how to design several stochastic second-order optimization methods by rewriting the optimization problem of interest as a system of nonlinear equations and applying SNR. For instance, by applying SNR to find a stationary point of a generalized linear model, we derive completely new and scalable stochastic second-order methods. We show that the resulting method is very competitive as compared to state-of-the-art variance reduced methods. Furthermore, using a variable splitting trick, we also show that the stochastic Newton method (SNM) is a special case of SNR and use this connection to establish the first global convergence theory of SNM. We establish the global convergence of SNR by showing that it is a variant of the online stochastic gradient descent (SGD) method, and then leveraging proof techniques of \textttSGD. As a special case, our theory also provides a new global convergence theory for the original Newton--Raphson method under strictly weaker assumptions as compared to the classic monotone convergence theory.

A unified derivative-free projection method model for large-scale nonlinear equations with convex constraints

<p style='text-indent:20px;'>Motivated by recent derivative-free projection methods proposed in the literature for solving nonlinear constrained equations, in this paper we propose a unified derivative-free projection method model for large-scale nonlinear equations with convex constraints. Under mild conditions, the global convergence and convergence rate of the proposed method are established. In order to verify the feasibility and effectiveness of the model, a practical algorithm is devised and the corresponding numerical experiments are reported, which show that the proposed practical method is efficient and can be applied to solve large-scale nonsmooth equations. Moreover, the proposed practical algorithm is also extended to solve the obstacle problem.</p>

An efficient projection-based algorithm without Lipschitz continuity for large-scale nonlinear pseudo-monotone equations

In this paper, an efficient algorithm is proposed for solving convex constrained equations. The method has the following characteristics: (i) its search direction satisfies the sufficient descent property which is independent of any line search conditions; (ii) the equations only satisfy pseudo-monotone property; (iii) its global convergence is proved without the Lipschitz continuity. We compare our numerical results for test problems with those of other derivative-free projection methods.

A derivative-free three-term Hestenes–Stiefel type method for constrained nonlinear equations and image restoration

In this paper, a derivative-free Hestenes–Stielfel type method is proposed to solve large-scale nonlinear equations with convex constraints. The proposed method adopts the line search proposed by Ou and Li [J. Comput. Appl. Math. 56(1-2) (2018), pp. 195–216]. Unlike most existing methods, the global convergence of the proposed method is established under the assumption that the underlying mapping is Lipschitz continuous and satisfies a weaker monotonicity condition. Preliminary numerical experiments indicate that the proposed method is effective and promising. Furthermore, the proposed method is used to solve image restoration problem in compressive sensing.

A Modified Conjugate Gradient Method for Solving Large-Scale Nonlinear Equations

In this paper, we propose a modified Polak–Ribière–Polyak (PRP) conjugate gradient method for solving large-scale nonlinear equations. Under weaker conditions, we show that the proposed method is globally convergent. We also carry out some numerical experiments to test the proposed method. The results show that the proposed method is efficient and stable.

PRP-like algorithm for monotone operator equations

Many studies have been devoted to develop and improve the iterative methods for solving convex constraint nonlinear equations problem (CCP). Based on the projection technique, we introduce a derivative-free method for approximating the solution of CCP. The proposed method is suitable for solving large-scale nonlinear equations due to its lower storage requirements. The directions generated by the proposed method at every iteration are bounded. Under some mild conditions, we establish the global convergence result of the proposed method. Numerical experiments are provided to show the efficiency of the method in solving CCP. Moreover, we tested the capability of the method in solving the monotone nonlinear operator equation equivalent to the $$\ell _1$$ -norm regularized minimization problem.

Multiwave interaction solutions for a (3 + 1)-dimensional generalized BKP equation

ABSTRACT The direct algebraic method, together with the inheritance solving strategy, is utilized to construct multiwave interaction solutions among solitons, rational waves and periodic waves for a -dimensional generalized BKP equation. In addition, an N-soliton decomposition algorithm derived from the simplified Hirota method, the conjugated parameters assignment and long-wave limit techniques is introduced. By the N-soliton decomposition algorithm, higher-order interaction solutions among solitons, breathers and lump waves for this equation are generated. The highlight of N-soliton decomposition algorithm is that it bypasses the solving of (super) large-scale nonlinear algebraic equations, so it can be used to obtain much higher-order multiwave interaction solutions.

A Modified PRP-CG Type Derivative-Free Algorithm with Optimal Choices for Solving Large-Scale Nonlinear Symmetric Equations

Inspired by the large number of applications for symmetric nonlinear equations, this article will suggest two optimal choices for the modified Polak–Ribiére–Polyak (PRP) conjugate gradient (CG) method by minimizing the measure function of the search direction matrix and combining the proposed direction with the default Newton direction. In addition, the corresponding PRP parameters are incorporated with the Li and Fukushima approximate gradient to propose two robust CG-type algorithms for finding solutions for large-scale systems of symmetric nonlinear equations. We have also demonstrated the global convergence of the suggested algorithms using some classical assumptions. Finally, we demonstrated the numerical advantages of the proposed algorithms compared to some of the existing methods for nonlinear symmetric equations.

A generalized hybrid CGPM-based algorithm for solving large-scale convex constrained equations with applications to image restoration

In this paper, by improving a line search criterion to yield steplength, and designing a hybrid conjugate parameter to construct sufficient descent direction, we propose a generalized hybrid conjugate gradient projection method for solving large-scale monotone nonlinear equations with convex constraints. For the proposed method, we prove its global convergence under some mild conditions, and study its convergence rate. Numerical comparisons with two existing methods show that our method is promising for solving large-scale nonlinear constrained equations. Furthermore, the experiment results of dealing with image restoration problems also verify that the proposed method is effective.

Derivative-free SMR conjugate gradient method for constraint nonlinear equations

Based on the SMR conjugate gradient method for unconstrained optimization proposed by Mohamed et al. [N. S. Mohamed, M. Mamat, M. Rivaie, S. M. Shaharuddin, Indones. J. Electr. Eng. Comput. Sci., \(\bf 11\) (2018), 1188--1193] and the Solodov and Svaiter projection technique, we propose a derivative-free SMR method for solving nonlinear equations with convex constraints. The proposed method can be viewed as an extension of the SMR method for solving unconstrained optimization. The proposed method can be used to solve large-scale nonlinear equations with convex constraints because of derivative-free and low storage. Under the assumption that the underlying mapping is Lipschitz continuous and satisfies a weaker monotonicity assumption, we prove its global convergence. Preliminary numerical results show that the proposed method is promising.

Deep learning algorithms for temperature field reconstruction of nonlinear tomographic absorption spectroscopy

Nonlinear tomography can be combined with line-of-sight measurement techniques and take the full advantage of the nonlinear relationship between the target fields to be reconstructed and the measured projections. For example, it has been integrated with absorption spectroscopy for simultaneous imaging of temperature and species concentration, and the technique is referred to as nonlinear tomographic absorption spectroscopy (NTAS) which is especially suitable to applications where the optical access is extremely limited. However, the major drawback of nonlinear tomography is its high computational costs for the inversion process, which involves the solution of a large-scale nonlinear equation system. The situation becomes more severe when thousands of tomographic frames need to be processed. This limitation can be potentially overcome by applying a deep learning algorithm, which can build efficient mapping between the projections and the target fields for rapid reconstruction. Nevertheless, only preliminary study of convolutional neural networks was reported so far, and there are also other well-established methods which have not been investigated yet. In this work, we cite NTAS as an example and demonstrate the reconstruction of temperature field using deep brief network (DBN) and recurrent neural network (RNN) for the first time. This work also aims to provide systematic comparative studies between these representative algorithms. As expected, all the deep learning algorithms are very efficient in solving NTAS problems, and a typical reconstruction time is on the order of milliseconds. In addition, the results suggest that, in NTAS problem, RNN is superior to the other methods in terms of both accuracy and noise immunity, and it is more favorable for practical applications.

A Cauchy Point Direction Trust Region Algorithm for Nonlinear Equations

In this paper, a Cauchy point direction trust region algorithm is presented to solve nonlinear equations. The search direction is an optimal convex combination of the trust region direction and the Cauchy point direction with the sufficiently descent property and the automatic trust region property. The global convergence of the proposed algorithm is proven under some conditions. The preliminary numerical results demonstrate that the proposed algorithm is promising and has better convergence behaviors than the other two existing algorithms for solving large-scale nonlinear equations.

Some modified Hestenes-Stiefel conjugate gradient algorithms with application in image restoration

It is efficient to use the Hestenes–Stiefe (HS) conjugate gradient algorithm in solving large-scale complex smooth optimization problems because of its simplicity and low calculation requirements. Additionally, this algorithm has been used to simultaneously solve large-scale nonsmooth problems, nonlinear equations, and practical application. In this paper, the authors propose some modified HS conjugate gradient algorithms that not only address large-scale nonlinear equations and nonsmooth convex problems but also have the following properties. i) The algorithms portray a decreasing trait and trust-region property without any additional condition. ii) It combines the steepest descent algorithm with the conjugate gradient algorithm. iii) They employ the global convergence theory and iv) can be successfully used to solve nonlinear optimization problems and image restoration.