Local Convergence Analysis Research Articles

Local Convergence Study for an Iterative Scheme with a High Order of Convergence

In this paper, we address a key issue in Numerical Functional Analysis: to perform the local convergence analysis of a fourth order of convergence iterative method in Banach spaces, examining conditions on the operator and its derivatives near the solution to ensure convergence. Moreover, this approach provides a local convergence ball, within which initial estimates lead to guaranteed convergence with details about the radii of domain of convergence and estimates on error bounds. Next, we perform a comparative study of the Computational Efficiency Index (CEI) between the analyzed scheme and some known iterative methods of fourth order of convergence. Our ultimate goal is to use these theoretical findings to address practical problems in engineering and technology.

Local properties and augmented Lagrangians in fully nonconvex composite optimization

A broad class of optimization problems can be cast in composite form, that is, considering the minimization of the composition of a lower semicontinuous function with a differentiable mapping. This paper investigates the versatile template of composite optimization without any convexity assumptions. First- and second-order optimality conditions are discussed. We highlight the difficulties that stem from the lack of convexity when dealing with necessary conditions in a Lagrangian framework and when considering error bounds. Building upon these characterizations, a local convergence analysis is delineated for a recently developed augmented Lagrangian method, deriving rates of convergence in the fully nonconvex setting.

Solving inverse obstacle scattering problem with latent surface representations

We propose a novel iterative numerical method to solve the three-dimensional inverse obstacle scattering problem of recovering the shape of an obstacle from far-field measurements. To address the inherent ill-posed nature of the inverse problem, we advocate the use of a trained latent representation of surfaces as the generative prior. This prior enjoys excellent expressivity within the given class of shapes, and meanwhile, the latent dimensionality is low, which greatly facilitates the computation. Thus, the admissible manifold of surfaces is realistic and the resulting optimization problem is less ill-posed. We employ the shape derivative to evolve the latent surface representation, by minimizing the loss, and we provide a local convergence analysis of a gradient descent type algorithm to a stationary point of the loss. We present several numerical examples, including also backscattered and phaseless data, to showcase the effectiveness of the proposed algorithm.

A One-Parameter Family of Methods with a Higher Order of Convergence for Equations in a Banach Space

The conventional approach of the local convergence analysis of an iterative method on Rm, with m a natural number, depends on Taylor series expansion. This technique often requires the calculation of high-order derivatives. However, those derivatives may not be part of the proposed method(s). In this way, the method(s) can face several limitations, particularly the use of higher-order derivatives and a lack of information about a priori computable error bounds on the solution distance or uniqueness. In this paper, we address these drawbacks by conducting the local convergence analysis within the broader framework of a Banach space. We have selected an important family of high convergence order methods to demonstrate our technique as an example. However, due to its generality, our technique can be used on any other iterative method using inverses of linear operators along the same line. Our analysis not only extends in Rm spaces but also provides convergence conditions based on the operators used in the method, which offer the applicability of the method in a broader area. Additionally, we introduce a novel semilocal convergence analysis not presented before in such studies. Both forms of convergence analysis depend on the concept of generalized continuity and provide a deeper understanding of convergence properties. Our methodology not only enhances the applicability of the suggested method(s) but also provides suitability for applied science problems. The computational results also support the theoretical aspects.

Local and semi-local convergence and dynamic analysis of a time-efficient nonlinear technique

The examination of nonlinear equations is essential in diverse domains such as science, business, and engineering because of the widespread occurrence of nonlinear phenomena. The primary obstacle in computational science is to create numerical algorithms that are both computationally efficient and possess a high convergence rate. This work tackles these problems by presenting a three-step nonlinear, time-efficient numerical method for solving nonlinear models. The selected method exhibits a convergence of sixth order and an efficiency index of 1.4310, with the added benefit of only requiring five function evaluations per iteration. This study diverges from earlier research by emphasizing the use of first-order derivatives instead of higher-order derivatives. As a result, the method becomes more versatile and applicable. It is possible to estimate solutions that are locally different in Banach spaces by looking at convergence on both local and semi-local levels. The stability and performance of the scheme are also checked by using polynomiography to do a visual analysis and compare it to other schemes in terms of convergence, speed, and CPU time.

Local convergence analysis of L1-ADI scheme for two-dimensional reaction-subdiffusion equation

Local convergence analysis of L1-ADI scheme for two-dimensional reaction-subdiffusion equation

Local convergence analysis of an inexact trust-region method for nonsmooth optimization

Local convergence analysis of an inexact trust-region method for nonsmooth optimization

Generalized Convergence for Multi-Step Schemes under Weak Conditions

We have developed a local convergence analysis for a general scheme of high-order convergence, aiming to solve equations in Banach spaces. A priori estimates are developed based on the error distances. This way, we know in advance the number of iterations required to reach a predetermined error tolerance. Moreover, a radius of convergence is determined, allowing for a selection of initial points assuring the convergence of the scheme. Furthermore, a neighborhood that contains only one solution to the equation is specified. Notably, we present the generalized convergence of these schemes under weak conditions. Our findings are based on generalized continuity requirements and contain a new semi-local convergence analysis (with a majorizing sequence) not seen in earlier studies based on Taylor series and derivatives which are not present in the scheme. We conclude with a good collection of numerical results derived from applied science problems.

Newton's method for generalized equations under weak conditions

A local convergence analysis is developed for Newton’s method in order to approximate a solution of a generalized equations in a Banach space setting. The convergence conditions are based on generalized continuity conditions on the Fr´echet derivative of the operator involved and the Aubin property. The specialized cases of our results extend earlier ones using similar information.

A class of pseudoinverse-free greedy block nonlinear Kaczmarz methods for nonlinear systems of equations

<abstract><p>In this research, we constructed a class of nonlinear greedy average block Kaczmarz methods to solve nonlinear problems without computing the Moore-Penrose pseudoinverse of the Jacobian matrix. These kinds of methods adopt the average technique of the Gaussian Kaczmarz method and combine the greedy strategy, which greatly reduces the amount of computation. The local convergence analysis and numerical experiments of the proposed methods are given. The numerical results show the effectiveness of the proposed methods.</p></abstract>

Generalized high-order iterative methods for solutions of nonlinear systems and their applications

<abstract><p>In this paper, we have constructed a family of three-step methods with sixth-order convergence and a novel approach to enhance the convergence order $ p $ of iterative methods for systems of nonlinear equations. Additionally, we propose a three-step scheme with convergence order $ p+3 $ (for $ p\geq3 $) and have extended it to a generalized $ (m+2) $-step scheme by merely incorporating one additional function evaluation, thus achieving convergence orders up to $ p+3m $, $ m\in\mathbb{N} $. We also provide a thorough local convergence analysis in Banach spaces, including the convergence radius and uniqueness results, under the assumption of a Lipschitz-continuous Fréchet derivative. Theoretical findings have been validated through numerical experiments. Lastly, the performance of these methods is showcased through the analysis of their basins of attraction and their application to systems of nonlinear equations.</p></abstract>

Local convergence analysis of frozen Steffensen-type methods under generalized conditions

The goal in this study is to present a unified local convergence analysis of frozen Steffensen-type methods under generalized Lipschitz-type conditions for Banach space valued operators. We also use our new idea of restricted convergence domains, where we find a more precise location, where the iterates lie leading to at least as tight majorizing functions. Consequently, the new convergence criteria are weaker than in earlier works resulting to the expansion of the applicability of these methods. The conditions do not necessarily imply the differentiability of the operator involved. This way our method is suitable for solving equations and systems of equations.

On the Newton-Broyden method for solving systems of nonlinear equations

The Newton-Broyden method for solving systems of nonlinear equations with operator decomposition is studied in this paper. A local convergence analysis is provided under classical Lipschitz conditions and results of numerical experiments are presented. The effectiveness of this method in solving various types of nonlinear systems is demonstrated.

Enhancing Equation Solving: Extending the Applicability of Steffensen-Type Methods

Local convergence analysis is mostly carried out using the Taylor series expansion approach, which requires the utilization of high-order derivatives, not iterative methods. There are other limitations to this approach, such as the following: the analysis is limited to finite-dimensional Euclidean spaces; no a priori computable error bounds on the distance or uniqueness of the solution results are provided. The local convergence analysis in this paper positively addresses these concerns in the more general setting of a Banach space. The convergence conditions involve only the operators in the methods. The more important semi-local convergence analysis not studied before is developed by using majorizing sequences. Both types of convergence analyses are based on the concept of generalized continuity. Although we study a certain class of methods, the same approach applies to extend the applicability of other schemes along the same lines.

A Two-Dimensional Variant of Newton’s Method and a Three-Point Hermite Interpolation: Fourth- and Eighth-Order Optimal Iterative Schemes

A nonlinear equation f(x)=0 is mathematically transformed to a coupled system of quasi-linear equations in the two-dimensional space. Then, a linearized approximation renders a fractional iterative scheme xn+1=xn−f(xn)/[a+bf(xn)], which requires one evaluation of the given function per iteration. A local convergence analysis is adopted to determine the optimal values of a and b. Moreover, upon combining the fractional iterative scheme to the generalized quadrature methods, the fourth-order optimal iterative schemes are derived. The finite differences based on three data are used to estimate the optimal values of a and b. We recast the Newton iterative method to two types of derivative-free iterative schemes by using the finite difference technique. A three-point generalized Hermite interpolation technique is developed, which includes the weight functions with certain constraints. Inserting the derived interpolation formulas into the triple Newton method, the eighth-order optimal iterative schemes are constructed, of which four evaluations of functions per iteration are required.

Convergence of Derivative-Free Iterative Methods with or without Memory in Banach Space

A method without memory as well as a method with memory are developed free of derivatives for solving equations in Banach spaces. The convergence order of these methods is established in the scalar case using Taylor expansions and hypotheses on higher-order derivatives which do not appear in these methods. But this way, their applicability is limited. That is why, in this paper, their local and semi-local convergence analyses (which have not been given previously) are provided using only the divided differences of order one, which actually appears in these methods. Moreover, we provide computable error distances and uniqueness of the solution results, which have not been given before. Since our technique is very general, it can be used to extend the applicability of other methods using linear operators with inverses along the same lines. Numerical experiments are also provided in this article to illustrate the theoretical results.

Local convergence analysis of augmented Lagrangian method for nonlinear semidefinite programming

Local convergence analysis of augmented Lagrangian method for nonlinear semidefinite programming

Development of Amari Alpha Divergence-Based Gradient-Descent Least Mean Square Algorithm

In this brief, a new least mean square (LMS) adaptive filtering algorithm based on Amari-Alpha information theoretic divergence is proposed which is named as Amari-Alpha LMS (AALMS). The local convergence and stability analysis show the upper bound of step size for tractable analysis. The steady-state performance of the proposed algorithm is analyzed, and the mean-square deviation (MSD) at steady-state is derived. Error-in-variable (EIV) model where input and desired signals both are corrupted with Gaussian noise is considered in this brief. Simulation results and computational complexity analysis show that the proposed AALMS algorithm performs better in comparison to well-known algorithms in terms of MSD in stationary and non-stationary scenarios. Comparison of different distance measures that can be derived from Amari-Alpha divergence is also carried out in terms of the MSD.

Local Stability and Convergence Analysis of Neural Network Controllers With Error Integral Inputs.

This article investigates the local stability and local convergence of a class of neural network (NN) controllers with error integrals as inputs for reference tracking. It is formally proved that if the input of the NN controller consists exclusively of error terms, the control system shows a non-zero steady-state error for any constant reference except for one specific point, for both single-layer and multi-layer NN controllers. It is further proved that adding error integrals to the input of the (single- and multi-layers) NN controller is one sufficient way to remove the steady-state error for any constant reference. Due to the nonlinearity of the NN controllers, the NN control systems are linearized at the equilibrium points. We provide proof that if all the eigenvalues of the linearized NN control system have negative real parts, local asymptotic stability and local exponential convergence are guaranteed. Two case studies were explored to verify the theoretical results: a single-layer NN controller in a 1-D system and a four-layer NN controller in a 2-D system applied to renewable energy integration. Simulations demonstrate that when NN controllers and the corresponding generalized proportional-integral (PI) controllers have the same eigenvalues, all control systems exhibit almost the same responses in a small neighborhood of their respective equilibrium points.

A Newton-like Midpoint Method for Solving Equations in Banach Space

The present paper includes the local and semilocal convergence analysis of a fourth-order method based on the quadrature formula in Banach spaces. The weaker hypotheses used are based only on the first Fréchet derivative. The new approach provides the residual errors, number of iterations, convergence radii, expected order of convergence, and estimates of the uniqueness of the solution. Such estimates are not provided in the approaches using Taylor expansions involving higher-order derivatives, which may not exist or may be very expensive or impossible to compute. Numerical examples, including a nonlinear integral equation and a partial differential equation, are provided to validate the theoretical results.