Quadratic Convergence Research Articles

An extended Ulm-like method for inverse singular value problems with multiple and/or zero singular values

In this paper, we propose an extended Ulm-like method for solving the inverse singular value problem (ISVP for short) with multiple and/or zero singular values. Compared with the Ulm-like method, the proposed method reduces the amount of calculations and is well-defined even when multiple and/or zero singular values appear. Under the nonsingularity assumption for the relative generalized Jacobian at a solution and by using a new technique, a convergence analysis is provided and the quadratic convergence is proved. Our results in the present paper improve and extend significantly the corresponding ones of Vong et al. (2011) and Shen et al. (2018) for the ISVP with distinct and positive singular values and/or with square matrices. In particular, we solve the interesting problem raised in Vong et al. (2011): whether the Ulm-like method and its convergence result can be extended to the cases of multiple singular values and of zero singular values. Numerical results reveal that the extended Ulm-like method is an efficient algorithm for the ISVPs even when multiple and/or zero singular values appear.

Newton’s Method Without Division

Newton’s Method for root-finding is modified to avoid the division step while maintaining quadratic convergence.

A layer-wise plasticity-based approach for the analysis of reinforced concrete shell structures using a mixed finite element

The analysis of reinforced concrete shell structures accounting for material nonlinearity is addressed. The structural response is numerically evaluated using a mixed shell finite element and a plasticity-based material behaviour. The finite element is a quadrilateral with four nodes and is based on self-equilibrated assumed stresses. The kinematic fields are interpolated only along the element boundary by polynomials up to the third order. The generalised shell stresses are evaluated through layer-wise integration of the Cauchy pointwise stresses. This allows appropriated three-dimensional elastoplastic constitutive equation to be employed and to include multiple reinforcing layers. The integration of the constitutive laws is performed using a dual decomposition method which preserves the assumed stress interpolation. Results, including civil engineering applications, show that the proposed approach is reliable and accurate in evaluating the static nonlinear equilibrium path. Additionally, the mixed finite element shows a quadratic rate of convergence in the collapse load and low error even for coarse meshes.

A Deforming Mixed-Hybrid Finite Element Model for Robust Groundwater Flow Simulation in 3D Unconfined Aquifers with Unstructured Layered Grids

Determining the water table shape and position in unconfined aquifers is fundamental to many groundwater flow assessment studies. The commonly used industry-standard fixed mesh models, contrary to popular belief, do not provide an accurate description of the phreatic surface. When using such models, the water table position is post-processed from the simulated groundwater heads, leading to an approximation error. This error becomes larger for coarse vertical grids. This paper introduces a novel moving mesh technique to simulate the groundwater table in three-dimensional unconfined aquifers under steady-state or transient conditions. We adopt the face-based mixed-hybrid finite element discretization approach in space, leading to a more accurate approximation of the specific discharge field. The model uses an adaptive unstructured but layered mesh which is iteratively adjusted until its top fits the phreatic surface. The developed algorithm accounts for a linearized form of the kinematic boundary condition prescribed on the moving boundary and also supports usual boundary conditions as well. The model was compared to the existing analytical, fixed mesh, and previously published solutions. The obtained results show that the developed model is superior in terms of its numerical stability, convergence behavior, and accuracy. Furthermore, the simulated phreatic surface is free from a cellwise interpolation error and independent of the vertical grid size as used in fixed mesh methods. We also found that the robustness of the moving mesh method cannot be surpassed by a fixed mesh alternative. The model’s efficiency is supported by an almost quadratic rate of convergence of the outer iteration loop.

An efficient finite strain constitutive model for amorphous thermoplastics: Fully implicit computational implementation and optimization-based parameter calibration

In this paper, a new visco-elastic visco-plastic (three-dimensional) constitutive model is consistently formulated under isothermal conditions to describe the nonlinear behavior of thermoplastic polymers at finite strains. The constitutive equations include three well-established rheological elements to have well-defined parametric behavior and capture the experimental response. In particular, a visco-elastic generalized Maxwell element, a visco-plastic Eyring dashpot, and a plasticity-induced (nonlinear) hardening element are selected and consistently combined. A fully implicit integration algorithm is derived, and highly efficient implementation is obtained by simplifying the system of equations to a single (scalar) nonlinear residual equation. A new two-stage optimization-based calibration procedure is developed to identify the material parameters in a completely unsupervised way. The constitutive model is validated against results available in the literature for polycarbonate, accounting for temperature and strain rate dependencies. The results show the efficiency of the overall numerical strategy and demonstrate that quadratic convergence rates are attained. Despite the strongly nonlinear finite strain response, it is possible to observe an excellent agreement for all stages of deformation.

Discriminative subspace learning via optimization on Riemannian manifold

Discriminative subspace learning is an important problem in machine learning, which aims to find the maximum separable decision subspace. Traditional Euclidean-based methods usually use Fisher discriminant criterion for finding an optimal linear mapping from a high-dimensional data space to a lower-dimensional subspace, which hardly guarantee a quadratic rate of global convergence and suffers from the singularity problem. Here, we propose the manifold optimization-based discriminant analysis (MODA) which is constructed by using the latent subspace alignment and the geometry of objective function with orthogonality constraint. MODA is solved by using Riemannian version of trust-region algorithm. Experimental results on various image datasets and electroencephalogram (EEG) datasets show that MODA achieves the best separability and is significantly superior to the competing algorithms. Especially for the time series of EEG signals, the accuracy of MODA is 20–30% higher than existing algorithms. The code for MODA is available at https://github.com/ncclabsustech/MODA-algorithm.

A New Adaptive Accelerated Levenberg–Marquardt Method for Solving Nonlinear Equations and Its Applications in Supply Chain Problems

In this paper, a new adaptive Levenberg–Marquardt method is proposed to solve the nonlinear equations including supply chain optimization problems. We present a new adaptive update rule which is a segmented function on the ratio between the actual and predicted reductions of the objective function to accept a large number of unsuccessful iterations and avoid jumping in local areas. The global convergence and quadratic convergence of the proposed method are proved by using the trust region technique and local error bound condition, respectively. In addition, we use the proposed algorithm to test on the symmetric and asymmetric linear equations. Numerical results show that the proposed method has good numerical performance and development prospects. Furthermore, we apply the algorithm to solve the fresh agricultural products supply chain optimization problems.

Convergence Analysis of Newton–Schur Method for Symmetric Elliptic Eigenvalue Problem

In this paper, we consider the Newton–Schur method in Hilbert space and obtain quadratic convergence. For the symmetric elliptic eigenvalue problem discretized by the standard finite element method and nonoverlapping domain decomposition method, we use the Steklov–Poincaré operator to reduce the eigenvalue problem on the domain into the nonlinear eigenvalue subproblem on , which is the union of subdomain boundaries. We prove that the convergence rate for the Newton–Schur method is , where the constant is independent of the fine mesh size and coarse mesh size , and and are errors after and before one iteration step, respectively. For one specific inner product on , a sharper convergence rate is obtained, and we can prove that . Numerical experiments confirm our theoretical analysis.

A robust stress update algorithm for elastoplastic models without analytical derivation of the consistent tangent operator and loading/unloading estimation

AbstractA robust and concise implicit stress integration algorithm of elastoplastic models is presented. It does not require the loading/unloading estimation and analytical derivation operation for the stress update. First, the elastoplastic stress update problem is recast into an unconstrained minimization problem by utilizing the smooth function to bypass the loading/unloading estimation. Then, the object problem is solved by the line search method instead of the Newton method for better convergence. The consistent tangent operator is evaluated by the complex step derivative approximation without the subtraction cancellation error, which provides the quadratic convergence rate of global iteration. The rationality of the numerical consistent tangent operator is validated by the one obtained by the analytical derivation. A recently developed non‐orthogonal elastoplastic (NEP) clay model is implemented using the new algorithm. The algorithm is confirmed through comparing the numerical solution and the analytical one for a cavity expansion problem. The algorithm performance is assessed based on a series of geotechnical boundary value problems. It is found that the new algorithm is more robust than the one employed by ABAQUS. The source code of the model implementation can be downloaded from https://github.com/zhouxin615.

Sparse least squares solutions of multilinear equations

In this paper, we propose a sparse least squares (SLS) optimization model for solving multilinear equations, in which the sparsity constraint on the solutions can effectively reduce storage and computation costs. By employing variational properties of the sparsity set, along with differentiation properties of the objective function in the SLS model, the first-order optimality conditions are analysed in terms of the stationary points. Based on the equivalent characterization of the stationary points, we propose the Newton Hard-Threshold Pursuit (NHTP) algorithm and establish its locally quadratic convergence under some regularity conditions. Numerical experiments conducted on simulated datasets including cases of Completely Positive(CP)-tensors and symmetric strong M-tensors illustrate the efficiency of our proposed NHTP method.

A New Modified Levenberg–Marquardt Method for Systems of Nonlinear Equations

Taking a new choice of the LM parameter λ k = μ k J k T F k δ with δ ∈ 0,2 , we give a new modified Levenberg–Marquardt method. Under the error bound condition c dist w , S ≤ J w T F w which is weaker than the nonsingular of Jacobian J w , we present that the new Levenberg–Marquardt method has at least superlinear convergence when δ ∈ 0,1 and quadratic convergence when δ ∈ 1,2 , respectively, which indicates that our new Levenberg–Marquardt method is performed for the systems of nonlinear equations. Also, numerical experiments indicate its efficiency on a set of systems of nonlinear equations.

A combination of the quasilinearization method and linear barycentric rational interpolation to solve nonlinear multi-dimensional Volterra integral equations

In this paper, an iterative scheme including a combination of the quasilinearization technique and multi-dimensional linear barycentric rational interpolation is applied to solve nonlinear multi-dimensional Volterra integral equations. First, employing the quasilinearization method, the nonlinear multi-dimensional Volterra integral equation is reduced to a sequence of linear Volterra integral equations. Under appropriate assumptions, the constructed iterative sequence is uniformly convergent to the unique solution of the problem. In general, finding an analytical solution for linear Volterra integral equations is impossible. Hence, in each iteration, using a collocation method based on multi-dimensional barycentric rational basis functions, the solution to the linear integral equation is approximated. The quadratic convergence of the quasilinearization approach and the error estimation of the combined method are investigated theoretically. In the end, the efficiency and the validity of this method are illustrated with some numerical examples and compared with those of the existing numerical methods.

A new full-Newton step feasible interior point method for convex quadratic programming

In this paper, we propose and analyse a new full-Newton step feasible interior point method for convex quadratic programming. The basic idea of this method is to replace a complementarity condition by a non-negative variable weight vector. With a zero of weight vector, the limit of the weighted path exists and satisfies the complementarity condition, the limit yields an optimal solution of problem. In each main iteration of the new algorithm consisted of only full-Newton steps with a quadratic rate of convergence. The advantage of this method is the use of a full-Newton step, that is no calculation of the step size is required. Finally, some numerical results are reported to show the practical performance of the proposed algorithm with different parameters.

Computational Approach for a Singularly Perturbed Differential Equations With Mixed Shifts Using a Non-Polynomial Spline

In this paper, a second order singularly perturbed differential difference equation with both the negative and positive shifts is considered. A fitted non-polynomial spline approach is applied to solve the problem. Taylor series expansion process is being used to produce an approximated form of the considered problem, and then a fitted non-polynomial spline approach is devised in the form of a three-term recurrence relation. The convergence of the method is examined, and a quadratic rate of convergence is achieved. The maximum absolute error with quadratic rate of convergence of the solution is recorded. Layer profile is examined using the graphs.

Low regularity exponential-type integrators for the “good” Boussinesq equation

Abstract In this paper, two semidiscrete low regularity exponential-type integrators are proposed and analyzed for the “good” Boussinesq equation, including a first-order integrator and a second-order one. Compared to the existing numerical methods, the convergence rate can be achieved under weaker regularity assumptions on the exact solution. Specifically, the first-order integrator is convergent linearly in $H^r$ for solutions in $H^{r+1}$ if $r>1/2$, i.e., the boundedness of one additional derivative of the solution is required to achieve the first-order convergence. When $r>7/6$, we can even prove linear convergence in $H^r$ for solutions in $H^{r+2/3}$. What’s more, half-order convergence is established in $H^{r}(r>3/2)$ for any solutions in $H^r$, i.e., no additional smoothness assumptions are needed. For the second-order integrator, the quadratic convergence in $H^{r}$$(r>1/2)$ (or $L^2$) is demonstrated, when the solutions belong to $H^{r+2}$ (or $H^{9/4}$). Numerical examples illustrating the convergence analysis are included. A comparison with other methods demonstrates the superiority of the newly proposed exponential-type integrators for rough data.

Majorization-minimization-based Levenberg–Marquardt method for constrained nonlinear least squares

A new Levenberg–Marquardt (LM) method for solving nonlinear least squares problems with convex constraints is described. Various versions of the LM method have been proposed, their main differences being in the choice of a damping parameter. In this paper, we propose a new rule for updating the parameter so as to achieve both global and local convergence even under the presence of a convex constraint set. The key to our results is a new perspective of the LM method from majorization-minimization methods. Specifically, we show that if the damping parameter is set in a specific way, the objective function of the standard subproblem in LM methods becomes an upper bound on the original objective function under certain standard assumptions. Our method solves a sequence of the subproblems approximately using an (accelerated) projected gradient method. It finds an varepsilon-stationary point after O(varepsilon ^{-2}) computation and achieves local quadratic convergence for zero-residual problems under a local error bound condition. Numerical results on compressed sensing and matrix factorization show that our method converges faster in many cases than existing methods.

A new hybrid algorithm for maximum likelihood estimation in a model of accident frequencies

In this paper, we are interested in the numerical computation of the constrained maximum likelihood estimator (MLE) of the parameter vector of a discrete statistical model used in statistics applied to road safety. The parameter vector is divided into two blocks: one block with the parameter of interest and the second block with secondary parameters. The MLE is the solution to a system of non-linear implicit equations difficult to solve in closed-form. To overcome this difficulty, we propose a hybrid algorithm (HA) mixing the use of a one-dimensional Newton-Raphson (NR) algorithm for the first equation of the system and a fixed-point strategy for the remaining equations. Our proposed algorithm involves no matrix inversion but it partially enjoys the quadratic convergence rate of the one-dimensional NR algorithm. We illustrate its performance on simulated data and we compare it to Newton-Raphson (NR) and quasi-Newton algorithms which are two of the most used optimization algorithms. The results suggest that our HA outperforms NR and quasi-Newton algorithms. It is accurate and converges quickly for all the starting values.

An efficient spectral minimization of the Dai-Yuan method with application to image reconstruction

<abstract><p>In this paper, a spectral Dai and Yuan conjugate gradient (CG) method is proposed based on the generalized conjugacy condition for large-scale unconstrained optimization, in which the spectral parameter is motivated by some interesting theoretical features of quadratic convergence associated with the Newton method. Accordingly, utilizing the strong Wolfe line search to yield the step-length, the search direction of the proposed spectral method is sufficiently descending and converges globally. By applying some standard Euclidean optimization test functions, numerical results reports show the advantage of the method over some modified Dai and Yuan CG schemes in literature. In addition, the method also shows some reliable results, when applied to solve an image reconstruction model.</p></abstract>

A significant improvement of a family of secant-type methods

Secant-type iterative methods are usually used to solve nonlinear systems of equations where the operator involved is nondifferentiable or its derivative is costly. From a uniparametric family of secant-type methods, that is reduced to the secant method and Newton’s method for some particular values of the parameter involved, with operational cost similar to that of the secant and Newton methods and superlinear convergence, we construct a biparametric family of iterative methods with quadratic convergence, study their local convergence, their efficiency and, based on the dynamics of the methods, their accessibility.

On the fractional Newton method with Caputo derivatives

UDK 519.615.5 MSC: 26A33 65H05 DOI: 10.21538/0134-4889-2022-28-4-273-276 Newton's method is commonly used to solve nonlinear algebraic equations due to its quadratic rate of convergence in the vicinity of the root. Multiple modifications of Newton's method are known, some lead to more stable calculations, although often at the expense of the rate of convergence.