Approximate Newton Method Research Articles

An improved approximate Newton method for implicit Runge–Kutta formulas

Implicit Runge–Kutta (IRK) methods (such as the s -stage Radau IIA method with s = 3 , 5 , or 7) for solving stiff ordinary differential equation systems have excellent stability properties and high solution accuracy orders, but their high computing costs in solving their nonlinear stage equations have seriously limited their applications to large scale problems. To reduce such a cost, several approximate Newton algorithms were developed, including a commonly used one called the simplified Newton method. In this paper, a new approximate Jacobian matrix and two new test rules for controlling the updating of approximate Jacobian matrices are proposed, yielding an improved approximate Newton method. Theoretical and numerical analysis show that the improved approximate Newton method can significantly improve the convergence and performance of the simplified Newton method.

Complexity Pursuit for Unifying Model

Complexity pursuit is an extension of projection pursuit to time series data and the method is closely related to blind separation of time-dependent source signals and independent component analysis. The goal is to find projections of time series that have interesting structure, defined using criteria related to Kolmogoroff complexity or coding length. In this paper, we first derive a simple approximation of coding length for unifying model that takes into account nongaussianity of sources, their autocorrelations and their smoothly changing nonstationary variances. Next, a fixed-point algorithm is proposed by using approximate Newton method. Finally, simulations verify the fixed-point algorithm converges faster than the existing gradient algorithm and it is more simple to implement due to it does not need any learning rate.

Use of Bilateral Filters for Super-Resolution of Single Image with Iterations for Minimizing Reconstruction Errors

The use of a nonlinear filter for super-resolution of a single image is presented. A nonlinear iterative solution procedure is derived from an approximate Newton method for minimizing errors in low-resolution images reconstructed from estimated high-resolution images. We use the bilateral filter for image smoothing in the reconstruction steps. Our method was shown to produce high-resolution images without halos around edges and with enhanced fine textures.

Convergence analysis of nonsmooth equations for the general nonlinear complementarity problem

This paper deals with a general nonlinear complementarity problem, where the underlying functions are assumed to be continuous. Based on a nonlinear complementarity function, it is transformed into a system of nonsmooth equations. Then, two kinds of approximate Newton methods for the nonsmooth equations are developed and their convergence are proved. Finally, numerical tests are also listed.

Differential material matrices for the finite integration technique

In this paper, differential conductivity and reluctivity matrices are constructed for formulations discretised by the finite integration technique and linearised by the Newton method. It turns out that these matrices are nondiagonal and unsymmetric, even in the case that orthogonal grids are used. A diagonal approximation of the differential matrices leads to an approximate Newton method where cross-magnetisation effects are no longer considered, except in the update between the successive nonlinear iteration steps. The approximate method does not guarantee second order convergence but outperforms the true Newton method when only a moderate accuracy is required.

Fluid-structure interaction with applications in biomechanics

In bioengineering applications problems of flow interacting with elastic solid are very common. We formulate the problem of interaction for an incompressible fluid and an incompressible elastic material in a fully coupled arbitrary Lagrangian–Eulerian (ALE) formulation. The mathematical description and the numerical schemes are designed in such a way that more complicated constitutive relations (and more realistic for bioengineering applications) can be incorporated easily. The whole domain of interest is treated as one continuum and the same discretization in space (FEM) and time (Crank–Nicholson) is used for both, solid and fluid, parts. The resulting nonlinear algebraic system is solved by an approximate Newton method. The combination of second order discretization and fully coupled solution method gives a method with high accuracy and robustness. To demonstrate the flexibility of this numerical approach we apply the same method to a mixture-based model of an elastic material with perfusion which also falls into the category of fluid structure interactions. A few simple example calculations with simple material models and large deformations of the solid part are presented.

An approximate Newton method for non-smooth equations with finite max functions

A new version of finite difference approximation of the generalized Jacobian for a finite max function is constructed. Numerical results are reported for the generalized Newton methods using this approximation.

Efficient subspace-based algorithm for adaptive bearing estimation and tracking

In some practical applications of array processing, the directions of the incident signals should be estimated adaptively, and/or the time-varying directions should be tracked promptly. In this paper, an adaptive bearing estimation and tracking (ABEST) algorithm is investigated for estimating and tracking the uncorrelated and correlated narrow-band signals impinging on a uniform linear array (ULA) based on the subspace-based method without eigendecomposition (SUMWE), where a linear operator is obtained from the array to form a basis for the null space by exploiting the array geometry and its shift invariance property. Specifically, the null space is estimated using the least-mean-square (LMS) or normalized LMS (NLMS) algorithm, and the directions are updated using the approximate Newton method. The transient analyses of the LMS and NLMS algorithms are studied, where the weight (i.e., the linear operator) is in the form of a matrix and there is a correlation between the additive noise and input data that involve the instantaneous correlations of the received array in the updating equation, and the step-size stability conditions are derived explicitly. In addition, the analytical expressions for the mean-square error (MSE) and mean-square deviation (MSD) learning curves of the LMS algorithm are clarified. The effectiveness of the ABEST algorithm is verified, and the theoretical analyses are corroborated through numerical examples. Simulation results show that the ABEST algorithm is computationally simple and has good adaptation and tracking abilities.

Optimal location management for two-tier PCS networks

One of the most important issues in Personal Communication Service (PCS) networks is location management, which keeps track of the Mobile Terminals (MTs) moving from place to place. In this paper, we analytically derive cost functions of location updates and paging for a dynamic movement-based location management scheme for PCS networks with two-tier mobility databases. We prove analytically that there is a unique optimal movement threshold that minimizes the total cost of Home Location Register location updates, Visitor Location Register location updates, and paging, per call arrival. An effective algorithm is proposed to find the optimal movement threshold. Furthermore, we propose a hybrid location management scheme, in which when the call-to-mobility ratio is larger than a threshold, the optimal dynamic movement-based scheme is adopted. Otherwise, the static location update is adopted. The Newton approximation method is adopted to find this threshold. Our study shows that the proposed hybrid scheme outperforms both the dynamic movement-based scheme and the static location update scheme.

Newton Methods for Solving Two Classes of Nonsmooth Equations

The paper is devoted to two systems of nonsmooth equations. One is the system of equations of max-type functions and the other is the system of equations of smooth compositions of max-type functions. The Newton and approximate Newton methods for these two systems are proposed. The Q-superlinear convergence of the Newton methods and the Q-linear convergence of the approximate Newton methods are established. The present methods can be more easily implemented than the previous ones, since they do not require an element of Clarke generalized Jacobian, of B-differential, or of b-differential, at each iteration point.

An algorithm for unsteady viscous flows at all speeds

An algorithm for the simulation of unsteady, viscous, stratified compressible flows, which remains valid at all speeds, is presented. The method is second-order accurate in both space and time and is independent of the Mach number. In order to remove the stiffness of the numerical problem due to the large disparity between the flow speed and the acoustic wave speed at low Mach number, an approximate Newton method, based on artificial compressibility, is proposed. Additionally, a modified àdvection upstream splitting method (AUSM+) scheme is used, which permits accurate computations of both compressible and incompressible flows. A detailed description of the method and an efficiency comparison with other approximate Newton methods described in the literature are given. Furthermore, it is shown that the accuracy of the algorithm is not dependent on the Mach number through the computations of various benchmark test cases. Copyright © 2000 John Wiley & Sons, Ltd.

On the modified Newton's approximation method for the solution of non-linear singular integral equations

On the modified Newton's approximation method for the solution of non-linear singular integral equations

Efficient Computational Model for Inductive Plasma Flows

A new approach to the numerical modeling of inductive plasmas is presented. The governing magnetohydrodynamic equations are discretized in a second-order accurate finite volume manner. A straightforward pressure-stabilized flow field solver is introduced as an alternative to the staggered-mesh solvers used in traditional algorithms. It is argued that the widely used integral boundary formulation for the electric field is computationally expensive and cannot be incorporated into an efficient iterative solution procedure. This approach is abandoned in favor of a 'far field' formulation of the electric field, such that powerful iterative methods can be applied to speed up the calculation. The discretized equations are solved through a damped Picard method and an approximate and full Newton method. Efficient linear algebra methods are used to solve the linear systems arising from the iterative methods. An appropriate linearization of the strongly positive Joule heating source term is important for the convergence at the linear level. The new model is tested on an LTE argon inductive plasma computation. The proposed approximate Newton method is found to converge substantially better than the Picard method. The full Newton method appears promising, although its successful use on fine meshes still requires improvements to the linear solver used.

An algorithm for unsteady viscous flows at all speeds

An algorithm for the simulation of unsteady, viscous, stratified compressible flows, which remains valid at all speeds, is presented. The method is second-order accurate in both space and time and is independent of the Mach number. In order to remove the stiffness of the numerical problem due to the large disparity between the flow speed and the acoustic wave speed at low Mach number, an approximate Newton method, based on artificial compressibility, is proposed. Additionally, a modified àdvection upstream splitting method (AUSM+) scheme is used, which permits accurate computations of both compressible and incompressible flows. A detailed description of the method and an efficiency comparison with other approximate Newton methods described in the literature are given. Furthermore, it is shown that the accuracy of the algorithm is not dependent on the Mach number through the computations of various benchmark test cases. Copyright © 2000 John Wiley & Sons, Ltd.

Newton iteration for partial differential equations and the approximation of the identity

It is known that the critical condition which guarantees quadratic convergence of approximate Newton methods is an approximation of the identity condition. This requires that the composition of the numerical inversion of the Frechet derivative with the derivative itself approximate the identity to an accuracy calibrated by the residual. For example, the celebrated quadratic convergence theorem of Kantorovich can be proven when this holds, subject to regularity and stability of the derivative map. In this paper, we study what happens when this condition is not evident “a priori” but is observed “a posteriori”. Through an in-depth example involving a semilinear elliptic boundary value problem, and some general theory, we study the condition in the context of dual norms, and the effect upon convergence. We also discuss the connection to Nash iteration.

An algorithm for low Mach number unsteady flows

An algorithm is proposed for the simulation of unsteady viscous stratified compressible flows. The advantage of the method is its capability to deal with a broad range of subsonic Mach numbers, including nearly incompressible flows, with a single modelling, based on the fully compressible Navier–Stokes equations. The method is second-order accurate both in space and time. To remove the stiffness of the numerical problem due to the large disparity between the flow and the acoustic wave speeds at low Mach number, an approximate Newton method, based on artificial compressibility, is used. After a detailed description of the method, the accuracy of the algorithm is checked by computing compressible and incompressible benchmark test cases. Results of flows over a backward facing step with or without stratification effects for a Reynolds number of 800 are compared with the steady solution of incompressible methods. The computation of a compressible natural convection allows to evaluate the accuracy of the method for flow including significant compressibility effects, whereas numerical results of a Poiseuille–Bénard channel flow for a Reynolds number of 10 demonstrate the efficiency of the present algorithm to compute unsteady flows.

Multistep approximation algorithms: Improved convergence rates through postconditioning with smoothing kernels

We show how certain widely used multistep approximation algorithms can be interpreted as instances of an approximate Newton method. It was shown in an earlier paper by the second author that the convergence rates of approximate Newton methods (in the context of the numerical solution of PDEs) suffer from a “loss of derivatives”, and that the subsequent linear rate of convergence can be improved to be superlinear using an adaptation of Nash–Moser iteration for numerical analysis purposes; the essence of the adaptation being a splitting of the inversion and the smoothing into two separate steps. We show how these ideas apply to scattered data approximation as well as the numerical solution of partial differential equations. We investigate the use of several radial kernels for the smoothing operation. In our numerical examples we use radial basis functions also in the inversion step.

Conjugate gradient and approximate Newton methods for an optimal probabilistic neural network for food color classification

The probabilistic neural network (PNN) is based on the esti- mation of the probability density functions. The estimation of these den- sity functions uses smoothing parameters that represent the width of the activation functions. A two-step numerical procedure is developed for the optimization of the smoothing parameters of the PNN: a rough optimiza- tion by the conjugate gradient method and a fine optimization by the approximate Newton method. The thrust is to compare the classification performances of the improved PNN and the standard back-propagation neural network (BPNN). Comparisons are performed on a food quality problem: french fry classification into three different color classes (light, normal, and dark). The optimized PNN correctly classifies 96.19% of the test data, whereas the BPNN classifies only 93.27% of the same data. Moreover, the PNN is more stable than the BPNN with regard to the random initialization. The optimized PNN requires 1464 s for training compared to only 71 s required by the BPNN. © 1998 Society of Photo-Optical Instrumentation Engineers. (S0091-3286(98)01711-5)

Approximate Newton Methods for Nonsmooth Equations

We develop general approximate Newton methods for solving Lipschitz continuous equations by replacing the iteration matrix with a consistently approximated Jacobian, thereby reducing the computation in the generalized Newton method. Locally superlinear convergence results are presented under moderate assumptions. To construct a consistently approximated Jacobian, we introduce two main methods: the classic difference approximation method and the e-generalized Jacobian method. The former can be applied to problems with specific structures, while the latter is expected to work well for general problems. Numerical tests show that the two methods are efficient. Finally, a norm-reducing technique for the global convergence of the generalized Newton method is briefly discussed.

A Switching-Method for nonlinear systems

A new iterative method is proposed for the solution of nonlinear systems. The method does not use explicit derivative informations and at each iteration automatically selects one of two distinct iterative schemes: a direct search method and a damped approximate Newton's method. So, the method is referred as Switching-Method. It is shown that the method is a global method with quadratic convergence. Numerical results show the very good practical performance of the method.