Traditional Newton Method Research Articles

The multi-parameter estimation of discrete distribution without closed-form solutions by the US algorithm

<p>Strong and steady convergence characterizes the upper-crossing/solution (US) algorithm, which is an effective method for identifying roots of a complicated nonlinear equation $ h(\theta) = 0 $. Only the case where one parameter of a distribution function can be directly specified by another parameter is taken into account by the research that is currently available. However, whether this approach can be applied in multi-parameter scenarios where one parameter cannot be clearly represented by the other is an issue deserving of more investigation. In order to extend the applicability of the US algorithm, this article used the Type Ⅰ discrete Weibull distribution with two parameters as an example. It then combined the US algorithm with the first-derivative lower bound (FLB) function method to estimate the complex situation where two parameters cannot be expressed as each other. Simulation studies and empirical analysis demonstrated that the US algorithm performs more accurately and steadily than the traditional Newton method.</p>

Bayesian Direction of Arrival Estimation with Prior Knowledge from Target Tracker

The performance of traditional direction of arrival (DOA) estimation methods always deteriorates at a low signal-to-noise ratio (SNR) or without sufficient observations. This paper investigates the Bayesian DOA estimation problem aided by the prior knowledge from the target tracker. The Bayesian Cramér–Rao lower bounds (CRLB) and the expected CRLB are first derived to evaluate the theoretical performance of Bayesian DOA estimation. Based on the maximum a posterior (MAP) estimator in the Bayesian framework, two methods are proposed. One is a two-step grid search method for a single target DOA case. The other is a gradient-based iterative solution for multiple targets DOA case, which extends the traditional Newton method by incorporating the prior knowledge. We also propose a minimum mean square error (MMSE) estimator using a Monte Carlo method, which requires trading off accuracy against computational complexity. By comparing with the maximum likelihood (ML) estimators and the MUSIC algorithm, the proposed three Bayesian estimators improve the DOA estimation performance in low SNR or with limited snapshots. Moreover, the performance is not affected by the correlation between sources.

Fetal Electrocardiography Extraction Based on Improved Fast Independent Components Analysis Algorithm.

Aiming at the problem that the fast independent components analysis (ICA) algorithm is sensitive to the initial value and easy to fall into local optimization, an improved fast ICA method is proposed for the extraction of fetal echocardiography (FECG) by combining the Simpson-Newton iterative and chaotic optimization algorithm to replace the traditional Newton iterative method. First, the Simpson formula is used to modify the traditional Newton method and a Simpson-Newton iterative algorithm is constructed. It shows that the Simpson-Newton iterative algorithm can significantly reduce the sensitivity of initial value selection, and has faster convergence speed. Then, combined with the Simpson-Newton method, the chaos optimization algorithm can obtain the approximate global optimal solution, which solves the problem that the traditional Newton iterative method tends to fall into local optimal and improves the separation performance of the fast ICA algorithm. Finally, based on chaos optimization, the proposed Simpson-Newton iterative fast ICA algorithm is applied to the extraction of FECG signals, and the extraction effect is evaluated by visual waveform and quantitative indicators. Furthermore, the algorithm is verified by different clinical signals. The experimental results show that the improved fast ICA algorithm can extract clear fetal heart signals, and there are almost no mixed maternal ECG signals in the extracted FECG signals. The extraction effect of the proposed method is thus optimal than that of the traditional fast ICA method.

Mixed Logit Model Based on Improved Nonlinear Utility Functions: A Market Shares Solution Method of Different Railway Traffic Modes

In recent years, with the development of high-speed railway in China, the railway operating mileages and passenger transport capacity have increased rapidly. Due to the high density of trains and the limited capacity of railways, it is necessary to solve market shares of different railway traffic modes in order to adjust the operation plans appropriately and run railway passenger transport products in line with passenger demand. Therefore, the purpose of this paper is to calculate market shares by formulating a mixed logit model based on improved nonlinear utility functions taking different factors into consideration, such as seat grades, fares, running time, passenger income levels and so on. Firstly according to maximum likelihood estimation, the likelihood function of this mixed logit model is proposed to maximize utility of all passenger groups. After that, we propose two improved algorithms based on the simulated annealing algorithm (ISAA-CC and ISAA-SS) to estimate the unknown parameters and solve the optimal solution of this model in order to enhance the computational efficiency. Finally, a real-world instance with related data of Beijing–Tianjin corridor, is implemented to demonstrate the performance and effectiveness of the proposed approaches. In addition, by performing this numerical experiment and comparing these two improved algorithms with the traditional Newton method, the ant colony algorithm and the simulated annealing algorithm, we prove that the improved algorithms we developed are superior to others in the optimal solution.

Inverse, Shifted Inverse, and Rayleigh Quotient Iteration as Newton's Method

The $l_2$ normalized inverse, shifted inverse, and Rayleigh quotient iterations are classic algorithms for approximating an eigenvector of a symmetric matrix. This work establishes rigorously that each iterate produced by one of these three algorithms can be viewed as a Newton's method iterate followed by a normalization. The equivalences given here are not meant to suggest changes to the implementations of the classic eigenvalue algorithms. However, they add further understanding to the formal structure of these iterations, and they provide an explanation for their good behavior despite the possible need to solve systems with nearly singular coefficient matrices. A historical development of these eigenvalue algorithms is presented. Using our equivalences and traditional Newton's method theory helps to gain understanding as to why normalized Newton's method, inverse iteration, and shifted inverse iteration are only linearly convergent and not quadratically convergent, as would be expected, and why a new linear system need not be solved at each iteration. We also give an explanation as to why our normalized Newton's method equivalent of Rayleigh quotient iteration is cubically convergent and not just quadratically convergent, as would be expected.

An adaptive Newton-method based on a dynamical systems approach

The traditional Newton method for solving nonlinear operator equations in Banach spaces is discussed within the context of the continuous Newton method. This setting makes it possible to interpret the Newton method as a discrete dynamical system and thereby to cast it in the framework of an adaptive step size control procedure. In so doing, our goal is to reduce the chaotic behavior of the original method without losing its quadratic convergence property close to the roots. The performance of the modified scheme is illustrated with various examples from algebraic and differential equations.

A Two-Point Newton Method Suitable for Nonconvergent Cases and with Super-Quadratic Convergence

An iterative formula based on Newton’s method alone is presented for the iterative solutions of equations that ensures convergence in cases where the traditional Newton Method may fail to converge to the desired root. In addition, the method has super-quadratic convergence of order 2.414 (i.e., ). Newton method is said to fail in certain cases leading to oscillation, divergence to increasingly large number, or offshooting away to another root further from the desired domain or offshooting to an invalid domain where the function may not be defined. In addition when the derivative at the iteration point is zero, Newton method stalls. In most of these cases, hybrids of several methods such as Newton, bisection, and secant methods are suggested as substitute methods and Newton method is essentially blended with other methods or altogether abandoned. This paper argues that a solution is still possible in most of these cases by the application of Newton method alone without resorting to other methods and with the same computational effort (two functional evaluations per iteration) like the traditional Newton method. In addition, the proposed modified formula based on Newton method has better convergence characteristics than the traditional Newton method.

Solving the Yule–Walker equations to generate synthetic, correlated wind speed variates

The work in this paper continues and improves on the research that has been carried out on the generation of synthetic, correlated wind speed variates that are defined by marginal Weibull distributions and auto- and cross-correlations. Previously, in order to establish the matrix of auto-regressive coefficients for a small number of wind variates, a simple recursive method had been used. However, attempts to apply the model, using simple numerical approaches, to an increasing number of wind turbines or locations, proved useless. Therefore, a direct solution of the Yule–Walker equations is proposed here. These equations, originally in matrix form and of size N2 for N wind variates, are put in symbolic notation and then broken down for numerical solution using the traditional Newton's method. The complete procedure is explained through a symbolic example and illustrated with two very different numerical cases: with the first, wind in very close locations; with the second, wind in dispersed locations of a region.

A Generalized Cross Validation Method for the Inverse Problem of 3-D Maxwell's Equation

The inverse problem of estimation of the electrical conductivity in the Maxwell’s equation is considered, which is reformulated as a nonlinear equation. The Generalized Cross Validation is used to estimate the global regularization parameter and the damped Gauss-Newton is applied to impose local regularization. The damped Gauss-Newton method requires no calculation of the Hessian matrix which is expensive for traditional Newton method. GCV method decreases the computational expense and overcomes the influence of nonlinearity and ill-posedness. The results of numerical simulation testify that this method is efficient.

A study of convergence on the Newton-homotopy continuation method

The traditional Newton method is a well-known and popular method for solving non-linear equation. It has high efficient in the convergence speed. However, we always need to guess the initial value in the iteration process. Good initial guess value can solve the equation quickly. Bad initial guess value usually will yield divergence. Homotopy continuation method is a kind of perturbation method. It can guarantee the answer by a certain path if we choose the auxiliary homotopy function, or call start system, well. This paper presents some useful rules for the choice of the auxiliary homotopy function to avoid the problem of divergence of traditional Newton method.

A numerical algorithm for stable 2D autoregressive filter design

Based on previous theoretical results we present in this paper a global estimation scheme for solving the stable 2D autoregressive filter problem. The different algorithms are based on the traditional Newton method and on the log barrier method that is employed in semi-definite programming. The Newton method is the faster one but the barrier method ensures that the iterates stay in the cone of positive semidefinites. In addition, a numerical test for the existence of a stable factorization of a two-variable squared magnitude response function is presented.

Improved Newton method for optimal power flow problem

This paper describes a new approach that improves the performance of the Newton's method for optimal power flow problem. The Lagrangian multipliers are calculated directly from the linearized system. The factorization is carried out on elements instead of structures in blocks. The main advantages are shorter processing times and reduced memory requirements. The paper compares this approach with the traditional Newton's method proposed by Sun et al. The effectiveness of the proposed approach has been examined by solving the 14-bus, 30-bus, 118-bus and Brazilian 810-bus systems.

Can Newton method be surpassed

A local algorithm is proposed for unconstrained optimization problem. Compared with the traditional Newton method with Choleski factorization, this algorithm has the same quadratic convergence. But its computation cost per iteration in average is less when the dimensionn≥55. The saving is estimated in the theoretical framework.

Solving systems of nonlinear equations In using a rotating hyperplane in

A procedure which accelerates the convergence of iterative methods for the numerical solution of systems of nonlinear algebraic and/or transcendental equations in is introduced. This procedure uses a rotating hyperplane in , whose rotation axis depends on the current approximation n-1 of components of the solution. The proposed procedure is applied here on the traditional Newton's method and on a recently proposed “dimension-reducing” method [5] which incorporates the advantages of nonlinear SOR and Newton's algorithms. In this way, two new modified schemes for solving nonlinear systems are correspondingly obtained. For both of these schemes proofs of convergence are given and numerical applications are presented.