Newton Method Research Articles

Benchmarking methods for computing local sensitivities in ordinary differential equation models at dynamic and steady states.

Estimating parameters of dynamic models from experimental data is a challenging, and often computationally-demanding task. It requires a large number of model simulations and objective function gradient computations, if gradient-based optimization is used. In many cases, steady-state computation is a part of model simulation, either due to steady-state data or an assumption that the system is at steady state at the initial time point. Various methods are available for steady-state and gradient computation. Yet, the most efficient pair of methods (one for steady states, one for gradients) for a particular model is often not clear. In order to facilitate the selection of methods, we explore six method pairs for computing the steady state and sensitivities at steady state using six real-world problems. The method pairs involve numerical integration or Newton's method to compute the steady-state, and-for both forward and adjoint sensitivity analysis-numerical integration or a tailored method to compute the sensitivities at steady-state. Our evaluation shows that all method pairs provide accurate steady-state and gradient values, and that the two method pairs that combine numerical integration for the steady-state with a tailored method for the sensitivities at steady-state were the most robust, and amongst the most computationally-efficient. We also observed that while Newton's method for steady-state computation yields a substantial speedup compared to numerical integration, it may lead to a large number of simulation failures. Overall, our study provides a concise overview across current methods for computing sensitivities at steady state. While our study shows that there is no universally-best method pair, it also provides guidance to modelers in choosing the right methods for a problem at hand.

New Generalized Derivatives for Solving Variational Inequalities Using the Nonsmooth Newton Methods

AbstractVariational inequality (VI) generalizes many mathematical programming problems and has a wide variety of applications. One class of VI solution methods is to reformulate a VI into a normal map nonsmooth equation system, which is then solved using nonsmooth equation-solving techniques. In this article, we propose a first practical approach for furnishing B-subdifferential elements of the normal map, which in turn enables solving the normal map equation system using variants of the B-subdifferential-based nonsmooth Newton method. It is shown that our new method requires less stringent conditions to achieve local convergence than some other established methods, and thus guarantees convergence in certain cases where other methods may fail. We compute a B-subdifferential element using the LD-derivative, which is a recently established generalized derivative concept. In our new approach, an LD-derivative is computed by solving a sequence of strictly convex quadratic programs, which can be terminated early under certain conditions. Numerical examples are provided to illustrate the convergence properties of our new method, based on a proof-of-concept implementation in Python.

Quasi-modified-Newton method-based selective harmonic elimination in cascaded H bridge inverters

Multilevel converters have gained significant popularity in medium-voltage and high-power applications due to their numerous advantages over traditional two-level converters. These advantages include reduced harmonic distortion, improved efficiency, and lower stress on power semiconductors. Selective harmonic elimination (SHE) is a modulation method that can be employed with multilevel converters to achieve high-quality output voltage waveforms. In this work, an extension of Broyden’s method, known as the Quasi-Modified Newton Method, is implemented for selective harmonic elimination and accurate calculation of switching angles for a wide range of modulation indices. The proposed method is applied to cascaded H bridge inverters operating at levels 5, 7, and 9. The method offers simplicity, reduced computational burden, and faster convergence, making it easily implementable, reducing total harmonic distortion (THD), and reducing RMSE and MAD errors. The paper includes simulation and experimental results that validate the accuracy and effectiveness of the proposed approach.

Fast Convex Optimization via Time Scale and Averaging of the Steepest Descent

In a Hilbert setting, we develop a gradient-based dynamic approach for fast solving convex optimization problems. By applying time scaling, averaging, and perturbation techniques to the continuous steepest descent (SD), we obtain high-resolution ordinary differential equations of the Nesterov and Ravine methods. These dynamics involve asymptotically vanishing viscous damping and Hessian-driven damping (either in explicit or implicit form). Mathematical analysis does not require developing a Lyapunov analysis for inertial systems. We simply exploit classical convergence results for SD and its external perturbation version, then use tools of differential and integral calculus, including Jensen’s inequality. The method is flexible, and by way of illustration, we show how it applies starting from other important dynamics in optimization. We consider the case in which the initial dynamic is the regularized Newton method, then the case in which the starting dynamic is the differential inclusion associated with a convex lower semicontinuous potential, and finally we show that the technique can be naturally extended to the case of a monotone cocoercive operator. Our approach leads to parallel algorithmic results, which we study in the case of fast gradient and proximal algorithms. Our averaging technique shows new links between the Nesterov and Ravine methods. Funding: The research of R.I. Boţ and D.-K. Nguyen was supported by the Austrian Science Fund (FWF), projects W 1260 and P 34922-N.

Galerkin’s Method and Multivariate Newton’s Method for the Nonlinear Deformation of the Magneto-Electro-Elastic Bi-Layered Laminates

In this paper, mathematical modelling for the large deformation of a magneto-electro-elastic rectangular bi-layered laminate with general boundary conditions is presented. Constitutive equations involving the magneto-electro-elastic (MEE) material properties are introduced, Maxwell equations accounts for the electric and magnetic effects are also utilized. First-order shear deformation theory (FSDT) considering the von Karman nonlinear strain is adopted, and the plain strain/stress assumption applicable for thin plate analysis is used. A rather compact set of governing equations related to kinematical variables, electric/magnetic potentials and the Airy stress function is obtained as a consequence of the preliminary condensation for the electro-magnetic state to the plate kinematics. Semi-analytic solution for a bi-layered BaTiO3-CoFe2O4 laminate with specified boundary conditions subjected to various external applied loads is performed. By employing the Bubnov-Galerkin method, the set of nonlinear partial differential equations is transformed to a set of third-order nonlinear algebraic equations for the static deformation due to applied load. Numerical results are carried out by using the multivariate Newton's method with respect to various volume fractions indicating the volume ratio between piezoelectric BaTiO3 layer and piezomagnetic CoFe2O4. From the result, the nonlinearity of the von Karman strain appears to enhance system rigidity as smaller deformations will be detected when external load is applied. Also, some other interesting results are obtained which could be useful to future analysis and design of multiphase composite plates.

Complex signals parameters optimization on the base of linear approximations using the gradient method and Newton’s method

The article examines the effectiveness of the gradient descent and Newton methods for optimizing the parameters of ensembles of complex signals. Algorithms have been developed and implemented that increase the accuracy of setting parameters and ensure reasonable optimization of spectral, temporal and statistical characteristics of signals. The effectiveness of the application of the methods was confirmed experimentally on the example of reducing the error and increasing the level of immunity. The obtained results substantiate the improvement of the parameters of complex signals, which proves the efficiency of use for wireless telecommunication systems in order to ensure stable and reliable operation in conditions of dynamic changes in the environment and a high level of interference.The article compares the mathematical methods of optimization, namely the gradient method and Newton's method, proposes mathematical models and constructs algorithms that empirically prove the effectiveness of the application of the studied mathematical methods in the specified scientific area - for optimizing the parameters of ensembles of complex signals. Scientific works [1-6, 9, 12] present algorithms based on the gradient method and Newton's method, but they do not consider in detail the comparative analysis of the effectiveness of these methods for optimizing the parameters of ensembles of complex signals for implementation in various scientific and practical tasks. The effectiveness of the algorithms proposed in the article was confirmed experimentally, which made it possible to reduce the error and improve the characteristics of ensembles of complex signals.As a result of the experiments using the methods of gradient descent and Newton, a significant reduction of the error and an improvement of the stability of the signals were achieved. Newton's method reduced the error from 0.1 to 0.0027, justifying the high accuracy of setting the signal parameters. The gradient descent method provided a stable reduction of the gradient norm from 12.75 to less than 1.23, effectively reducing the interference level, i.e. increasing the interference immunity.

Moreau-Yosida regularization to optimal control of the monodomain model with pointwise control and state constraints in cardiac electrophysiology

In this work, we study the optimal control problem of a coupled reaction-diffusion system, which is a monodomain model in cardiac electrophysiology with pointwise bilateral control and state constraints. We adopt the Moreau-Yosida regularization as a penalization technique to deal with the state constraints. The regularized problem’s first-order optimality condition is derived. In addition, sufficient second-order optimality condition is derived for the regularized problem using the virtual control concept by proving equivalence between Moreau-Yosida regularization and the virtual control concept. The convergence of optimal controls of the regularized problems to the optimal control of the original problem is proved. Moreover, the semi-smooth Newton method for numerically finding the optimal solution to the regularization problem is presented. Finally, numerical experiments are conducted, and the results allow us to understand the extinction of the wave excitation in cardiac defibrillation in the presence of both control and state constraints.

A general alternating-direction implicit Newton method for solving continuous-time algebraic Riccati equation

The complex continuous-time algebraic Riccati equation (CCARE) is quadratic, which is closely related to the analysis of the optimal control problem. In this paper, we apply Newton method as the outer iteration and an efficient general alternating-direction implicit (GADI) method as the inner iteration to solve CCARE. Meanwhile, we propose the inexact Newton-GADI method to further improve the efficiency of the algorithm. We give the convergence analysis of our proposed method and prove that its convergence rate is faster than the classical Newton-ADI method. Finally, some numerical examples are given to illustrate the effectiveness of our algorithms and the correctness of the theoretical analysis.

A class of regularized adaptive multichannel equalization algorithms for speech dereverberation

The multiple input/output inverse theorem (MINT) is a typical speech dereverberation technique based on acoustic channel equalization. Although the adaptive MINT (A-MINT) method has been developed to meet the requirements of real-time application systems, its performance often deteriorates due to the effect of the identification errors of room impulse responses (RIRs). In this paper, we present a regularized A-MINT approach to reduce the sensitivity of the adaptive equalizer to the RIR identification errors. According to the regularized MINT criterion, Newton's and quasi-Newton methods are used to establish two adaptive speech dereverberation algorithms, respectively, which obtain faster convergent rate and better dereverberation performance. The effectiveness of the proposed algorithms is validated through numerical experiments with the RIRs measured in real acoustic environments.

A fast compact difference scheme on graded meshes for solving the time-fractional nonlinear KdV equation

The fractional KdV equation has significant physical implications, the study of its efficient method has great scientific significance. A fast compact difference (FCD) scheme on graded meshes is used to address the time-fractional nonlinear KdV equation with an initial singularity. The temporal discretization employs the fast L1 algorithm, while a fourth-order compact scheme is utilized for spatial discretization. The Newton's method is used to approximate the nonlinear term. The fast L1 algorithm can greatly shorten the CPU calculation time without losing the precision of numerical solution. We verify the existence and uniqueness of numerical solutions, while also provide unconditional stability and convergence analysis. Theoretical analysis is validated by numerical results, indicating the FCD scheme on graded meshes has high-accuracy and time-saving performance.

Multicriteria Optimization of Stochastic Robust Control of the Tracking System

A multicriteria optimization of stochastic robust control with two degrees of freedom of a tracking system with anisotropic regulators has been developed to increase accuracy and reduce sensitivity to uncertain object parameters. Such objects are located on a moving base, on which sensors for angles, angular velocities and angular accelerations are installed. Improvements in the accuracy of control with two degrees of freedom include closed-loop feedback control and open-loop feedback control through the use of reference and perturbation effects. The multicriteria optimization of the stochastic robust control tracking system with two degrees of freedom with anisotropic controllers is reduced to the iterative solution of a system of four coupled Riccati equations, the Lyapunov equation, and the determination of the anisotropy norm of the system by an expression of a special form, which is numerically solved using the homotopy method, which includes vectorization matrices and iterations according to Newton's method. The objective vector of robust control is calculated in the form of a solution of a vector game, the vector gains of which are direct indicators of the quality that the system should achieve in different modes of its operation. The calculation of the vector gains of this game is related to the simulation of a synthesized system with anisotropic regulators for different modes of operation with different input signals and object parameter values. The solutions of this vector game are calculated on the basis of a set of Pareto-optimal solutions taking into account the binary relations of preferences on the basis of the metaheuristic algorithm of multi-swarm Archimedes optimization. Based on the results of the synthesis of stochastic robust control of a tracking system with two degrees of freedom with anisotropic controllers, it is shown that the use of synthesized controllers made it possible to increase the accuracy of system control, reduce the time of transient processes by 3–5 times, reduce the variance of errors by 2.7 times, and reduce the sensitivity of the system to the change of object parameters compared to typical regulators.

A Generalized Learning Approach to Deep Neural Networks

Optimization of machine learning architectures is essential in determining the efficacy and the applicability of any neural architecture to real world problems. In this work a generalized Newton's method (GNM) is presented as a powerful approach to learning in deep neural networks (DNN). This technique was compared to two popular approaches, namely the stochastic gradient descent (SGD) and the Adam algorithm, in two popular classification tasks. The performance of the proposed approach confirmed it as an attractive alternative to state-of-the-art first order solutions. Due to the good results presented in the case of shallow DNN, in the last part of the article an hybrid optimization method is presented. This method consists in combining two optimization algorithms, i.e. GNM and Adam or GNM and SGD, during the training phase within the layers of the neural network. This configuration aims to benefit from the strengths of both first- and second-order algorithms. In this case a convolutional neural network is considered and its parameters are updated with a different optimization algorithm. Also in this case, the hybrid approach returns the best performance with respect to the first order algorithms.

A Sparse Smoothing Newton Method for Solving Discrete Optimal Transport Problems

The discrete optimal transport (OT) problem, which offers an effective computational tool for comparing two discrete probability distributions, has recently attracted much attention and played essential roles in many modern applications. This paper proposes to solve the discrete OT problem by applying a squared smoothing Newton method via the Huber smoothing function for solving the corresponding KKT system directly. The proposed algorithm admits appealing convergence properties and is able to take advantage of the solution sparsity to greatly reduce computational costs. Moreover, the algorithm can be extended to solve problems with similar structures, including the Wasserstein barycenter (WB) problem with fixed supports. To verify the practical performance of the proposed method, we conduct extensive numerical experiments to solve a large set of discrete OT and WB benchmark problems. Our numerical results show that the proposed method is efficient compared to state-of-the-art linear programming (LP) solvers. Moreover, the proposed method consumes less memory than existing LP solvers, which demonstrates the potential usage of our algorithm for solving large-scale OT and WB problems.

Identifying the mixed and forced convection flow with transverse magnetic field

AbstractOptimal control of the steady, laminar, magnetohydrodynamics (MHD) mixed convection flow of an electrically conducting fluid is considered under the effect of the transverse magnetic field in a square duct. The viscous and Joule dissipations are included and the flow is driven by a constant pressure gradient. The triple nonlinear set of momentum, induction, and energy equations are solved in dimensionless form by using the mixed finite element method (FEM) with the implementation of Newton's method for nonlinearity with the discretize‐then‐linearize approach. Accordingly, FEM solutions are obtained for various values of the problem parameters to ensure the efficiency of the underlying scheme. This study aims to investigate the problem of controlling the steady flow by using the physically significant parameters of the problem as control variables in the case of a mixed convection flow. In this respect, classification of the type of convection, forced or free, is achieved by controlling the Grashof number (Gr). Besides, single and pairwise controls with Hartmann number (M), Prandtl number (Pr), and magnetic Reynolds number (Rm) are also used to regain the prescribed fluid behaviors and required magnetic field. Control simulations are conducted with the sequential‐least‐squares‐programming (SLSQP) algorithm in the optimization.

Research on dynamic modeling and control strategy of four anti-swing cable system for ship-mounted cranes

Ship-mounted cranes are becoming increasingly essential to modern ocean shipping. However, due to the continuous influence of waves, currents, and winds, achieving accurate and fast lifting is difficult because of the large payload swing during the lifting operation. Therefore, this study proposes a Four Anti-swing Cable System (FASCS) for ship-mounted cranes. Firstly, a dynamic model of the FASCS was established by using Robotics and Newton method, and a tension control method (TCM) is designed to reduce the swing angle of the payload. Concurrently, a sliding mode variable structure controller with improved reaching law (SMVSC-IRL) is designed to control the cooperative movement of the anti-swing cables and prevent the occurrence of snap. The dynamic characteristic analysis by MATLAB/Simulink shows that the swing angle suppression effect reaches 89.3% on average, and the projected area of the payload trajectory was reduced by 60%, which can significantly improve the efficiency of ship-mounted cranes lifting operations. In addition, the length and speed of cables in errors can approach 0 within 7 s, and the strong robustness of the designed SMVSC to control the cooperative motion of the anti-swing cables is proved. The results of this study contribute to the in-depth optimization and engineering verification of the FASCS for ship-mounted cranes.

[formula omitted]-periodic wave solutions of the [formula omitted] supersymmetric KdV equation

In this paper, the N-periodic wave solutions of the N=2 supersymmetric KdV equation are studied by combining the super Hirota bilinear form with the super Riemann-theta function, which can be used to describe new phenomena on super quasi-periodic waves with the fermionic field. With the aid of the Gauss–Newton method, the three-periodic and four-periodic wave solutions are obtained. In particular, these quasi-periodic waves can produce parallel, crossed and degenerated patterns. The analytical method related to the characteristic lines is used to analyze the dynamic characteristics of the three-periodic and four-periodic waves. In addition, it has been indicated that N-periodic waves can exist in the supersymmetric integrable systems.

Application of the iteratively regularized Gauss–Newton method to parameter identification problems in Computational Fluid Dynamics

Field Inversion and Machine Learning is an active field of research in Computational Fluid Dynamics (CFD). This approach can be leveraged to obtain a closed-form correction for a given turbulence model to improve the predictions. The fundamental approach is to insert a parameter into the system of RANS equations and determine it in a way such that, for example, a given pressure distribution is better approximated compared to the one obtained with the original set of equations. The goal of this article is twofold. Numerical arguments are presented that these kinds of problems can be severely ill-posed. In the second part, an approach is presented to directly reconstruct the turbulent viscosity field along with an example. The Iteratively Regularized Gauss-Newton Method (IRGNM) is used for a realization. The construction of a problem-adapted norm for a finite volume method is presented. Finally, an outlook is presented on how this approach can be used to possibly modify or improve turbulence models such that not only one, but a larger number of test cases are considered.

Proximal Gauss-Newton Method for Box-constrained Parameter Identification of a Nonlinear Railway Suspension System

The identification of railway vehicle components’ characteristics from measured data is a challenging task with compelling applications in health monitoring, fault detection, and system prognosis. Usually, though, such systems are highly nonlinear, and naive identification techniques may lead to unstable methods and inaccurate results. In this paper, we show that these issues can be easily tackled with the recently introduced proximal Gauss–Newton method, which we employ to identify the parameters of a railway nonlinear suspension system. In the proposed model, the parameters are subject to safety bounds in form of box constraints, which allows preventing nonphysical solutions. The suspension system we consider is highly nonlinear due to the presence of an airspring in the secondary suspension, which we introduce in a simplified Berg model. Numerical examples, featuring data corrupted by various noise levels, demonstrate the accuracy and efficiency of our proposed method. Comparisons with state-of-the-art approaches are also provided.

Floating-point photonic iterative solver demonstrated for Newton–Raphson method

Though photonic computing systems offer advantages in speed, scalability, and power consumption, they often have a limited dynamic encoding range due to low signal-to-noise ratios. Compared to digital floating-point encoding, photonic fixed-point encoding limits the precision of photonic computing when applied to scientific problems. In the case of iterative algorithms such as those commonly applied in machine learning or differential equation solvers, techniques like precision decomposition and residue iteration can be applied to increase accuracy at a greater computing cost. However, the analog nature of photonic symbols allows for modulation of both amplitude and frequency, opening the possibility of encoding both the significand and exponent of floating-point values on photonic computing systems to expand the dynamic range without expending additional energy. With appropriate schema, element-wise floating-point multiplication can be performed intrinsically through the interference of light. Herein, we present a method for configurable, signed, floating-point encoding and multiplication on a limited precision photonic primitive consisting of a directly modulated Mach–Zehnder interferometer. We demonstrate this method using Newton's method to find the Golden Ratio within ±0.11%, with six-level exponent encoding for a signed trinary digit-equivalent significand, corresponding to an effective increase of 243× in the photonic primitive's dynamic range.

Efficient finite element strategy using enhanced high-order and second-derivative-free variants of Newton's method

In this work, we propose a stable finite element approximation by extending higher-order Newton's method to the multidimensional case for solving nonlinear systems of partial differential equations. This approach relies solely on the evaluation of Jacobian matrices and residuals, eliminating the need for computing higher-order derivatives. Achieving third and fifth-order convergence, it ensures stability and allows for significantly larger time steps compared to explicit methods. We thoroughly address accuracy and convergence, focusing on the singular p-Laplacian problem and the time-dependent lid-driven cavity benchmark. A globalized variant incorporating a continuation technique is employed to effectively handle high Reynolds number regimes. Through two-dimensional and three-dimensional numerical experiments, we demonstrate that the improved cubically convergent variant outperforms others, leading to substantial computational savings, notably halving the computational cost for the lid-driven cavity test at large Reynolds numbers.