Nonlinear Least-squares Problem Research Articles

Revisiting the fitting of the Nelson–Siegel and Svensson models

The Nelson–Siegel and the Svensson models are two widely used models for the term structure of interest rates. These models are quite simple and intuitive, but fitting them to market data is numerically challenging and various difficulties have been reported. In this paper, we provide a novel mathematical analysis of the fitting problem based on parametric optimization. We formulate the fitting problem as a separable nonlinear least-squares problem, in which the linear parameters can be eliminated. We provide a thorough discussion on the conditioning of the inner part of the reformulated problem and show that many of the reported difficulties encountered when solving it are inherent to the problem formulation itself and cannot be tackled by choosing a particular optimization algorithm. Our stability analysis provides novel insights that we use to show that some of the ill-conditioning can be avoided, and that a suitably chosen penalty approach can be used to address the remaining ill-conditioning. Numerical results indicate that this approach has the expected impact while being independent of any choice of a particular optimization algorithm. We further establish smoothness properties of the reduced objective function, putting global optimization methods for the reduced problem on a sound mathematical basis.

Convergent least-squares optimization methods for variational data assimilation

Data assimilation combines prior (or background) information with observations to estimate the initial state of a dynamical system over a given time-window. A common application is in numerical weather prediction where a previous forecast and atmospheric observations are used to obtain the initial conditions for a numerical weather forecast. In four-dimensional variational data assimilation (4D-Var), the problem is formulated as a nonlinear least-squares problem, usually solved using a variant of the classical Gauss-Newton (GN) method. However, we show that GN may not converge if poorly initialized. In particular, we show that this may occur when there is greater uncertainty in the background information compared to the observations, or when a long time-window is used in 4D-Var allowing more observations. The difficulties GN encounters may lead to inaccurate initial state conditions for subsequent forecasts. To overcome this, we apply two convergent GN variants (line search and regularization) to the long time-window 4D-Var problem and investigate the cases where they locate a more accurate estimate compared to GN within a given budget of computational time and cost. We show that these methods are able to improve the estimate of the initial state, which may lead to a more accurate forecast. Highlights Poor initialization of Gauss-Newton method may result in failure to converge. Safeguarded Gauss-Newton improves initial state estimate within limited time/cost. Results using twin experiments with long time-window and chaotic Lorenz models. Apply state of the art least-squares convergence theory to data assimilation. Improvements to initial state estimate may lead to a more accurate forecast.

On diagonally structured scheme for nonlinear least squares and data-fitting problems

Recently, structured nonlinear least-squares (NLS) based algorithms gained considerable emphasis from researchers; this attention may result from increasingly applicable areas of these algorithms in different science and engineering domains. In this article, we coined a new efficient structured-based NLS algorithm. We developed a diagonal Hessian-based formulation for solving NLS problems. We derived the quasi-Newton update based on a diagonal matrix scheme subject to a modified structured secant condition. Also, we show that the algorithm’s search direction satisfies a sufficient descent condition under some standard assumptions. Subsequently, we also prove the global convergence of the algorithm and then eventually show its linear convergence rate for strongly convex functions. Furthermore, to show case the proposed algorithm’s performance, we experimented numerically by comparing it with other approaches on some benchmark test functions available in the literature. Finally, the introduced scheme is applied to solve some data-fitting problems

On Spectral Dai-Yuan Structured Technique for Solving Nonlinear Least-Squares Optimisation

Handling nonlinear least squares (NLS) problems as unconstrained optimisation problems has led to the development and adaptation of numerous numerical approaches. Calculating the Hessian matrix expensively for each iteration is one of the problems with the current numerical approaches for solving NLS problems. Some methods designed to rely solely on first-derivative information while ignoring second-order details tend to underperform when dealing with problems involving non-zero residuals. This paper suggests a modification to the Dai-Yuan type (DY) formula by deriving a spectral parameter for the proposed search direction. The sufficient descent results of the new formula are established using the standard assumptions under the strong Wolfe line search. Furthermore, experiments are conducted on some benchmark functions to demonstrate the computational efficiency of the proposed technique. The results highlight the effectiveness of the algorithm compared to existing methods.Keywords: Conjugate gradient, convergence theory, Hessian matrix, optimisation, Quasi-Newton, spectral, structured secant method

Kinetic modeling of the plasma pharmacokinetic profiles of ADAMTS13 fragment and its Fc-fusion counterpart in mice.

Introduction: Fusion of the fragment crystallizable (Fc) to protein therapeutics is commonly used to extend the circulation time by enhancing neonatal Fc-receptor (FcRn)-mediated endosomal recycling and slowing renal clearance. This study applied kinetic modeling to gain insights into the cellular processing contributing to the observed pharmacokinetic (PK) differences between the novel recombinant ADAMTS13 fragment (MDTCS) and its Fc-fusion protein (MDTCS-Fc). Methods: For MDTCS and MDTCS-Fc, their plasma PK profiles were obtained at two dose levels following intravenous administration of the respective proteins to mice. The plasma PK profiles of MDTCS were fitted to a kinetic model with three unknown protein-dependent parameters representing the fraction recycled (FR) and the rate constants for endocytosis (kup, for the uptake into the endosomes) and for the transfer from the plasma to the interstitial fluid (kpi). For MDTCS-Fc, the model was modified to include an additional parameter for binding to FcRn. Parameter optimization was done using the Cluster Gauss-Newton Method (CGNM), an algorithm that identifies multiple sets of approximate solutions ("accepted" parameter sets) to nonlinear least-squares problems. Results: As expected, the kinetic modeling results yielded the FR of MDTCS-Fc to be 2.8-fold greater than that of MDTCS (0.8497 and 0.3061, respectively). In addition, MDTCS-Fc was predicted to undergo endocytosis (the uptake into the endosomes) at a slower rate than MDTCS. Sensitivity analyses identified the association rate constant (kon) between MDTCS-Fc and FcRn as a potentially important factor influencing the plasma half-life in vivo. Discussion: Our analyses suggested that Fc fusion to MDTCS leads to changes in not only the FR but also the uptake into the endosomes, impacting the systemic plasma PK profiles. These findings may be used to develop recombinant protein therapeutics with extended circulation time.

Intelligent computing for electromagnetohydrodynamic bioconvection flow of micropolar nanofluid with thermal radiation and stratification: Levenberg–Marquardt backpropagation algorithm

The Levenberg–Marquardt (LM) backpropagation optimization algorithm, an artificial neural network algorithm, is used in this study to perform integrated numerical computing to evaluate the electromagnetohydrodynamic bioconvection flow of micropolar nanofluid with thermal radiation and stratification. The model is then reduced to a collection of boundary value problems, which are solved with the help of a numerical technique and the proposed scheme, i.e., the LM algorithm, which is an iterative approach to determine the minimum of a nonlinear function defined as the sum of squares. As a blend of the steepest descent and the Gauss–Newton method, it has become a typical approach for nonlinear least-squares problems. Furthermore, the stability and consistency of the algorithm are ensured. For validation purposes, the results are also compared with those of previous research and the MATLAB bvp4c solver. Neural networking is also utilized for velocity, temperature, and concentration profile mapping from input to output. These findings demonstrate the accuracy of forecasts and optimizations produced by artificial neural networks. The performance of the bvp4c solver, which is used to reduce the mean square error, is used to generalize a dataset. The artificial neural network-based LM backpropagation optimization algorithm operates using data based on the ratio of testing (13%), validation (17%), and training (70%). This stochastic computing work presents an activation log-sigmoid function based LM backpropagation optimization algorithm, in which tens of neurons and hidden and output layers are used for solving the learning language model. The overlapping of the results and the small computed absolute errors, which range from 10−3 to 10−10 and from 106 to 108 for each model class, indicate the accuracy of the artificial neural network-based LM backpropagation optimization algorithm. Furthermore, each model case’s regression performance is evaluated as if it were an ideal model. In addition, function fitness and histogram are used to validate the dependability of the algorithm. Numerical approaches and artificial neural networks are an excellent combination for fluid dynamics, and this could lead to new advancements in many domains. The findings of this research could contribute to the optimization of fluid systems, resulting in increased efficiency and production across various technical domains.

Enhancing Smart Irrigation Efficiency: A New WSN-Based Localization Method for Water Conservation

The shortage of water stands as a global challenge, prompting considerable focus on the management of water consumption and irrigation. The suggestion is to introduce a smart irrigation system based on wireless sensor networks (WSNs) aimed at minimizing water consumption while maintaining the quality of agricultural crops. In WSNs deployed in smart irrigation, accurately determining the locations of sensor nodes is crucial for efficient monitoring and control. However, in many cases, the exact positions of certain sensor nodes may be unknown. To address this challenge, this paper presents a new localization method for localizing unknown sensor nodes in WSN-based smart irrigation systems using estimated range measurements. The proposed method can accurately determine the positions of unknown nodes, even when they are located at a distance from anchors. It utilizes the Levenberg–Marquardt (LM) optimization algorithm to solve a nonlinear least-squares problem and minimize the error in estimating the unknown node locations. By leveraging the known positions of a subset of sensor nodes and the inexact distance measurements between pairs of nodes, the localization problem is transformed into a nonlinear optimization problem. To validate the effectiveness of the proposed method, extensive simulations and experiments were conducted. The results demonstrate that the proposed method achieves accurate localization of the unknown sensor nodes. Specifically, it achieves 19% and 58% improvement in estimation accuracy when compared to distance vector-hop (DV-Hop) and semidefinite relaxation-LM (SDR-LM) algorithms, respectively. Additionally, the method exhibits robustness against measurement noise and scalability for large-scale networks. Ultimately, integrating the proposed localization method into the smart irrigation system has the potential to achieve approximately 28% reduction in water consumption.

An Enhanced Path Finder Algorithm for the estimation of the stator current envelope to detect rotor bar breakage in an induction motor

The Enhanced Path Finder Algorithm (EPFA) is used in this research to present an innovative approach for examining the state of the rotor bars in an induction motor, employing a hybrid metaheuristic algorithm. Compared to current optimization techniques, the suggested approach permits more balanced exploration and exploitation. The method addresses the drawbacks of traditional analysis of fault related frequencies, such as short data length, spectrum leakages, and inadequate sampling frequency. The primary goal of this research is to identify the twice-slip frequency component and its harmonics in the current envelope that correlates with the fault. The A signal model of the stator current envelope and the residual error function have been used to define a nonlinear least-squares problem. An induction machine model has been developed in 2D ANSYS Maxwell software where faults have been implemented and studied. Finally, the efficiency of the fault detection scheme has been verified with a practical data set.

Rocket landing guidance based on second-order Picard-Chebyshev-Newton type algorithm

This paper proposes a rocket substage vertical landing guidance method based on the second-order Picard-Chebyshev-Newton type algorithm. Firstly, the continuous-time dynamic equation is discretized based on the natural second-order Picard iteration formulation and the Chebyshev polynomial. Secondly, the landing problem that considers terminal constraints is transformed into a nonlinear least-squares problem with respect to the terminal constraint function and solved with the Gauss-Newton method. In addition, the projection method is introduced to the iteration process of the Gauss-Newton method to realize the inequality constraints of the thrust. Finally, the closed-loop strategy for rocket substage vertical landing guidance is proposed and the numerical simulations of the rocket vertical landing stage are carried out. The simulation results demonstrate that compared with the sequential convex optimization algorithm, the proposed algorithm has higher computational efficiency.

Gaussian process regression and conditional Karhunen-Loève models for data assimilation in inverse problems

We present a model inversion algorithm, CKLEMAP, for data assimilation and parameter estimation in partial differential equation models of physical systems with spatially heterogeneous parameter fields. These fields are approximated using low-dimensional conditional Karhunen-Loève expansions (CKLEs), which are constructed using Gaussian process regression (GPR) models of these fields trained on the parameters' measurements. We then assimilate measurements of the state of the system and compute the maximum a posteriori (MAP) estimate of the CKLE coefficients by solving a nonlinear least-squares problem. When solving this optimization problem, we efficiently compute the Jacobian of the vector objective by exploiting the sparsity structure of the linear system of equations associated with the forward solution of the physics problem.The CKLEMAP method provides better scalability compared to the standard MAP method. In the MAP method, the number of unknowns to be estimated is equal to the number of elements in the numerical forward model. On the other hand, in CKLEMAP, the number of unknowns (CKLE coefficients) is controlled by the smoothness of the parameter field and the number of measurements, and is generally much smaller than the number of discretization nodes, which leads to a significant reduction of computational cost with respect to the standard MAP method. To show this advantage in scalability, we apply CKLEMAP to estimate the transmissivity field in a two-dimensional steady-state subsurface flow model of the Hanford Site by assimilating synthetic measurements of transmissivity and hydraulic head. We find that the execution time of CKLEMAP scales nearly linearly as N1.33, where N is the number of discretization nodes, while the execution time of standard MAP scales as N2.91. The CKLEMAP method improved execution time without sacrificing accuracy when compared to the standard MAP method.

Acquiring elastic properties of thin composite structure from vibrational testing data

Abstract The paper is devoted to a problem of acquiring elastic properties of a composite material from the vibration testing data with a simplified experimental acquisition scheme. The specimen is considered to abide by the linear elasticity laws and subject to viscoelastic damping. The boundary value problem for transverse movement of such a specimen in the frequency domain is formulated and solved with finite-element method. The correction method is suggested for the finite element matrices to account for the mass of the accelerometer. The problem of acquiring the elastic parameters is then formulated as a nonlinear least-square optimization problem. The usage of the automatic differentiation technique for stable and efficient computation of the gradient and hessian allows to use well-studied first and second order local optimization methods. We also explore the possibility of generating initial guesses for local minimization by heuristic global methods. The results of the numerical experiments on simulated data are analyzed in order to provide insights for the following real life experiments.

An improved dog-leg method for form-finding of tensegrity structures

This paper presents a novel and highly efficient form-finding method for tensegrity structures. The method is based on converting the nonlinear equilibrium equations with the nodal coordinates vector as the variables into nonlinear least-squares problems. Subsequently, the Dog-Leg method is applied to solve these transformed problems. Furthermore, the Dog-Leg method is combined with the MBFGS (Modified Broyden-Fletcher-Goldfarb-Shanno) method to overcome the singularity problem of the stiffness matrix. The resulting improved Dog-Leg method, i.e., DL-MBFGS method, can effectively address form-finding problems not only in self-stressed and non-self-stressed tensegrity structures, but also for tensegrities subjected to external loads and other scenarios including mechanisms and large displacements problems, which shows its great versatility and robustness. Several numerical examples are presented to validate the accuracy and effectiveness of the proposed method.

Analysis of Scale-Variant Robust Kernel Optimization for Nonlinear Least-Squares Problems

Analysis of Scale-Variant Robust Kernel Optimization for Nonlinear Least-Squares Problems

The Method of Moving Frames for Surface Global Parametrization

This article introduces a new representation of surface global parametrization based on Cartan’s method of moving frames . We show that a system of structure equations , characterizing the local coordinates changes with respect to a local frame system, completely characterizes the set of possible cone parametrizations. The discretization of this system provably provides necessary and sufficient conditions for the existence of a valid mapping. We are able to derive a versatile algorithm for surface parametrization, allowing feature constraints and singularities. As the first structure equation is independent of the global coordinate system, we do not require prior knowledge of cuts or cone positions. So, a single non-linear least-square problem is enough to place quantized cones while minimizing a given distortion energy. We are therefore able to take full advantage of the link between the parametrization geometry and the topology of its cone metric to solve challenging constrained parametrization problems.

Efficiently Range-Coupled Position Estimation for an Aerial Swarm With the Confidence Evaluation of Onboard Sensing

This work proposes an efficient range-coupled localization approach for aerial swarms considering the variance of onboard sensors in cluttered environments. First, a reciprocal bond estimation algorithm is proposed to estimate the position; it constructs an optimization framework based on the range-coupled observability analysis. Subsequently, considering the variable performance characteristics of onboard sensors in cluttered environments, a moving confidence evaluation algorithm is proposed to improve the resilience of the position estimation system by assessing the concurrent reliability of all sensors. Mathematically, the aforementioned two algorithms are integrally formulated using a nonlinear least-squares equation. Notably, solving this nonlinear problem is typically a low-efficiency process that deteriorates with the scale of the swarm. To overcome this issue, a gradient-aware Levenberg–Marquardt algorithm is herein proposed to enhance the computational efficiency of this system to solve this nonlinear least-squares problem. Finally, swirling experiments involving four drones are carried out to verify the performance of the proposed methods. The results demonstrate that the time cost of optimization in each servo period is only approximately 7.15 ms, and the computational efficiency exhibits at least a 2.78 times improvement over the existing methods. Meanwhile, even in a smoky environment, the estimation precision can be as great as 8 cm, which is comparable to results obtained using state-of-the-art methods.

A Modified Structured Spectral HS Method for Nonlinear Least Squares Problems and Applications in Robot Arm Control

This paper proposes a modification to the Hestenes-Stiefel (HS) method by devising a spectral parameter using a modified secant relation to solve nonlinear least-squares problems. Notably, in the implementation, the proposed method differs from existing approaches, in that it does not require a safeguarding strategy and its Hessian matrix is positive and definite throughout the iteration process. Numerical experiments are conducted on a range of test problems, with 120 instances to demonstrate the efficacy of the proposed algorithm by comparing it with existing techniques in the literature. However, the results obtained validate the effectiveness of the proposed method in terms of the standard metrics of comparison. Additionally, the proposed approach is applied to address motion-control problems in a robotic model, resulting in favorable outcomes in terms of the robot’s motion characteristics.

Numerical Simulations of Tank Sloshing Problems Based on Moving Pseudo-Boundary Method of Fundamental Solution

The moving pseudo-boundary method of fundamental solutions (MFS) was employed to solve the Laplace equation, which describes the potential flow in a two-dimensional (2D) numerical wave tank. The MFS is known for its ease of programming and the advantage of its high precision. The solution of the boundary value can be expressed by a linear combination of the fundamental solutions. The major issue with such an implementation is the optimal distribution of source nodes in the pseudo-boundary. Traditionally, the positions of the source nodes are assumed to be fixed to keep the set of equations closed. However, in the moving boundary problem, the distribution of source nodes may influence the stability of numerical calculations. Moreover, MFS is unstable in time iterations. Hence, it is necessary to constantly revise the weighting coefficients of fundamental solutions. In this study, the source nodes were free, and their locations were determined by solving a nonlinear least-squares problem using the Levenberg–Marquardt algorithm. To solve the above least-squares problem, the MATLAB© routine lsqnonlin was adopted. Additionally, the weighting coefficients of fundamental solutions were solved as a nonlinear least-squares problem using the aforementioned method. The numerical results indicated that the numerical simulation method adopted in this paper is accurate and reliable in solving the problem of 2D tank sloshing. The main contribution of this study is to expand the application of the MFS in engineering by integrating it with the optimal configuration problem of pseudo-boundaries to solve practical engineering problems.

The Hierarchical Newton’s Method for Numerically Stable Prioritized Dynamic Control

This work links optimization approaches from hierarchical least-squares programming to instantaneous prioritized whole-body robot control. Concretely, we formulate the hierarchical Newton’s method which solves prioritized nonlinear least-squares problems in a numerically stable fashion even in the presence of kinematic and algorithmic singularities of the approximated kinematic constraints. These results are then transferred to control problems which exhibit the additional variability of time. This is necessary to formulate acceleration-based controllers and to incorporate the second-order dynamics. However, we show that the Newton’s method without complicated adaptations is not appropriate in the acceleration domain. We therefore formulate a velocity-based controller which exhibits second-order proportional derivative (PD) convergence characteristics. Our developments are verified in toy robot control scenarios as well as in complex robot experiments which stress the importance of prioritized control and its singularity resolution.

Gauss--Newton--Kurchatov method for the solution of non-linear least-square problems using ω-condition

Abstract We propose to study the convergence of an iterative method used for solving non-linear least-square problems having differentiable as well as non-differentiable functions. We use the ω-condition on both first order divided difference of non-differentiable part and first order derivative of differentiable part to establish the condition for convergence of the method. We also present some numerical experiments as test beds for the proposed method. In all the numerical examples, we have compared our results with a well-known Gauss–Newton–Potra method and shown that our convergence analysis gives better error bounds.

Simultaneous inversion for source field and mantle electrical conductivity using the variable projection approach

Time-varying electromagnetic field observed on the ground or at a spacecraft consists of contributions from (i) electric source currents, such as those in the ionosphere and magnetosphere, and (ii) corresponding fields induced by source currents within the conductive Earth’s interior by virtue of electromagnetic induction. Knowledge about the spatio-temporal structure of inducing currents is a key component in ionospheric and magnetospheric studies, and is also needed in space weather hazard evaluation, whereas the induced currents depend on the Earth’s subsurface electrical conductivity distribution and allow us to probe this physical property. In this study, we present an approach that reconstructs the inducing source and subsurface conductivity structures simultaneously, preserving consistency between the two models by exploiting the inherent physical link. To achieve this, we formulate the underlying inverse problem as a separable nonlinear least-squares (SNLS) problem, where inducing current and subsurface conductivity parameters enter as linear and nonlinear model unknowns, respectively. We solve the SNLS problem using the variable projection method and compare it with other conventional approaches. We study the properties of the method and demonstrate its feasibility by simultaneously reconstructing the ionospheric and magnetospheric currents along with a 1-D average mantle conductivity distribution from the ground magnetic observatory data.Graphical