Nonlinear Least-squares Problem Research Articles

DOA estimation with known waveforms in the presence of unknown time delays and Doppler shifts

A novel DOA estimation method for known waveform sources with different unknown time delays and Doppler shifts is proposed. Based on the idea of maximum likelihood and the matrix projection theory, a decoupled cost function is first constructed and then the problem of estimating time delay and Doppler shift is transformed into a nonlinear least squares (NLS) problem. To solve the NLS problem efficiently without multidimensional search, a Toeplitz dominant rule is established to perform initial estimates with a reduced dimension. Finally, with the aid of time delay and Doppler shift estimates, DOAs and complex amplitudes of the incoming signals are obtained. Simulation results show that the proposed method can achieve a performance close to CRB at high SNR and with a large number of snapshots.

Real-time pose and shape reconstruction of two interacting hands with a single depth camera

We present a novel method for real-time pose and shape reconstruction of two strongly interacting hands. Our approach is the first two-hand tracking solution that combines an extensive list of favorable properties, namely it is marker-less, uses a single consumer-level depth camera, runs in real time, handles inter- and intra-hand collisions, and automatically adjusts to the user's hand shape. In order to achieve this, we embed a recent parametric hand pose and shape model and a dense correspondence predictor based on a deep neural network into a suitable energy minimization framework. For training the correspondence prediction network, we synthesize a two-hand dataset based on physical simulations that includes both hand pose and shape annotations while at the same time avoiding inter-hand penetrations. To achieve real-time rates, we phrase the model fitting in terms of a nonlinear least-squares problem so that the energy can be optimized based on a highly efficient GPU-based Gauss-Newton optimizer. We show state-of-the-art results in scenes that exceed the complexity level demonstrated by previous work, including tight two-hand grasps, significant inter-hand occlusions, and gesture interaction. 1

ISAR Imaging and Cross-Range Scaling for Maneuvering Targets by Using the NCS-NLS Algorithm

The azimuth resolution of an inverse synthetic aperture radar (ISAR) image is determined by the angular diversity of the target. In order to improve the azimuth resolution of an ISAR image, it is necessary to view the target at more different aspect angles, which is commonly achieved by extending the coherent processing interval (CPI). However, as the CPI lengthens, the effective rotation of a target can no longer be approximated to be uniform and the rotational acceleration should be taken into consideration. The non-uniform rotation can bring about the 2-D space-variant phase error, which degrades the focusing quality of an ISAR image seriously. In this paper, a novel ISAR imaging and a cross-range scaling algorithm for a maneuvering target with a non-uniform rotation are proposed. In the proposed algorithm, the spatially variant phase error is compensated for by the joint phase error compensation method based on nonlinear chirp scaling (NCS) principle. The estimation of the effective rotational information required in the NCS-based joint phase error compensation method and the cross-range scaling is transformed into a nonlinear least-squares (NLS) problem. The NLS problem is solved by Gauss–Newton method and FFTs with high efficiency. The results acquired from the processing of both the simulation and real data validate the effectiveness of the proposed algorithm.

Modified inexact Levenberg–Marquardt methods for solving nonlinear least squares problems

In the present paper, we propose a modified inexact Levenberg–Marquardt method (LMM) and its global version by virtue of Armijo, Wolfe or Goldstein line-search schemes to solve nonlinear least squares problems (NLSP), especially for the underdetermined case. Under a local error bound condition, we show that a sequence generated by the modified inexact LMM converges to a solution superlinearly and even quadratically for some special parameters, which improves the corresponding results of the classical inexact LMM in Dan et al. (Optim Methods Softw 17:605–626, 2002). Furthermore, the quadratical convergence of the global version of the modified inexact LMM is also established. Finally, preliminary numerical experiments on some medium/large scale underdetermined NLSP show that our proposed algorithm outperforms the classical inexact LMM.

A derivative-free Gauss\u2013Newton method

We present DFO-GN, a derivative-free version of the Gauss–Newton method for solving nonlinear least-squares problems. DFO-GN uses linear interpolation of residual values to build a quadratic model of the objective, which is then used within a typical derivative-free trust-region framework. We show that DFO-GN is globally convergent and requires at most {mathcal {O}}(epsilon ^{-2}) iterations to reach approximate first-order criticality within tolerance epsilon . We provide an implementation of DFO-GN and compare it to other state-of-the-art derivative-free solvers that use quadratic interpolation models. We demonstrate numerically that despite using only linear residual models, DFO-GN performs comparably to these methods in terms of objective evaluations. Furthermore, as a result of the simplified interpolation procedure, DFO-GN has superior runtime and scalability. Our implementation of DFO-GN is available at https://github.com/numericalalgorithmsgroup/dfogn (https://doi.org/10.5281/zenodo.2629875).

Common pulse retrieval algorithm: a fast and universal method to retrieve ultrashort pulses

We present a common pulse retrieval algorithm (COPRA) that can be used for a broad category of ultrashort laser pulse measurement schemes including frequency-resolved optical gating (FROG), interferometric FROG, dispersion scan, time domain ptychography, and pulse shaper assisted techniques such as multiphoton intrapulse interference phase scan (MIIPS). We demonstrate its properties in comprehensive numerical tests and show that it is fast, reliable and accurate in the presence of Gaussian noise. For FROG it outperforms retrieval algorithms based on generalized projections and ptychography. Furthermore, we discuss the pulse retrieval problem as a nonlinear least-squares problem and demonstrate the importance of obtaining a least-squares solution for noisy data. These results improve and extend the possibilities of numerical pulse retrieval. COPRA is faster and provides more accurate results in comparison to existing retrieval algorithms. Furthermore, it enables full pulse retrieval from measurements for which no retrieval algorithm was known before, e.g., MIIPS measurements.

Geometric Calibration of Continuum Robots: Joint Space and Equilibrium Shape Deviations

Currently, surgical continuum robots (CRs) are predominantly used as telemanipulators where modeling errors are overcome by the user. Such errors preclude their use for autonomous tasks. In this paper, we investigate the calibration of CRs with specific focus on capturing joint space errors due to homing offsets, assembly errors causing twist about the robot's backbone, and uncertainty in the equilibrium bending shapes of segments of these robots. A kinematic framework focusing on multibackbone CR is presented with emphasis on deriving calibration identification Jacobians. This framework captures the coupling between twist and the equilibrium shapes of a continuum segment as a function of its bending angle. To capture equilibrium shape variations as a function of bending, a homotopy of curves is defined and represented by respective modal coefficients. The estimation of the calibration parameters is cast as a nonlinear least-squares problem. The framework is validated by simulations and experimentally using a single-port access surgery robot. We believe this calibration framework will facilitate semiautomation of surgical tasks carried out by CRs.

A novel unscented Kalman filter for recursive state-input-system identification of nonlinear systems

Abstract The unscented Kalman filter (UKF) has proven to be an effective approach for the identification of nonlinear systems from limited output measurements. However, the conventional UKF requires that measurements of the input excitations are available to successfully perform nonlinear system identification, which limits its application in cases where it is difficult or impractical to measure the inputs. In this paper a novel unscented Kalman filter with unknown input (UKF-UI) is proposed for the simultaneous identification of nonlinear structural systems and external excitations. Based on the estimation-based procedures of the conventional UKF, the analytical recursive solutions of the proposed UKF-UI are derived in an analogous fashion resulting in a recursive nonlinear least-squares problem for the unknown input. Moreover, data fusion of partially measured acceleration and displacement responses is used to alleviate the drifts typically observed in the estimated inputs and displacements. Numerical and experimental validation examples are used to demonstrate the effectiveness of the proposed UKF-UI algorithm for the simultaneous identification of nonlinear parameters and unknown external excitations using data fusion of partially measured system responses.

A parameter estimation method using linear response statistics: Numerical scheme.

This paper presents a numerical method to implement the parameter estimation method using response statistics that was recently formulated by the authors. The proposed approach formulates the parameter estimation problem of Itô drift diffusions as a nonlinear least-squares problem. To avoid solving the model repeatedly when using an iterative scheme in solving the resulting least-squares problems, a polynomial surrogate model is employed on appropriate response statistics with smooth dependence on the parameters. The existence of minimizers of the approximate polynomial least-squares problems that converge to the solution of the true least square problem is established under appropriate regularity assumption of the essential statistics as functions of parameters. Numerical implementation of the proposed method is conducted on two prototypical examples that belong to classes of models with a wide range of applications, including the Langevin dynamics and the stochastically forced gradient flows. Several important practical issues, such as the selection of the appropriate response operator to ensure the identifiability of the parameters and the reduction of the parameter space, are discussed. From the numerical experiments, it is found that the proposed approach is superior compared to the conventional approach that uses equilibrium statistics to determine the parameters.

Broken Rotor Bar Fault Detection of Induction Motors Using a Joint Algorithm of Trust Region and Modified Bare-bones Particle Swarm Optimization

A precise detection of the fault feature parameter of motor current is a new research hotspot in the broken rotor bar (BRB) fault diagnosis of induction motors. Discrete Fourier transform (DFT) is the most popular technique in this field, owing to low computation and easy realization. However, its accuracy is often limited by the data window length, spectral leakage, fence effect, etc. Therefore, a new detection method based on a global optimization algorithm is proposed. First, a BRB fault current model and a residual error function are designed to transform the fault parameter detection problem into a nonlinear least-square problem. Because this optimization problem has a great number of local optima and needs to be resolved rapidly and accurately, a joint algorithm (called TR-MBPSO) based on a modified bare-bones particle swarm optimization (BPSO) and trust region (TR) is subsequently proposed. In the TR-MBPSO, a reinitialization strategy of inactive particle is introduced to the BPSO to enhance the swarm diversity and global search ability. Meanwhile, the TR is combined with the modified BPSO to improve convergence speed and accuracy. It also includes a global convergence analysis, whose result proves that the TR-MBPSO can converge to the global optimum with the probability of 1. Both simulations and experiments are conducted, and the results indicate that the proposed detection method not only has high accuracy of parameter estimation with short-time data window, e.g., the magnitude and frequency precision of the fault-related components reaches 10−4, but also overcomes the impacts of spectral leakage and non-integer-period sampling. The proposed research provides a new BRB detection method, which has enough precision to extract the parameters of the fault feature components.

Convergence and evaluation-complexity analysis of a regularized tensor-Newton method for solving nonlinear least-squares problems

Given a twice-continuously differentiable vector-valued function r(x), a local minimizer of $$\Vert r(x)\Vert _2$$ is sought. We propose and analyse tensor-Newton methods, in which r(x) is replaced locally by its second-order Taylor approximation. Convergence is controlled by regularization of various orders. We establish global convergence to a first-order critical point of $$\Vert r(x)\Vert _2$$ , and provide function evaluation bounds that agree with the best-known bounds for methods using second derivatives. Numerical experiments comparing tensor-Newton methods with regularized Gauss–Newton and Newton methods demonstrate the practical performance of the newly proposed method.

����� ������-�������-����� ��� ���I�I���� ����� ��������� �������I� �� ������������ ���� �I�����

There are a lot of dierent physical processes modeling of which lead us to a nonlinear least squares problems. Usually, they arise in cases of modeling or evaluating physical processes by values obtained as a result of measurements. In this article, we consider problems in which the nonlinear function consists of a dierentiable and nondierentiable part. For numerical solving such nonlinear least-squares problems, we proposed the Gauss-NewtonPotra method. Since the full Jacobian does not exist, this method uses instead of Jacobian the sum of the derivative of the dierentiable part of the operator and the combination of rst-order divided dierences of the nondierentiable part of the operator. The local convergence analysis of the method is conducted under the generalized Lipschitz conditions for rst-order derivatives and rst- and second-order divided dierences. These conditions, instead of Lipschitz constant, contain some positive integrable function. The classical Lipschitz conditions are a partial case of the generalized Lipschitz conditions. An equation for the radius of convergence domain and error estimates were obtained. A convergence order of the method in case of zero residual was established. We carry out numerical experiments on a set of test problems and compare results with other known methods by the number of iterations. The obtained results show that the Gauss-Newton-Potra method is more ecient than iterative-dierence methods.

Modified Gram-Schmidt Method-Based Variable Projection Algorithm for Separable Nonlinear Models.

Separable nonlinear models are very common in various research fields, such as machine learning and system identification. The variable projection (VP) approach is efficient for the optimization of such models. In this paper, we study various VP algorithms based on different matrix decompositions. Compared with the previous method, we use the analytical expression of the Jacobian matrix instead of finite differences. This improves the efficiency of the VP algorithms. In particular, based on the modified Gram-Schmidt (MGS) method, a more robust implementation of the VP algorithm is introduced for separable nonlinear least-squares problems. In numerical experiments, we compare the performance of five different implementations of the VP algorithm. Numerical results show the efficiency and robustness of the proposed MGS method-based VP algorithm.

A multiscale kinetics model for the analysis of starch amylolysis

Simple exponential decaying functions are commonly used for fitting the kinetics of starch digested by amylolytic enzymes. A common assumption is that a sole exponential function can account for the kinetics of the whole digestible starch. Recent studies using logarithm-of-slope (LOS) plots showed that digestion kinetics can exhibit multi-scale behavior, an effect reflecting starch fractions with different digestion characteristics. This work proposed an extension of the widely used Goñi et al.'s model to account for two starch fractions; one fraction linked with fast digestion rate and other with slow digestion rates. The fitting of experimental data was carried out by solving numerically a nonlinear least-squares problem. The estimated parameters have a straightforward interpretation in terms of reaction rates and digestible/resistant starch fractions. Two experimental examples were used for illustrating the performance of the multi-exponential function.

A brief survey of methods for solving nonlinear least-squares problems

In this paper, we present a brief survey of methods for solving nonlinear least-squares problems. We pay specific attention to methods that take into account the special structure of the problems. Most of the methods discussed belong to the quasi-Newton family (i.e. the structured quasi-Newton methods (SQN)). Our survey comprises some of the traditional and modern developed methods for nonlinear least-squares problems. At the end, we suggest a few topics for further research.

State estimation of zinc air batteries using neural networks

The main task of battery management systems is to keep the working area of the battery in a safe state. Estimation of the state of charge and the state of health is therefore essential. The traditional way uses the voltage level of a battery to determine those values. Modern metal air batteries provide a flat voltage characteristic which necessitates new approaches. One promising technique is the electrochemical impedance spectroscopy, which measures the AC resistance for a set of different frequencies. Previous approaches match the measured impedances with a nonlinear equivalent circuit, which needs a lot of time to solve a nonlinear least-squares problem. This paper combines the electrochemical impedance spectroscopy with neural networks to speed up the state estimation using the example of zinc air batteries. Moreover, these networks are trained with different subsets of the spectra as input data in order to determine the required number of frequencies.

A structured diagonal Hessian approximation method with evaluation complexity analysis for nonlinear least squares

This work proposes a Jacobian-free strategy for addressing large-scale nonlinear least-squares problems, in which structured secant conditions are used to define a diagonal approximation for the Hessian matrix. Proper safeguards are devised to ensure descent directions along the generated sequence. Worst-case evaluation analysis is provided within the framework of a non-monotone line search. Numerical experiments contextualize the proposed strategy, by addressing structured problems from the literature, also solved by related and recently presented conjugate gradient and multivariate spectral gradient strategies, as well as the classic Fletcher–Reeves conjugate gradient, and the Raydan–Barzilai–Borwein methods. The comparative computational results show a favorable performance of the proposed approach, mainly as far as robustness is concerned.

A Levenberg–Marquardt method for large nonlinear least-squares problems with dynamic accuracy in functions and gradients

In this paper we consider large scale nonlinear least-squares problems for which function and gradient are evaluated with dynamic accuracy and propose a Levenberg---Marquardt method for solving such problems. More precisely, we consider the case in which the exact function to optimize is not available or its evaluation is computationally demanding, but approximations of it are available at any prescribed accuracy level. The proposed method relies on a control of the accuracy level, and imposes an improvement of function approximations when the accuracy is detected to be too low to proceed with the optimization process. We prove global and local convergence and complexity of our procedure and show encouraging numerical results on test problems arising in data assimilation and machine learning.

Signal-modelling methods applied to the free-field calibration of hydrophones and projectors in laboratory test tanks

In reverberant laboratory tanks, free-field acoustic calibrations using hydrophones and projectors are limited by the arrival of boundary reflections, and by start-up transients caused by the resonant behaviour of the transducers. This paper describes the application of a signal modelling method which enables measurements to be undertaken at acoustic frequencies below the limits imposed by the echo-free time of the test tank. In the approach, the signals obtained during calibration are modelled, initially by a simple model for the steady-state transducer response, and then by an extended model consisting of terms that are used to describe both the steady-state and resonant behaviours of the device(s). This model may be further extended to include terms that describe the response both to the direct signal and to reflections of the signal from the tank boundaries. A non-linear least-squares problem involving data for all discrete frequencies of measurement is then solved to provide improved estimates of the model parameters and echo arrival times. The method is applied to the calibration of low Q-factor transducers such as hydrophones in the frequency range 250 Hz–1 kHz, and to high Q-factor source transducers in the frequency range 1 kHz–5 kHz, using measurements undertaken in a modest-sized tank where the echo-free time does not allow steady-state conditions to be reached. The calibrations were validated by comparison with both pressure calibrations in a small coupler, and free-field calibrations at an open water facility, with the agreement obtained of better than 1 dB, well within expanded uncertainties (which range between 0.5 and 1.2 dB).

On an Elliptical Trust-Region Procedure for Ill-Posed Nonlinear Least-Squares Problems

In this paper, we address the stable numerical solution of ill-posed nonlinear least-squares problems with small residual. We propose an elliptical trust-region reformulation of a Levenberg–Marquardt procedure. Thanks to an appropriate choice of the trust-region radius, the proposed procedure guarantees an automatic choice of the free regularization parameters that, together with a suitable stopping criterion, ensures regularizing properties to the method. Specifically, the proposed procedure generates a sequence that even in case of noisy data has the potential to approach a solution of the unperturbed problem. The case of constrained problems is considered, too. The effectiveness of the procedure is shown on several examples of ill-posed least-squares problems.