Nonlinear Least-squares Problem Research Articles

Nonintrusive Current Sensing for Multicore Cables Considering Inclination With Magnetic Field Measurement

Magnetic sensor arrays have been employed to measure currents for multicore cables due to their advantages of compact size, lightweight, low power consumption, and large dynamic range. However, the existing methods are applicable only to cases where the cable must be perpendicular to the sensor array plane due to the lack of capability of the 2-D magnetic field model to consider conductor inclination. To solve this problem, this article presents a method using a circular array of single-axis magnetic sensors to measure magnetic fields along and perpendicular to the sensor array plane. A 3-D magnetic field model that involves conductor incline angles is first derived. On the basis of the derived model, two main steps are performed to realize current sensing. First, an off-site calibration method is used to calibrate magnetic sensor parameters, including sensor position, orientation, gain factor, and phase shift. Second, not only the conductor currents and positions but also the incline angles are simultaneously calculated based on the detected magnetic fields and calibrated sensor parameters. Both steps are formulated as nonlinear least-square problems and solved efficiently. Both simulation and experiment are performed to demonstrate the superiority of the proposed method compared to the existing ones based on the 2-D magnetic field model when inclinations are introduced. The proposed method may have the potential for designing a clamp-on current sensor dedicated to multicore cables similar to hand-held current probes with a relatively large inner diameter for single-conductor systems.

New fourth- and sixth-order classes of iterative methods for solving systems of nonlinear equations and their stability analysis

In this paper, a two-step class of fourth-order iterative methods for solving systems of nonlinear equations is presented. We further extend the two-step class to establish a new sixth-order family which requires only one additional functional evaluation. The convergence analysis of the proposed classes is provided under several mild conditions. A complete dynamical analysis is made, by using real multidimensional discrete dynamics, in order to select the most stable elements of both families of fourth- and sixth-order of convergence. To get this aim, a novel tool based on the existence of critical points has been used, the parameter line. The analytical discussion of the work is upheld by performing numerical experiments on some application-oriented problems. We provide an implementation of the proposed scheme on nonlinear optimization problem and zero-residual nonlinear least-squares problems taken from the constrained and unconstrained testing environment test set. Finally, based on numerical results, it has been concluded that our methods are comparable with the existing ones of similar nature in terms of order, efficiency, and computational time and also that the stability results provide the most efficient member of each class of iterative schemes.

A Subsurface Targets’ Classification Method Utilizing Gradient Learning Technique

Recent advances in the time-domain electromagnetic (TDEM) method have dramatically improved detection and discrimination of subsurface targets. Inversion of observed response using a 3-D orthogonal magnetic dipolar model provides location, orientation, and intrinsic responses of the target based on deterministic optimization methods, which is dependent on the initial values and could be trapped in local minimum solutions. In this letter, we applied a supervised descent method (SDM) to the inversion of electromagnetic induction (EMI) data accurately by individually and simultaneously training every single sample in the training set to avoid the direct use of the SDM that causes inaccurate classification results. This method provides a new way to incorporate prior information using gradient learning and reduce the computational complexity as it does not compute partial derivatives in the nonlinear least-squares problem or groups difference in the heuristic random search algorithm. Then the simulation and field experiments are performed to verify the feasibility of this method. Both the simulation and experimental results demonstrate that the SDM shows good performance and robustness in the classification of subsurface anomalous targets.

Constant Envelope MIMO-OFDM Precoding for Low Complexity Large-Scale Antenna Array Systems

Herein, we consider constant envelope precoding in a multiple-input multiple-output orthogonal frequency division multiplexing system (CE MIMO-OFDM) for frequency selective channels. In CE precoding the signals for each transmit antenna are designed to have constant amplitude regardless of the channel realization and the information symbols that must be conveyed to the users. This facilitates the use of power-efficient components, such as phase shifters (PS) and nonlinear power amplifiers, which are key for the feasibility of large-scale antenna array systems because of their low cost and power consumption. The CE precoding problem is firstly formulated as a least-squares problem with a unit modulus constraint and solved using an algorithm based on coordinate descent. The large number of optimization variables in the case of the MIMO-OFDM system motivates the search for a more computationally efficient solution. To tackle this, we reformulate the CE precoding design into an unconstrained nonlinear least-squares problem, which is solved efficiently using the Gauss-Newton algorithm. Simulation results underline the efficiency of the proposed solutions and show that they outperform state of the art techniques.

Chebyshev pseudospectral method in the reconstruction of orthotropic conductivity

In this paper, we present a method to reconstruct the spatially varying conductivity tensor in isotropic and orthotropic materials, involved in a two-dimensional transient anisotropic model with Robin boundary conditions. For the reconstruction, the partial differential equation is solved by a semi-discrete method that combines a pseudospectral collocation method for spatial variables and Crank–Nicolson for time. The conductivity tensor is reconstructed through a non-linear least-squares problem solved by Levenberg–Marquardt method (LMM), along with Morozov's discrepancy principle as stopping rule to cope with noise in the data. Unlike classic LMM implementations that mitigate poor conditioning in calculating iterates using nonsingular diagonal scaling matrices, in this paper, singular regularization matrices are used. The impact of such a modification is illustrated with numerical experiments using discrete differential operators as scaling matrices. Numerical results show that accurate conductivity values can be obtained using a fairly small number of discretization points at a very low computational cost.

Supporting multi-point fan design with dimension reduction

AbstractMotivated by the idea of turbomachinery active subspace performance maps, this paper studies dimension reduction in turbomachinery 3D CFD simulations. First, we show that these subspaces exist across different blades—under the same parametrisation—largely independent of their Mach number or Reynolds number. This is demonstrated via a numerical study on three different blades. Then, in an attempt to reduce the computational cost of identifying a suitable dimension reducing subspace, we examine statistical sufficient dimension reduction methods, including sliced inverse regression, sliced average variance estimation, principal Hessian directions and contour regression. Unsatisfied by these results, we evaluate a new idea based on polynomial variable projection—a non-linear least-squares problem. Our results using polynomial variable projection clearly demonstrate that one can accurately identify dimension reducing subspaces for turbomachinery functionals at a fraction of the cost associated with prior methods. We apply these subspaces to the problem of comparing design configurations across different flight points on a working line of a fan blade. We demonstrate how designs that offer a healthy compromise between performance at cruise and sea-level conditions can be easily found by visually inspecting their subspaces.

Modal-Based Kinematics and Contact Detection of Soft Robots.

Soft robots offer an alternative approach to manipulate within a constrained space while maintaining a safe interaction with the external environment. Owing to its adaptable compliance characteristic, external contact force can easily deform the robot shapes and lead to undesired robot kinematic and dynamic properties. Accurate contact detection and contact location estimation are of critical importance for soft robot modeling, control, trajectory planning, and eventually affect the success of task completion. In this article, we focus on the investigation of a one degree of freedom (1-DoF) soft pneumatic bending robot, which is regarded as one of the fundamental components to construct complex, multi-DoFs soft robots. This 1-DoF soft robot is modeled through the integral representation of the spatial curve, where direct and instantaneous kinematics are calculated explicitly through a modal method. The fixed centrode deviation method is used to detect the external contact and estimate the contact location. Simulation results and experimental studies indicate that the contact location can be accurately estimated by solving a nonlinear least-square optimization problem. Experimental validation shows that the proposed algorithm is able to successfully estimate the contact location with the estimation error of 1.46 mm.

Detection of Structural Damage in Rotating Beams Using Modal Sensitivity Analysis and Sparse Regularization

Rotating beams are often encountered in the wind turbines and the rotors, and detection of the damages in rotating beams as earlier as possible is central to ensuring the safety and serviceability of practical structures. To this end, a modal sensitivity approach in conjunction with the sparse regularization is proposed in this paper. First, the eigen equations for the flap-wise and chord-wise vibrations of a rotating beam are established upon Hamilton’s principle. Then, damage detection is formulated as a nonlinear least-squares problem that finds the damage coefficients to minimize the error between the measured and calculated data. To solve the nonlinear least-squares problem, the sensitivity method that requires the modal sensitivity analysis is developed. In real applications, damage detection is usually an ill-posed problem and to circumvent the ill-posedness, the sparse regularization is introduced due to the fact that the numbers of actual damage locations are often scarce. Numerical examples are studied and results show that the proposed approach is more accurate than the enhanced sensitivity approach and the flap-wise modal data outperforms the chord-wise modal data in damage detection of rotating beams.

A regularization method for constrained nonlinear least squares

We propose a regularization method for nonlinear least-squares problems with equality constraints. Our approach is modeled after those of Arreckx and Orban (SIAM J Optim 28(2):1613–1639, 2018. https://doi.org/10.1137/16M1088570) and Dehghani et al. (INFOR Inf Syst Oper Res, 2019. https://doi.org/10.1080/03155986.2018.1559428) and applies a selective regularization scheme that may be viewed as a reformulation of an augmented Lagrangian. Our formulation avoids the occurrence of the operator $$A(x)^T A(x)$$, where A is the Jacobian of the nonlinear residual, which typically contributes to the density and ill conditioning of subproblems. Under boundedness of the derivatives, we establish global convergence to a KKT point or a stationary point of an infeasibility measure. If second derivatives are Lipschitz continuous and a second-order sufficient condition is satisfied, we establish superlinear convergence without requiring a constraint qualification to hold. The convergence rate is determined by a Dennis–More-type condition. We describe our implementation in the Julia language, which supports multiple floating-point systems. We illustrate a simple progressive scheme to obtain solutions in quadruple precision. Because our approach is similar to applying an SQP method with an exact merit function on a related problem, we show that our implementation compares favorably to IPOPT in IEEE double precision.

The application of nonlinear least-squares estimation algorithms in atmospheric density model calibration

PurposeThe purpose of this paper is to focus on the performance of three typical nonlinear least-squares estimation algorithms in atmospheric density model calibration.Design/methodology/approachThe error of Jacchia-Roberts atmospheric density model is expressed as an objective function about temperature parameters. The estimation of parameter corrections is a typical nonlinear least-squares problem. Three algorithms for nonlinear least-squares problems, Gauss–Newton (G-N), damped Gauss–Newton (damped G-N) and Levenberg–Marquardt (L-M) algorithms, are adopted to estimate temperature parameter corrections of Jacchia-Roberts for model calibration.FindingsThe results show that G-N algorithm is not convergent at some sampling points. The main reason is the nonlinear relationship between Jacchia-Roberts and its temperature parameters. Damped G-N and L-M algorithms are both convergent at all sampling points. G-N, damped G-N and L-M algorithms reduce the root mean square error of Jacchia-Roberts from 20.4% to 9.3%, 9.4% and 9.4%, respectively. The average iterations of G-N, damped G-N and L-M algorithms are 3.0, 2.8 and 2.9, respectively.Practical implicationsThis study is expected to provide a guidance for the selection of nonlinear least-squares estimation methods in atmospheric density model calibration.Originality/valueThe study analyses the performance of three typical nonlinear least-squares estimation methods in the calibration of atmospheric density model. The non-convergent phenomenon of G-N algorithm is discovered and explained. Damped G-N and L-M algorithms are more suitable for the nonlinear least-squares problems in model calibration than G-N algorithm and the first two algorithms have slightly fewer iterations.

A Method for Solving LiDAR Waveform Decomposition Parameters Based on a Variable Projection Algorithm

Light detection and ranging (LiDAR) is commonly used to create high-resolution maps; however, the efficiency and convergence of parameter estimation are difficult. To address this issue, we evaluated the structural characteristics of received LiDAR signals by decomposing them into Gaussian functions and applied the variable projection algorithm of the separable nonlinear least-squares problem to the process of waveform fitting. First, using a variable projection algorithm, we separated the linear (amplitude) and nonlinear (center position and width) parameters in the Gaussian function model; the linear parameters are expressed with nonlinear parameters by the function. Thereafter, the optimal estimation of the characteristic parameters of the Gaussian function components was transformed into a least-squares problem only comprising nonlinear parameters. Finally, the Levenberg–Marquardt algorithm was used to solve these nonlinear parameters, whereas the linear parameters were calculated simultaneously in each iteration, and the estimation results satisfying the nonlinear least-square criterion were obtained. Five groups of waveform decomposition simulation data and ICESat/GLAS satellite LiDAR waveform data were used for the parameter estimation experiments. During the experiments, for the same accuracy, the separable nonlinear least-squares optimization method required fewer iterations and lesser calculation time than the traditional method of not separating parameters; the maximum number of iterations was reached before the traditional method converged to the optimal estimate. The method of separating variables only required 14 iterations to obtain the optimal estimate, reducing the computational time from 1128 s to 130 s. Therefore, the application of the separable nonlinear least-squares problem can improve the calculation efficiency and convergence speed of the parameter solution process. It can also provide a new method for parameter estimation in the Gaussian model for LiDAR waveform decomposition.

Gauss–Newton–Kurchatov Method for the Solution of Nonlinear Least-Squares Problems

We propose and study an iterative method for the solution of a nonlinear least-squares problem with nondifferentiable operator. In this method, instead of the Jacobian, we use the sum of derivative of the differentiable part of the operator and divided difference with specially chosen points of the nondifferentiable part of operator. We prove a theorem substantiating the process of convergence of the proposed method and establish its rate. We also present the results of numerical experiments carried out for the test problems with nondifferentiable operators.

A Discontinuous Dual Reciprocity Method in Conjunction with a Regularized Levenberg–Marquardt Method for Source Term Recovery in Inhomogeneous Anisotropic Materials

This paper outlines a new approach to identify a source term of a [Formula: see text]D elliptic equation for anisotropic nonhomogenous media. The proposed methodology is based on the minimization of an objective function representing differences between the measured potential and those calculated by using the discontinuous dual reciprocity boundary element method, the measurements are required to render a unique solution and supposed to be pointwise in the problem domain. Since the additional data may be contaminated by measurement noises or the numerical computing errors, we adopt a regularizing Levenberg–Marquardt method to solve the nonlinear least-squares problem attained from the inverse source problem. The numerical performance of the proposed approach is studied at the end for both geometries: smooth and piecewise smooth one. The results show a very good agreement with the analytical solutions under exact and noisy data.

Local Phase Velocity Based Imaging of Viscoelastic Phantoms and Tissues.

Assessment of soft tissue elasticity and viscosity is of interest in several clinical applications. In this study, we present the feasibility of the local phase velocity based imaging (LPVI) method to create images of phase velocity and viscoelastic parameters in viscoelastic tissue-mimicking materials and soft tissues. In viscoelastic materials, it is necessary to utilize wave-mode isolation using a narrow bandpass filter combined with a directional filter in order to robustly reconstruct phase velocity images with LPVI in viscoelastic media over a wide range of frequencies. A pair of sequential focused acoustic radiation force push beams, focused once on the left-hand side and once on the right-hand side of the probe, was used to produce broadband propagating shear waves. The local shear wave phase velocity is then recovered in the frequency domain for multiple frequencies, for both acquired data sets. Then, a 2-D shear wave velocity map is reconstructed by combining maps from two separate acquisitions. By testing the method on simulated data sets and performing in vitro phantom and in vivo liver tissue experiments, we show the ability of the proposed technique to generate shear wave phase velocity maps at various frequencies in viscoelastic materials. Moreover, a nonlinear least-squares problem is solved in order to locally estimate elasticity and viscosity parameters. The LPVI method with added directional and wavenumber filters can produce phase velocity images, which can be used to characterize the viscoelastic materials.

Accurate and ultra-fast estimation of Brillouin frequency shift for distributed fiber sensors

The original Lorentzian profile-based nonlinear least-squares problem for BFS estimation is converted to a linear least-squares one. Then, Brillouin frequency shift (BFS) can be readily calculated. The typical nonlinear least-squares Lorentzian, Gaussian, pseudo-Voigt and Voigt fits using Levenberg-Marquardt algorithm, the correlation-based algorithm and the proposed Lorentzian model-based linear least-squares fitting algorithm are used to estimate BFS of the measured Brillouin spectra with different values of SNR (signal-to-noise ratio) and frequency step. The results reveal that the proposed algorithm has similar accuracy with the nonlinear fits. However, the computation time of the typical nonlinear fits is 187.56–11481.67 times as long as the proposed algorithm. The proposed algorithm can estimate BFS with less computational burden and higher accuracy than the correlation-based algorithm. If SNR is low, the frequency step should be set to a low value.

The minimal-norm Gauss-Newton method and some of its regularized variants

Nonlinear least-squares problems appear in many real-world applications. When a nonlinear model is used to reproduce the behavior of a physical system, the unknown parameters of the model can be estimated by fitting experimental observations by a least-squares approach. It is common to solve such problems by Newton's method or one of its variants such as the Gauss-Newton algorithm. In this paper, we study the computation of the minimal-norm solution to a nonlinear least-squares problem, as well as the case where the solution minimizes a suitable semi-norm. Since many important applications lead to severely ill-conditioned least-squares problems, we also consider some regularization techniques for their solution. Numerical experiments, both artificial and derived from an application in applied geophysics, illustrate the performance of the different approaches.

Deep Generative Endmember Modeling: An Application to Unsupervised Spectral Unmixing

Endmember (EM) spectral variability can greatly impact the performance of standard hyperspectral image analysis algorithms. Extended parametric models have been successfully applied to account for the EM spectral variability. However, these models still lack the compromise between flexibility and low-dimensional representation that is necessary to properly explore the fact that spectral variability is often confined to a low-dimensional manifold in real scenes. In this paper we propose to learn a spectral variability model directly form the observed data, instead of imposing it \emph{a priori}. This is achieved through a deep generative EM model, which is estimated using a variational autoencoder (VAE). The encoder and decoder that compose the generative model are trained using pure pixel information extracted directly from the observed image, what allows for an unsupervised formulation. The proposed EM model is applied to the solution of a spectral unmixing problem, which we cast as an alternating nonlinear least-squares problem that is solved iteratively with respect to the abundances and to the low-dimensional representations of the EMs in the latent space of the deep generative model. Simulations using both synthetic and real data indicate that the proposed strategy can outperform the competing state-of-the-art algorithms.

Local Convergence Analysis of the Levenberg–Marquardt Framework for Nonzero-Residue Nonlinear Least-Squares Problems Under an Error Bound Condition

The Levenberg–Marquardt method is widely used for solving nonlinear systems of equations, as well as nonlinear least-squares problems. In this paper, we consider local convergence properties of the method, when applied to nonzero-residue nonlinear least-squares problems under an error bound condition, which is weaker than requiring full rank of the Jacobian in a neighborhood of a stationary point. Differently from the zero-residue case, the choice of the Levenberg–Marquardt parameter is shown to be dictated by (i) the behavior of the rank of the Jacobian and (ii) a combined measure of nonlinearity and residue size in a neighborhood of the set of (possibly non-isolated) stationary points of the sum of squares function.

Parameter Identification of Nonlinear Aeroelastic System with Time-Delayed Feedback Control

In this paper, an enhanced response sensitivity approach is proposed for parameter identification of nonlinear time-delay aeroelastic systems. Two common aeroelastic models are investigated, namely, the aeroelastic system under the unsteady Theodorsen aerodynamic forces and the airfoil-store system under the steady incompressible flow. The work of this paper is mainly threefold. First, the motion equations of two aeroelastic models including time-delay signals are derived, and the response sensitivity analysis with respect to the system parameters is conducted. Then, parameter identification is formulated as a nonlinear least-squares problem, and the enhanced response sensitivity approach is developed to get the solution. Finally, numerical examples are studied to verify the effectiveness of the proposed method. The results reveal that the proposed approach can give accurate, robust, and efficient identification of the parameters for time-delay aeroelastic systems.

Improving the Flexibility and Robustness of Model-based Derivative-free Optimization Solvers

We present two software packages for derivative-free optimization (DFO): DFO-LS for nonlinear least-squares problems and Py-BOBYQA for general objectives, both with optional bound constraints. Inspired by the Gauss-Newton method, DFO-LS constructs simplified linear regression models for the residuals and allows flexible initialization for expensive problems, whereby it can begin making progress after as few as two objective evaluations. Numerical results show DFO-LS can gain reasonable progress on some medium-scale problems with fewer objective evaluations than is needed for one gradient evaluation. DFO-LS has improved robustness to noise, allowing sample averaging, regression-based model construction, and multiple restart strategies with an auto-detection mechanism. Our extensive numerical experimentation shows that restarting the solver when stagnation is detected is a cheap and effective mechanism for achieving robustness, with superior performance over sampling and regression techniques. The package Py-BOBYQA is a Python implementation of BOBYQA (Powell 2009), with novel features such as the implementation of robustness to noise strategies. Our numerical experiments show that Py-BOBYQA is comparable to or better than existing general DFO solvers for noisy problems. In our comparisons, we introduce an adaptive accuracy measure for data profiles of noisy functions, striking a balance between measuring the true and the noisy objective improvement.