Nonlinear Least-squares Minimization Research Articles

Constructing Approximations to Bivariate Piecewise-Smooth Functions

This paper demonstrates that the space of piecewise-smooth bivariate functions can be well-approximated by the space of the functions defined by a set of simple (non-linear) operations on smooth uniform tensor product splines. The examples include bivariate functions with jump discontinuities or normal discontinuities across curves, and even across more involved geometries such as a three-corner discontinuity. The provided data may be uniform or non-uniform, and noisy, and the approximation procedure involves non-linear least-squares minimization. Also included is a basic approximation theorem for functions with jump discontinuity across a smooth curve.

An inverse problem for pseudoparabolic equation: existence, uniqueness, stability, and numerical analysis

In this paper, we study an inverse problem for a linear third-order pseudoparabolic equation. The investigated inverse problem consists of determining a space-dependent coefficient of the right-hand side of a pseudoparabolic equation. As an additional information a final overdetermination condition is considered. Under the suitable conditions on the data of the problem, the unique solvability of the considered inverse problem is established. The stability of solutions is also proved. The established results are also true for inverse problems for parabolic equations, which could be obtained as a regularization of the studied pseudoparabolic equation. In addition, the pseudoparabolic problem is discretized using the cubic B-spline functions and recast as a nonlinear least-squares minimization of the Tikhonov regularization function. Numerically, this is effectively solved using the MATLAB subroutine lsqnonlin. Both exact and noisy data are inverted. Numerical results for three benchmark test examples are presented and discussed. Moreover, the von Neumann stability analysis is also discussed.

A Study of Correlative Modeling on Viscosity for Some Organic Binary Liquid Systems

In order to systematically investigate the correlating capability of viscosity models, six well-known correlative models of Grunberg-Nissan (GN), Hind (HND), Heric (HRC), Ausländer (AUS) for dynamic viscosity, η, and McAllister 3-body (MAC3) and McAllister 4-body (MAC4) for kinematic viscosity, ν, were employed and tested for viscosity data of 83 organic binary liquid systems consisting of 33 different aromatic hydrocarbons (ArH), alkanes (RH), cycloalkanes (CyRH) and alkanols (ROH). Keeping ArH as a common component and increasing the chain length of other components, the systems were categorized as Category 1: ArH + RH, Category 2: ArH + CyRH, Category 3: ArH + ArH and Category 4: ArH + ROH. For all the models fitting parameters along with the statistical parameters such as SPD σ(%), ASPD σ(%), OASPD σ(%) and GOASPD σ(%) were computed by the Nonlinear Least Squares Minimization (NLSM) technique with the ‘Solver’ add-in package. Among the four categories, Category 3, OASPD, σ(%) values lie between 0.16 and 0.21, indicating that all the models fitted extremely well. However, for dynamic viscosities, the η AUS model demonstrated the best correlating capabilities with GOASPD σ(%) of 0.48, while the HND model performed the poorest with GOASPD σ(%) at 3.8. For the kinematic viscosities, ν in both the MAC3 and MAC4 models yielded satisfactory results with GOASPD σ(%) as < 1.0 %.

An inverse source problem for a pseudoparabolic equation with memory

<abstract><p>This paper is devoted to investigating the well-posedness, as well as performing the numerical analysis, of an inverse source problem for linear pseudoparabolic equations with a memory term. The investigated inverse problem involves determining a right-hand side that depends on the spatial variable under the given observation at a final time along with the solution function. Under suitable assumptions on the problem data, the existence, uniqueness and stability of a strong generalized solution of the studied inverse problem are obtained. In addition, the pseudoparabolic problem is discretized using extended cubic B-spline functions and recast as a nonlinear least-squares minimization of the Tikhonov regularization function. Numerically, this problem is effectively solved using the MATLAB subroutine <italic>lsqnonlin</italic>. Both exact and noisy data are inverted. Numerical results for a benchmark test example are presented and discussed. Moreover, the von Neumann stability analysis is also discussed.</p></abstract>

Efficient transient correlation of thermal lumped network models to reference data

In this paper, an efficient algorithm for correlating the transient behaviour of thermal lumped models to reference data is reported. The algorithm is based on an iterative non-linear least-squares minimization process that allows for fast convergence. The correlation process requires the calculation of the so-called Moore–Penrose pseudoinverse of the Jacobian matrix. However, obtaining the transient Jacobian matrix can be computationally expensive for large models. A new method for calculating the transient Jacobian matrix of lumped thermal models, called Jacobian Propagation, is also presented. For large models, the Jacobian Propagation method, when combined with an implicit solver, has a negligible computational cost compared to the solving process. The correlation algorithm and the Jacobian Propagation method have been successfully tested with two thermal models, obtaining better performance than previous approaches.

Inverse problems of damped wave equations with Robin boundary conditions: an application to blood perfusion

Knowledge of the properties of biological tissues is essential in monitoring any abnormalities that may be forming and have a major impact on organs malfunctioning. Therefore, these disorders must be detected and treated early to save lives and improve the general health. Within the framework of thermal therapies, e.g. hyperthermia or cryoablation, the knowledge of the tissue temperature and of the blood perfusion rate are of utmost importance. Therefore, motivated by such a significant biomedical application, this paper investigates, for the first time, the uniqueness and stable reconstruction of the space-dependent (heterogeneous) perfusion coefficient in the thermal-wave hyperbolic model of bio-heat transfer from Cauchy boundary data using the powerful technique of Carleman estimates. Additional novelties consist in the consideration of Robin boundary conditions, as well as developing a mathematical analysis that leads to stronger stability estimates valid over a shorter time interval than usually reported in the literature of coefficient identification problems for hyperbolic partial differential equations. Numerically, the inverse coefficient problem is recast as a nonlinear least-squares minimization that is solved using the conjugate gradient method (CGM). Both exact and noisy data are inverted. To achieve stability, the CGM is stopped according to the discrepancy principle. Numerical results for a physical example are presented and discussed, showing the convergence, accuracy and stability of the inversion procedure.

Approximating the span of principal components via iterative least-squares

In the course of the last century, Principal Component Analysis (PCA) have become one of the pillars of modern scientific methods. Although PCA is normally addressed as a statistical tool aiming at finding orthogonal directions on which the variance is maximized, its first introduction by Pearson at 1901 was done through defining a non-linear least-squares minimization problem of fitting a plane to scattered data points. Thus, it seems natural that PCA and linear least-squares regression are somewhat related, as they both aim at fitting planes to data points. In this paper, we present a connection between the two approaches. Specifically, we present an iterated linear least-squares approach, yielding a sequence of subspaces, which converges to the space spanned by the leading principal components (i.e., principal space).

Determination of time-dependent coefficient in time fractional heat equation

The aim of this work is to determine the time-dependent heat coefficient in a type of inverse problem for one dimensional time-fractional heat equations defined by the Caputo operator for (0<α<1) by applying a method based on the finite difference scheme and Tikhonov’s regularization. First, a stable implicit finite difference scheme is applied to find a numerical solution to the (forward) direct problem. While the inverse problem was reformulated as a nonlinear least-square minimization problem with a simple physical bound on the unknown coefficient and solved efficiently by MATLAB routine lsqnonlin from the optimization toolbox. But the latter problem will remain ill-posed because the presence of any error in the input data will lead to a large error in the output data. So, Tikhonov’s technique is applied to obtain stable results. In the end, two numerical test examples show that the proposed method is stable, accurate, and works well with different noise levels

Statistical Analysis of the Measurement Noise in Dynamic Impedance Spectra.

Different strategies can be used to acquire dynamic impedance spectra during a cyclic voltammetry experiment. The spectra are then analyzed by fitting them with a model using a weighted non‐linear least‐squares minimization algorithm. The choice of the weighting factors is not trivial and influences the value of the extracted parameters. At variance with the classic electrochemical impedance spectroscopy, dynamic impedance measurements are performed under non‐stationary conditions, making them typically more prone to errors arising from the voltage and current analog‐to‐digital conversion. Under the assumption that the noise in the voltage and current signals have a constant variance along the measurement and that it is uncorrelated between distinct samples, we calculate an expression for the expected variance of the error of the resulting immittances, which considers the specific procedure used to extract the spectra under the time‐varying nature of the measurements. The calculated variance of the error is then used as a rigorous way to evaluate the weighting factors of the least‐squares minimization, assuming that the fitted model is ideally exact and that there are no systematic errors. By exploring two classical electrochemical systems and fitting the measured spectra with a transfer function measurement model, namely with the Padé approximants, we show that the variance evaluated with our method captures the frequency dependence of the resulting residuals and can be used for reliably performing the complex non‐linear least‐squares fitting procedure.

A new logistic growth model applied to COVID-19 fatality data

Background:Recent work showed that the temporal growth of the novel coronavirus disease (COVID-19) follows a sub-exponential power-law scaling whenever effective control interventions are in place. Taking this into consideration, we present a new phenomenological logistic model that is well-suited for such power-law epidemic growth. Methods:We empirically develop the logistic growth model using simple scaling arguments, known boundary conditions and a comparison with available data from four countries, Belgium, China, Denmark and Germany, where (arguably) effective containment measures were put in place during the first wave of the pandemic. A non-linear least-squares minimization algorithm is used to map the parameter space and make optimal predictions. Results:Unlike other logistic growth models, our presented model is shown to consistently make accurate predictions of peak heights, peak locations and cumulative saturation values for incomplete epidemic growth curves. We further show that the power-law growth model also works reasonably well when containment and lock down strategies are not as stringent as they were during the first wave of infections in 2020. On the basis of this agreement, the model was used to forecast COVID-19 fatalities for the third wave in South Africa, which was in progress during the time of this work. Conclusion:We anticipate that our presented model will be useful for a similar forecasting of COVID-19 induced infections/deaths in other regions as well as other cases of infectious disease outbreaks, particularly when power-law scaling is observed.

Identifying an unknown heat source term in the third-order pseudo-parabolic equation from nonlocal integral observation

The aim of this paper is to find the time-dependent heat source term numerically in the third-order pseudo-parabolic equation with initial and boundary conditions supplemented by nonlocal integral observation. This is a very interesting and challenging nonlinear inverse source problem with important applications in various fields of mechanics and physics. Unique solvability theorem of this inverse problem is supplied. However, since the problem is still ill-posed (small errors in the integral input data cause large errors in the output force) the solution needs to be regularized. Therefore, to obtain a stable solution, a regularized objective function is minimized to retrieve the unknown heat source coefficient. The third-order pseudo-parabolic problem is discretized using the finite difference method (FDM) and recast as nonlinear least-squares minimization of the Tikhonov regularization function with simple bounds on the unknown force coefficient. Numerically, this is effectively solved using the MATLAB subroutine lsqnonlin. Both analytical and perturbed (noisy) data are inverted. Numerical results for three benchmark test examples are presented and discussed. In addition, the von Neumann stability analysis is also discussed.

Efficient ptychographic phase retrieval via a matrix-free Levenberg-Marquardt algorithm.

The phase retrieval problem, where one aims to recover a complex-valued image from far-field intensity measurements, is a classic problem encountered in a range of imaging applications. Modern phase retrieval approaches usually rely on gradient descent methods in a nonlinear minimization framework. Calculating closed-form gradients for use in these methods is tedious work, and formulating second order derivatives is even more laborious. Additionally, second order techniques often require the storage and inversion of large matrices of partial derivatives, with memory requirements that can be prohibitive for data-rich imaging modalities. We use a reverse-mode automatic differentiation (AD) framework to implement an efficient matrix-free version of the Levenberg-Marquardt (LM) algorithm, a longstanding method that finds popular use in nonlinear least-square minimization problems but which has seen little use in phase retrieval. Furthermore, we extend the basic LM algorithm so that it can be applied for more general constrained optimization problems (including phase retrieval problems) beyond just the least-square applications. Since we use AD, we only need to specify the physics-based forward model for a specific imaging application; the first and second-order derivative terms are calculated automatically through matrix-vector products, without explicitly forming the large Jacobian or Gauss-Newton matrices typically required for the LM method. We demonstrate that this algorithm can be used to solve both the unconstrained ptychographic object retrieval problem and the constrained "blind" ptychographic object and probe retrieval problems, under the popular Gaussian noise model as well as the Poisson noise model. We compare this algorithm to state-of-the-art first order ptychographic reconstruction methods to demonstrate empirically that this method outperforms best-in-class first-order methods: it provides excellent convergence guarantees with (in many cases) a superlinear rate of convergence, all with a computational cost comparable to, or lower than, the tested first-order algorithms.

Defining a master curve of abdominal aortic aneurysm growth and its potential utility of clinical management

ObjectiveThe maximum diameter measurement of an abdominal aortic aneurysm (AAA), which depends on orthogonal and axial cross-sections or maximally inscribed spheres within the AAA, plays a significant role in the clinical decision making process. This study aims to build a total of 21 morphological parameters from longitudinal CT scans and analyze their correlations. Furthermore, this work explores the existence of a “master curve” of AAA growth, and tests which parameters serve to enhance its predictability for clinical use. Methods106 CT scan images from 25 Korean AAA patients were retrospectively obtained. We subsequently computed morphological parameters, growth rates, and pair-wise correlations, and attempted to enhance the predictability of the growth for high-risk aneurysms using non-linear curve fitting and least-square minimization. ResultsAn exponential AAA growth model was fitted to the maximum spherical diameter, as the best representative of the growth among all parameters (r-square: 0.94) and correctly predicted to 15 of 16 validation scans based on a 95% confidence interval. AAA volume expansion rates were highly correlated (r=0.75) with thrombus accumulation rates. ConclusionsThe exponential growth model using spherical diameter provides useful information about progression of aneurysm size and enables AAA growth rate extrapolation during a given surveillance period.

Investigations on Mediterranean biomass pyrolysis ability by thermogravimetric analyses: thermal behaviour and sensitivity of kinetic parameters

A comparison of the thermal behaviours of different Mediterranean biomasses, based on the evaluation of their pyrolysis characteristic temperatures, their reactivity and kinetic parameters is presented. Parameters such as the activation energy and the pre-exponential factor of the pyrolysis reactions are determined by different methods (Kissinger, Kissinger–Akahira–Sunose [KAS], Coats–Redfern, nonlinear least-squares minimization [NLSM] and model distributed activation energy model [DAEM]). Furthermore, a sensitivity analysis of the kinetic parameters based on different methods is conducted. The comparison of this work with the literature, showed that thermal characteristic parameters determined using the thermogravimetric analysis (TGA) are often neglected and not used in biomass pyrolysis at laboratory scale. The kinetic parameters seem to be highly sensitive to the used kinetic methods. For a given biomass, such as the Aleppo pine husk residue, for example, the activation energy can vary from 65.80 to 197.08 kJ·mol -1 depending on the used method. For this biomass, the highest average activation energy (190 kJ·mol -1 ) was estimated by the KAS and DAEM methods. The Kissinger method yields to an activation energy close to that of cellulose calculated by the NLSM method. For all biomasses, the activation energy remains between 150 and 200 kJ·mol -1 except for the Coats–Redfern method, where this value is in the range of 50–100 kJ·mol -1 . Therefore, it is important to have a means of recommending the most appropriate method for the determination of kinetic parameters.

Barnes Update Applied in the Gauss-Newton Method: An Improved Algorithm to Locate Bond Breaking Points.

A mechanochemical reaction is a reaction induced by mechanical energy. A general accepted model for this type of reaction consists of a first-order perturbation on the associated potential energy surface (PES) of the unperturbed molecular system due to mechanical stress or pulling force. Within this theoretical framework, the so-called optimal barrier breakdown points or optimal bond breaking points (BBPs) are critical points of the unperturbed PES where the Hessian matrix has a zero eigenvector that coincides with the gradient vector. Optimal BBPs are "catastrophe points" that are particularly important because their associated gradient indicates how to optimally harness tensile forces to induce reactions by transforming a chemical reaction into a barrierless process. Building on a previous method based on a nonlinear least-squares minimization to locate BBPs (Bofill et al., J. Chem. Phys. 2017, 147, 152710-10), we propose a new algorithm to locate BBPs of any molecular system based on the Gauss-Newton method combined with the Barnes update for a nonsymmetric Jacobian matrix, which is shown to be more appropriate than the Broyden update. The efficiency of the new method is demonstrated for a multidimensional model PES and two medium size molecular systems of interest in enzymatic catalysis and mechanochemistry.

INverse problem of identifying a time-dependent coefficient and free boundary in heat conduction equation by using the meshless local Petrov-Galerkin (MLPG) method via moving least squares approximation

In this paper we investigated the inverse problem of identifying an unknown time-dependent coefficient and free boundary in heat conduction equation. By using the change of variable we reduced the free boundary problem into a fixed boundary problem. In direct solver problem we employed the meshless local Petrov-Galerkin (MLPG) method based on the moving least squares (MLS) approximation. Inverse reduced problem with fixed boundary is nonlinear and we formulated it as a nonlinear least-squares minimization of a scalar objective function. Minimization is performed by using of f mincon routine from MATLoptimization toolbox accomplished with the Interior - point algorithm. In order to deal with the time derivatives, a two-step time discretization method is used. It is shown that the proposed method is accurate and stable even under a large measurement noise through several numerical experiments.

An Updated Model for Circumbinary Planets Orbiting the sdB Binary NY Virginis

Abstract We report 18 new primary minima timing observations of the short-period eclipsing binary system NY Virginis. We combined these minima with previously published primary minima to update circumbinary exoplanet models in this system based on O − C timing variations. We performed a nonlinear least-squares minimization search using a quadratic ephemeris and either one or two exoplanets. The only model with an acceptable fit includes a period derivative and two planets in eccentric orbits e = 0.15, 0.15 with minimum masses of 2.7 and 5.5 Jovian masses. Analysis of the orbit stability shows that this solution is stable for at least 108 years, but a small increase in eccentricity (e ≥ 0.20) for either planet renders the orbits unstable in less than 106 years. A number of model parameters are significantly degenerate, so additional observations are required to determine planetary parameters with high statistical confidence.

Analyzing dwell times with the Generalized Method of Moments.

The Generalized Method of Moments (GMM) is a statistical method for the analysis of samples from random processes. First developed for the analysis of econometric data, the method is here formulated to extract hidden kinetic parameters from measurements of single molecule dwell times. Our method is based on the analysis of cumulants of the measured dwell times. We develop a general form of an objective function whose minimization can return estimates of decay parameters for any number of intermediates directly from the data. We test the performance of our technique using both simulated and experimental data. We also compare the performance of our method to nonlinear least-squares minimization (NL-LSQM), a commonly-used technique for analysis of single molecule dwell times. Our findings indicate that the GMM performs comparably to NL-LSQM over most of the parameter range we explore. It offers some benefits compared with NL-LSQM in that it does not require binning, exhibits slightly lower bias and variance with small sample sizes (N<20), and is somewhat superior in identifying fast decay times with these same low count data sets. Additionally, a comparison with the Classical Method of Moments (CMM) shows that the CMM can fail in many cases, whereas the GMM always returns estimates. Our results show that the GMM can be a useful tool and complements standard approaches to analysis of single molecule dwell times.

The study of properties on generalized Beta distribution

Statistical distributions such as Normal, Log-Normal, Gamma and Beta distributions have been studied to determine the fitting ability of the distributions in different field of studies. The objective of this research is to study the properties of the generalized Beta distribution which is a 6 parameter model from the Beta family as well as to evaluate the prediction level of the model. The advantages of the generalized Beta distribution is they can be very versatile and flexible where it will provide a good description to many different types of data, including unimodal, uniantimodal, increasing, decreasing, or bath-tub shape distribution depending on the parameter values. The distributions were fitted to rainfall data collected from Sg Lui, Selangor using Non-Linear Least-Squares Minimization package of a Python software with the maximum likelihood estimation as the parameter estimation method. Simulation was proceeded using Accept-Reject algorithm where their predictive level were evaluated using model selection criteria such as Mean Absolute Error (MAE) and Root Mean Square Error (RMSE).

Non-invasive assessment of patient-specific aortic haemodynamics from four-dimensional flow MRI data.

We introduce a parameter estimation framework for automatically and robustly personalizing aortic haemodynamic computations from four-dimensional magnetic resonance imaging data. The framework is based on a reduced-order multiscale fluid-structure interaction blood flow model, and on two calibration procedures. First, Windkessel parameters of the outlet boundary conditions are personalized by solving a system of nonlinear equations. Second, the regional mechanical wall properties of the aorta are personalized by employing a nonlinear least-squares minimization method. The two calibration procedures are run sequentially and iteratively until both procedures have converged. The parameter estimation framework was successfully evaluated on 15 datasets from patients with aortic valve disease. On average, only 1.27 ± 0.96 and 7.07 ± 1.44 iterations were required to personalize the outlet boundary conditions and the regional mechanical wall properties, respectively. Overall, the computational model was in close agreement with the clinical measurements used as objectives (pressures, flow rates, cross-sectional areas), with a maximum error of less than 1%. Given its level of automation, robustness and the short execution time (6.2 ± 1.2 min on a standard hardware configuration), the framework is potentially well suited for a clinical setting.