Nonlinear Least-squares Minimization Research Articles

ElecSus: Extension to arbitrary geometry magneto-optics

We present a major update to ElecSus, a computer program and underlying model to calculate the electric susceptibility of an alkali-metal atomic vapour. Knowledge of the electric susceptibility of a medium is essential to predict its absorptive and dispersive properties. In this version we implement several changes which significantly extend the range of applications of ElecSus, the most important of which is support for non-axial magnetic fields (i.e. fields which are not aligned with the light propagation axis). Supporting this change requires a much more general approach to light propagation in the system, which we have now implemented. We exemplify many of these new applications by comparing ElecSus to experimental data. In addition, we have developed a graphical user interface front-end which makes the program much more accessible, and have improved on several other minor areas of the program structure. Program summaryProgram Title: ElecSusProgram Files doi:http://dx.doi.org/10.17632/h7cj8bz4bd.1Licensing provisions: Apache License, Version 2.0Programming language: PythonExternal routines/libraries: SciPy library [1] 0.15.0 or later, NumPy [1], matplotlib [2], sympy [3], lmfit 0.9.5 or later [4], wxpython (required for GUI only)Nature of problem: Calculating the weak-probe electric susceptibility of an alkali-metal vapour. The electric susceptibility can be used to calculate spectra such as transmission and Stokes parameters. Measurements of experimental parameters can be made by fitting the theory to data.Solution method: The transition frequencies and wavelengths are calculated using a matrix representation of the Hamiltonian in the completely uncoupled basis. A suite of fitting methods are provided in order to allow user supplied experimental data to be fit to the theory, thereby allowing experimental parameters to be extracted.Restrictions: Results are only valid in the weak-probe regime.[1] T. E. Oliphant, Comput. Sci. Eng. 9, 10 (2007). http://www.scipy.org/[2] J. D. Hunter, Comput. Sci. Eng. 9, 10 (2007). http://matplotlib.org/[3] A. Meurer et. al, PeerJ Comp. Sci. 3, e103 (2017) http://www.sympy.org/[4] M. Newville et al., LMFIT: Non-Linear Least-Square Minimization and Curve-Fitting for Python, Zenodo (2014). DOI:10.5281/zenodo.11813 https://lmfit.github.io/lmfit-py/

Effect of permeability on foam-model parameters: An integrated approach from core-flood experiments through to foam diversion calculations

We present a set of steady-state foam-flood experimental data for four sandstones with different permeabilities, ranging between 6 and 1900 mD, and with similar porosity. We derive permeability-dependent foam parameters with two modelling approaches, those of Boeije and Rossen (2015a) and a non-linear least-square minimization approach (Eftekhari et al., 2015). The two approaches can yield significantly different foam parameters. Thus, we critically assess their ability in deriving reliable foam parameter estimates. In particular, the way the two approaches treat shear-thinning foam behaviour and foam coalescence is discussed. The foam parameter set acquired from the latter approach is further used as input in foam diversion calculations: this serves to evaluate mobility predictions in non-communicating reservoir layers. This study aims to provide a framework to integrate experimental work, modelling and simple qualitative diversion calculations to provide a background for the upscaling of foam studies, with particular focus on heterogeneous systems.

An inverse problem of finding the time-dependent thermal conductivity from boundary data

We consider the inverse problem of determining the time-dependent thermal conductivity and the transient temperature satisfying the heat equation with initial data, Dirichlet boundary conditions, and the heat flux as overdetermination condition. This formulation ensures that the inverse problem has a unique solution. However, the problem is still ill-posed since small errors in the input data cause large errors in the output solution. The finite difference method is employed as a direct solver for the inverse problem. The inverse problem is recast as a nonlinear least-squares minimization subject to physical positivity bound on the unknown thermal conductivity. Numerically, this is effectively solved using the lsqnonlin routine from the MATLAB toolbox. We investigate the accuracy and stability of results on a few test numerical examples.

Uranium, radium and thorium in soils with high-resolution gamma spectroscopy, MCNP-generated efficiencies, and VRF non-linear full-spectrum nuclide shape fitting

A new method for analysis of uranium and radium in soils by gamma spectroscopy has been developed using VRF (“Visual RobFit”) which, unlike traditional peak-search techniques, fits full-spectrum nuclide shapes with non-linear least-squares minimization of the chi-squared statistic. Gamma efficiency curves were developed for a 500 mL Marinelli beaker geometry as a function of soil density using MCNP. Collected spectra were then analyzed using the MCNP-generated efficiency curves and VRF to deconvolute the 90 keV peak complex of uranium and obtain 238 U and 235 U activities. 226 Ra activity was determined either from the radon daughters if the equilibrium status is known, or directly from the deconvoluted 186 keV line. 228 Ra values were determined from the 228 Ac daughter activity. The method was validated by analysis of radium, thorium and uranium soil standards and by inter-comparison with other methods for radium in soils. The method allows for a rapid determination of whether a sample has been impacted by a man-made activity by comparison of the uranium and radium concentrations to those that would be expected from a natural equilibrium state.

Retrieving the time-dependent thermal conductivity of an orthotropic rectangular conductor

The aim of this paper is to determine the thermal properties of an orthotropic planar structure characterized by the thermal conductivity tensor in the coordinate system of the main directions (Oxy) being diagonal. In particular, we consider retrieving the time-dependent thermal conductivity components of an orthotropic rectangular conductor from nonlocal overspecified heat flux conditions. Since only boundary measurements are considered, this inverse formulation belongs to the desirable approach of non-destructive testing of materials. The unique solvability of this inverse coefficient problem is proved based on the Schauder fixed point theorem and the theory of Volterra integral equations of the second kind. Furthermore, the numerical reconstruction based on a nonlinear least-squares minimization is performed using the MATLAB optimization toolbox routine lsqnonlin. Numerical results are presented and discussed in order to illustrate the performance of the inversion for orthotropic parameter identification.

Automatic Registration of Wide Area Motion Imagery to Vector Road Maps by Exploiting Vehicle Detections.

To enrich large-scale visual analytics applications enabled by aerial wide area motion imagery (WAMI), we propose a novel methodology for accurately registering a geo-referenced vector roadmap to WAMI by using the locations of detected vehicles and determining a parametric transform that aligns these locations with the network of roads in the roadmap. Specifically, the problem is formulated in a probabilistic framework, explicitly allowing for spurious detections that do not correspond to on-road vehicles. The registration is estimated via the expectation-maximization (EM) algorithm as the planar homography that minimizes the sum of weighted squared distances between the homography-mapped detection locations and the corresponding closest point on the road network, where the weights are estimated posterior probabilities of detections being on-road vehicles. The weighted distance minimization is efficiently performed using the distance transform with the Levenberg-Marquardt nonlinear least-squares minimization procedure, and the fraction of spurious detections is estimated within the EM framework. The proposed method effectively sidesteps the challenges of feature correspondence estimation, applies directly to different imaging modalities, is robust to spurious detections, and is also more appropriate than feature matching for a planar homography. Results over three WAMI data sets captured by both visual and infrared sensors indicate the effectiveness of the proposed methodology: both visual comparison and numerical metrics for the registration accuracy are significantly better for the proposed method as compared with the existing alternatives.

Identification of the time-dependent conductivity of an inhomogeneous diffusive material

In this paper, we consider a couple of inverse problems of determining the time-dependent thermal/hydraulic conductivity from Cauchy data in the one-dimensional heat/diffusion equation with space-dependent heat capacity/specific storage. The well-posedness of these inverse problems in suitable spaces of continuously differentiable functions are studied. For the numerical realisation, the problems are discretised using the finite-difference method and recast as nonlinear least-squares minimization problems with a simple positivity lower bound on the unknown thermal/hydraulic conductivity. Numerically, this is effectively solved using the lsqnonlin routine from the MATLAB toolbox. Regularization is included wherever necessary. Numerical results are presented and discussed for several benchmark test examples showing that accurate and stable numerical solutions are achieved. The outcomes of this study will be relevant and of significant importance to the applied mathematics inverse problems community working on thermal/hydraulic property determination in heat transfer and porous media.

A Moving Pseudo-Boundary MFS for Three-Dimensional Void Detection

We propose a new moving pseudo-boundary method of fundamental solutions (MFS) for the determination of the boundary of a three-dimensional void (rigid inclusion or cavity) within a conducting homogeneous host medium from overdetermined Cauchy data on the accessible exterior boundary. The algorithm for imaging the interior of the medium also makes use of radial spherical parametrization of the unknown star-shaped void and its centre in three dimensions. We also include the contraction and dilation factors in selecting the fictitious surfaces where the MFS sources are to be positioned in the set of unknowns in the resulting regularized nonlinear least-squares minimization. The feasibility of this new method is illustrated in several numerical examples. AMS subject classifications: 65N35,65N21,65N38

RISE: An Incremental Trust-Region Method for Robust Online Sparse Least-Squares Estimation

Many point estimation problems in robotics, computer vision, and machine learning can be formulated as instances of the general problem of minimizing a sparse nonlinear sum-of-squares objective function. For inference problems of this type, each input datum gives rise to a summand in the objective function, and therefore performing online inference corresponds to solving a sequence of sparse nonlinear least-squares minimization problems in which additional summands are added to the objective function over time. In this paper, we present Robust Incremental least-Squares Estimation (RISE), an incrementalized version of the Powell's Dog-Leg numerical optimization method suitable for use in online sequential sparse least-squares minimization. As a trust-region method, RISE is naturally robust to objective function nonlinearity and numerical ill-conditioning and is provably globally convergent for a broad class of inferential cost functions (twice-continuously differentiable functions with bounded sublevel sets). Consequently, RISE maintains the speed of current state-of-the-art online sparse least-squares methods while providing superior reliability.

A Moving Pseudo-Boundary MFS for Three-Dimensional Void Detection

AbstractWe propose a new moving pseudo-boundary method of fundamental solutions (MFS) for the determination of the boundary of a three-dimensional void (rigid inclusion or cavity) within a conducting homogeneous host medium from overdetermined Cauchy data on the accessible exterior boundary. The algorithm for imaging the interior of the medium also makes use of radial spherical parametrization of the unknown star-shaped void and its centre in three dimensions. We also include the contraction and dilation factors in selecting the fictitious surfaces where the MFS sources are to be positioned in the set of unknowns in the resulting regularized nonlinear least-squares minimization. The feasibility of this new method is illustrated in several numerical examples.

SU-D-217A-02: Effects of Enegy-Window Width and Spectral Effective Energy on Estimation of Gamma Camera Deadtime Using the Decay Method.

Assessment of count-rate performance (CRP) is part of routine quality assurance (QA) for gamma cameras. Manufacturers often specify deadtime (t) based on the NEMA decay method (NEMA 1-2007). The spectral conditions prescribed by NEMA are often difficult to duplicate during routine clinical QA testing and the appropriate energy window is poorly defined. The objective of this investigation is to evaluate the effects of energy-window selection and spectral conditions on estimates of t using the NEMA decay method. CRP was evaluated intrinsically over a period of 48 hours with Tc-99m for two Siemens Symbia gamma camera detectors using the NEMA decay method. CRP measurement was repeated with varying amounts of scattering material (0-8 cm of acrylic) placed between the source and detectors to transform the incident spectrum. CRP measurements were also repeated using different photopeak-window widths (2-75%) as well as an open energy-window. The CRP data were fit to the paralyzable detector model via non-linear least-squares minimization to estimate t. Estimates of t increased linearly with decreasing spectral effective energy. The effective energy was varied from 142-to-99 keV, which consequently altered the estimates of t by ∼0.095μs or 21% (p<0.01). Additionally, estimates of t increased as a power-law when the fraction of measured counts in the photopeak window decreased relative to open-window (total) counts. The ratio photopeak-window counts to open- window counts were varied from 1-to-0.12 which consequently altered the estimates of t by ∼8μs or 180%. The estimated change in t between 15% and 20% photopeak window is -0.13μs or 12.5%. Repeat measurements demonstrated that estimates of t using the decay method are precise to <2%. Deadtime (t) determined using the decay method varied significantly with the spectral effective energy and photopeak-window width. Careful attention to measurement conditions are imperative when measuring t using the decay method for comparison against manufacturer's specifications or trending as part of routine QA program.

Geometry of nonlinear least squares with applications to sloppy models and optimization

Parameter estimation by nonlinear least-squares minimization is a common problem that has an elegant geometric interpretation: the possible parameter values of a model induce a manifold within the space of data predictions. The minimization problem is then to find the point on the manifold closest to the experimental data. We show that the model manifolds of a large class of models, known as sloppy models, have many universal features; they are characterized by a geometric series of widths, extrinsic curvatures, and parameter-effect curvatures, which we describe as a hyper-ribbon. A number of common difficulties in optimizing least-squares problems are due to this common geometric structure. First, algorithms tend to run into the boundaries of the model manifold, causing parameters to diverge or become unphysical before they have been optimized. We introduce the model graph as an extension of the model manifold to remedy this problem. We argue that appropriate priors can remove the boundaries and further improve the convergence rates. We show that typical fits will have many evaporated parameters unless the data are very accurately known. Second, "bare" model parameters are usually ill-suited to describing model behavior; cost contours in parameter space tend to form hierarchies of plateaus and long narrow canyons. Geometrically, we understand this inconvenient parametrization as an extremely skewed coordinate basis and show that it induces a large parameter-effect curvature on the manifold. By constructing alternative coordinates based on geodesic motion, we show that these long narrow canyons are transformed in many cases into a single quadratic, isotropic basin. We interpret the modified Gauss-Newton and Levenberg-Marquardt fitting algorithms as an Euler approximation to geodesic motion in these natural coordinates on the model manifold and the model graph, respectively. By adding a geodesic acceleration adjustment to these algorithms, we alleviate the difficulties from parameter-effect curvature, improving both efficiency and success rates at finding good fits.

Virus Transport through Unsaturated Zone: Analysis and Parameter Identification

AbstractThis paper presents numerical and parameter estimation models for the analysis of virus transport and the identification of transport parameters in the unsaturated zone. The numerical model couples a mass conservative fully implicit finite difference model simulating moisture flow in the unsaturated zone with the hybrid finite volume model for virus transport. The accuracy of the numerical scheme is tested for both advection- and dispersion-dominated transport. The comparison of the numerical model with the analytical solution indicates that the numerical model’s predictions are in excellent agreement with the analytical predictions. The parameter estimation is formulated as a nonlinear least-squares minimization problem in which the parameters are estimated by minimizing the deviations between the model-predicted and experimentally observed virus concentrations. A parameter estimation procedure is developed by coupling the numerical model simulating one-dimensional virus transport in the unsatura...

A secant method for nonlinear least-squares minimization

Quasi-Newton methods have played a prominent role, over many years, in the design of effective practical methods for the numerical solution of nonlinear minimization problems and in multi-dimensional zero-finding. There is a wide literature outlining the properties of these methods and illustrating their performance (e.g., Dennis and Schnabel, Numerical Methods for Unconstrained Optimization and Nonlinear Equations, 1996). In addition, most modern optimization libraries house a quasi-Newton collection of codes and they are widely used. The quasi-Newton contribution to practical nonlinear optimization is unchallenged. In this paper we propose and investigate an efficient quasi-Newton (secant) approach to the nonlinear least-squares problem, made practical due to the selective application of automatic differentiation (AD) technology. We also observe that AD technology can increase the efficiency of the standard quasi-Newton (positive definite secant) approach to the full nonlinear minimization approach to this problem and we compare these two AD-assisted methods. Finally, we compare the AD-assisted approaches to a standard globalized Gauss-Newton method.

Tomographic Reconstruction of Elastic Constants in Composite Materials Using Numerical and Experimental Laser Ultrasonic Data

Nondestructive characterization of composite materials has been attempted using the concept of ultrasonic tomography. A new method, Composite-ART (C-ART) is proposed for measuring the 2-d distribution of elastic constants. This leads to a nonlinear relation between the time-of-flight and the field variables. The solution approach employs a nonlinear least-square minimization with appropriate constraints to attain a reliable solution. The method turns out be a multiple parameter reconstruction as the elastic constants characterize a pixel. Extensive studies using simulated data are carried out to establish the robustness of the method. These studies provide the guidelines with regards to choice of the relaxation parameter λ and an initial guess as the controlling parameters for the reconstruction of a cross-section. The sensitivity of the controlling parameters towards the solution behavior is also discussed in detail. The experimental studies were done using laser-based ultrasound as the source radiation allowing one to exploit the point source and point detector approximation. Unidirectional composites with defects were scanned and good quality reconstructions are obtained. It is observed that a low relaxation parameter helps in presence of noise due to measurement error.

Styrene/Acrylic Acid Random Copolymers Synthesized by Nitroxide-Mediated Polymerization: Effect of Free Nitroxide on Kinetics and Copolymer Composition

Styrene/acrylic acid (S/AA) mixtures were copolymerized in concentrated 50 wt % 1,4-dioxane solutions at 120 °C at two ratios of additional free nitroxide mediator, N,N-tert-butyl-N-[1′-diethylphosphono-2,2′-dimethylpropyl] nitroxide (SG1), relative to 2-[N-tert-butyl-2,2-(dimethylpropyl)aminooxy]propionic acid (BlocBuilder, Arkema) alkoxyamine unimolecular initiator (4.5 and 9 mol % [SG1]/[BlocBuilder]). Without SG1 at initial acrylic acid monomer feed concentrations fAA,0 > 40 mol %, the apparent rate constants increased sharply, noticeable exotherms were observed, and polydispersities increased from 1.20 at fAA,0 = 0 mol % to 1.48 at fAA,0 = 80 mol %. With 4.5 mol % [SG1]/[BlocBuilder], polymerization rates were slower and not as strongly affected by fAA,0 although exotherms were still noticeable at high fAA,0. Polydispersities remained ∼1.3 and only increased to >1.4 at high conversions for fAA,0 = 80 mol %. Exotherms were rendered nearly negligible when 9 mol % [SG1]/[BlocBuilder] was used, leading to copolymers with much narrower molecular weight distributions and kpK values for comonomer mixtures bracketed between those of styrene and acrylic acid homopolymerizations at 120 °C (kp = propagation rate constant, K = equilibrium constant). Copolymer reactivity ratios estimated for samples produced using 9 mol % [SG1]/[BlocBuilder] by nonlinear least-squares minimization were rAA = 0.25 ± 0.11 and rS = 0.93 ± 0.37, in agreement with previous literature.

Application of genetic algorithms for the extraction of electrical parameters of multicrystalline silicon

The application of the genetic algorithms (GAs) concept in measurements science deals with a fitting procedure used for the numerical prediction of physical parameters from an experimental data curve. In this work, GAs were applied for the extraction of electrical parameters of multicrystalline silicon solar cells. The experimental technique used is the light-beam-induced-current (LBIC). From LBIC measurements, we deduced the values of the diffusion length L and the grain boundary recombination velocity Vr of the minority carriers of the multicrystalline silicon wafers using the fitting procedure. The nonlinear fitting procedure is based on the minimization of the standard deviation of the theoretical LBIC profile from the experimental one. However, this criterion is not convex, and using traditional deterministic optimization algorithms leads to local minima solutions. To overcome this problem, the nonlinear least-square minimization technique was computed with the GAs strategy, increasing the probability of obtaining the best minimum value of the cost function in very reasonable time. The results of the proposed algorithm are presented and compared with the Levenberg–Marquardt algorithm and the Gauss–Newton method. The GAs-based numerical technique was found to be a promising and a powerful technique for numerical evaluation of the electrical parameters of silicon solar cells.

Deconvolution of Variable-Rate Reservoir Performance Data Using B-Splines

Summary We use B-splines for representing the derivative of the unknown unit-rate drawdown pressure and numerical inversion of the Laplace transform to formulate a new deconvolution algorithm. When significant errors and inconsistencies are present in the data functions, direct and indirect regularization methods are incorporated. We provide examples of under- and over-regularization, and we discuss procedures for ensuring proper regularization. We validate our method using synthetic examples generated without and with errors (up to 10%). Upon validation, we then demonstrate our deconvolution method using a variety of field cases, including traditional well tests, permanent downhole gauge data, and production data. Our work suggests that the new deconvolution method has broad applicability in variable rate/pressure problems and can be implemented in typical well-test and production-data-analysis applications. Introduction The constant-rate drawdown pressure behavior of a well/reservoir system is the primary signature used to classify/establish the characteristic reservoir model. Transient-well-test procedures typically are designed to create a pair of controlled flow periods (a pressure-drawdown/-buildup sequence) and to convert the last part of the response (the pressure buildup) to an equivalent constant-rate drawdown by means of special time transforms. However, the presence of wellbore storage, previous flow history, and rate variations may mask or distort characteristic features in the pressure and rate responses. With the ever-increasing ability to observe downhole rates, it has long been recognized that variable-rate deconvolution should be a viable option to traditional well-testing methods because deconvolution can provide an equivalent constant-rate response for the entire time span of observation. This potential advantage of variable-rate deconvolution has become particularly obvious with the appearance of permanent downhole instrumentation. First and foremost, variable-rate deconvolution is mathematically ill-conditioned; while numerous methods have been developed and applied to deconvolve "ideal" data, very few deconvolution methods perform well in practice. The ill-conditioned nature of the deconvolution problem means that small changes in the input data cause large variations in the deconvolved constant-rate pressures. Mathematically, we are attempting to solve a first-kind Volterra equation [see Lamm (2000)] that is ill-posed. However, in our case the kernel of the Volterra-type equation is the flow-rate function (i.e., the generating function); this function is not known analytically but, rather, is approximated from the observed flow rates. In practical terms, this issue adds to the complexity of the problem (Stewart et al. 1983). In the literature related to variable-rate deconvolution, we find the development of two basic concepts. One concept is to incorporate an a priori knowledge regarding the properties of the deconvolved constant-rate response. The observations of Coats et al. (1964) on the strict monotonicity of the solution led Kuchuk et al. (1990) to impose a "nonpositive second derivative" constraint on pressure response. In some respects, this tradition is maintained in the work given by von Schroeter et al. (2004), Levitan (2003), and Gringarten et al. (2003) when they incorporate non-negativity in the "encoding of the solution." We note that in the examples given, this concept (non-negativity/monotonicity of the solution) requires less-straightforward numerical methods (e.g., nonlinear least-squares minimization). The second concept is to use a certain level of regularization (von Schroeter et al. 2004; Levitan 2003; Gringarten et al. 2003), where "regularization" is defined as the act or process of making a system regular or standard (smoothing or eliminating nonstandard or irregular response features). Regularization can be performed indirectly, by representing the desired solution with a restricted number of "elements," or directly, by penalizing the nonsmoothness of the solution. In either case, the additional degree of freedom (the regularization parameter) has to be established, where this is facilitated by the discrepancy principle (effectively tuning the regularization parameter to a maximum value while not causing intolerable deviation between the model and the observations). In some fashion, each deconvolution algorithm developed to date combines these two concepts (non-negativity/monotonicity of the solution or regularization).

Amplitude modulated sinusoidal signal decomposition for audio coding

In this letter, we present a decomposition for sinusoidal coding of audio, based on an amplitude modulation of sinusoids via a linear combination of arbitrary basis vectors. The proposed method, which incorporates a perceptual distortion measure, is based on a relaxation of a nonlinear least-squares minimization. Rate-distortion curves and listening tests show that, compared to a constant-amplitude sinusoidal coder, the proposed decomposition offers perceptually significant improvements in critical transient signals

An approach based on the SIR measurement model for determining the ionization chamber efficiency curves, and a study of [formula omitted] and [formula omitted] photon emission intensities

The measurement model used to determine ionization chamber efficiency curves accounts from the outset for impurity corrections and beta spectrum shapes. The curves are represented by exponentials of polynomials whose coefficients are adjusted using non-linear least-squares minimization. The curves are validated by comparing with SIR key comparison reference values (KCRVs) and other published curves. The associated covariance matrix is also evaluated. Deviations from model predictions for 65 Zn and 201 Tl using recommended nuclear data are studied.