Weighted Nonlinear Least Squares Research Articles

Additive modeling systems to simultaneously predict aboveground biomass and carbon for Litsea glutinosa of agroforestry model in tropical highlands

Aim of study: To develop and cross-validate simultaneous modeling systems for estimating components and total tree aboveground biomass and carbon of Litsea glutinosa in an agroforestry model with cassava.
 Area of study: In the Central Highlands of Vietnam, the agroforestry model widely planted on fallow land of ethnic minorities is a mixture of 65% L. glutinosa in combination with 35% cassava (Manihot esculenta).
 Material and methods: Twenty-two 300-m2 circular sample plots were located, representing the range of tree age, plantation density, and a 6-7 year rotation cycle. In each sample plot, one selected tree with a diameter at breast height equal to the plot quadratic mean diameter was destructively sampled. The relationships among tree aboveground biomass and carbon (AGB/AGC) and their components with dendrometric variables diameter, height, age, and crown area were examined using factor analysis. To fit systems of equations for AGB/AGC and their components, we compared two methods: weighted nonlinear least-squares (WNLS) and weighted nonlinear seemingly unrelated regression (WNSUR).
 Main results: The results of the leave-one-out cross-validation showed that the simultaneous WNSUR approach to modeling systems of four tree components, total biomass, and carbon provided better results than independent WNLS models.
 Research highlights: The simultaneous WNSUR modeling system provided improved and reliable estimates of tree components, total biomass, and carbon for L. glutinosa in an agroforestry model with cassava compared to independently fitted WNLS models.

A Majorization-Minimization Algorithm for Hybrid TOA-RSS Based Localization in NLOS Environment

This letter investigates the problem of accurate localization of a target node in wireless sensor networks using time of arrival (TOA) and received signal strength (RSS) measurements in an adverse non-line of sight (NLOS) environment. The proposed methodology works without any requirement of the NLOS path identification or the prior knowledge of NLOS bias distribution. A non-linear weighted least squares (NLWLS) problem is formulated through general approximations on the hybrid data model. The formulated NLWLS problem is solved using a computationally efficient majorization-minimization (MM) algorithm in which the NLWLS objective is iteratively minimized via simple update steps. The proposed MM algorithm is guaranteed to converge to a stationary point of the NLWLS objective. Simulation results and computational complexity analysis validate that the proposed MM algorithm attains fast convergence with lower latency. Moreover, the proposed hybrid localization algorithm outperforms the state-of-art methods in terms of estimation accuracy and computational complexity.

Error-Reduced Elliptic Positioning via Joint Estimation of Location and a Balancing Parameter

Elliptic positioning (EP) has been a topic of lively interest to localization practitioners owing to widespread adoption of the bistatic configuration in many location-enabling technologies nowadays. This letter addresses the problem of non-Gaussian error mitigation in EP, by formulating it as joint estimation of target position coordinates and a balancing parameter (BP) for the bias errors. An alternating minimization algorithm is put forward to break the original formulation down into a conventional weighted nonlinear least squares (WNLS) location estimator and a closed-form BP-update step. With its objective being properly decomposed, the WNLS subproblem is converted into the difference-of-convex programming framework, to which an efficient iterative solution based on the concave-convex procedure is applicable. Simulation results demonstrate that the proposed error-reduced EP approach can outperform a number of existing methods in terms of localization accuracy.

Robust Multistatic Target Localization in the Presence of NLOS Errors and Outliers

In this letter, we address the problem of multistatic target localization using time-sum-of-arrival (TSOA) measurements in the presence of errors due to non-line-of-sight (NLOS) propagation and outliers. To handle the effect of bias errors induced due to NLOS/outliers, an additional balancing parameter is incorporated in the data model. Then, a non-linear weighted least squares (NLWLS) based design objective is formulated for the joint estimation of source position and the balancing parameter. We then propose a simple iterative algorithm, which enjoys closed-form updates for the source position and the balancing parameter, based on the majorization-minimization (MM) technique to arrive at a minimizer of the NLWLS objective. Various numerical simulations are presented in the letter to demonstrate the accuracy of the proposed algorithm when compared to other existing algorithms in the literature.

PET-ABC: fully Bayesian likelihood-free inference for kinetic models

Aims. We describe an intuitive, easy to use method called PET-ABC that enables full Bayesian statistical inference from single subject dynamic PET data. The performance of PET-ABC was compared with weighted non-linear least squares (WNLS) in terms of reliability of kinetic parameter estimation and statistical power for model selection. Methods. Dynamic PET data based on 1-tissue and 2-tissue compartmental models were simulated with 2 noise models and 3 noise levels. PET-ABC was used to evaluate the reliability of parameter estimates under each condition. It was also used to perform model selection for a simulated noisy dataset composed of a mixture of 1- and 2-tissue compartment kinetics. Finally, PET-ABC was used to analyze a non-steady state dynamic [11C] raclopride study performed on a fully conscious rat administered either 2 mg.kg−1 amphetamine or saline 20 min after tracer injection. Results. PET-ABC yielded posterior point estimates for model parameters with smaller variance than WNLS, as well as probability density functions indicating confidence intervals for those estimates. It successfully identified the superiority of a 2-tissue compartment model to fit the simulated mixed model data. For the drug challenge study, the post observation probability of striatal displacement of the PET signal was 0.9 for amphetamine and approximately 0 for saline, indicating a high probability of amphetamine-induced endogenous dopamine release in the striatum. PET-ABC also demonstrated superior statistical power to WNLS (0.87 versus 0.09) for selecting the correct model in a simulated ligand displacement study. Conclusions. PET-ABC is a simple and intuitive method that provides complete Bayesian statistical analysis of single subject dynamic PET data, including the extent to which model parameter estimates and model choice are supported by the data. Software for PET-ABC is freely available as part of the PETabc package https://github.com/cgrazian/PETabc.

Joint Calibration and Synchronization of Two Arrays of Microphones and Loudspeakers Using Particle Swarm Optimization

This work presents a methodology for the joint calibration and synchronization of two arrays of microphones and loudspeakers. The problem is modeled as estimation of the rigid motion of one array with respect to the other, as well as estimation of the synchronization mismatch between the two. The proposed method uses dedicated signals emitted by the loudspeakers of the two arrays to compute a set of time of arrival (TOA) estimates. Through a simple transformation, estimated TOAs are converted into a set of linearly independent time difference of arrival (TDOA) measurements, which are modeled by a system of nonlinear equations in the unknown parameters of interest. A maximum likelihood estimate is then given as the solution to a nonlinear weighted least squares (NWLS) problem, which is optimized applying a parallelizable variant of Particle Swarm Optimization (PSO). In this paper, we also derive the Cramér-Rao lower bound (CRLB), and benchmark it against the proposed method in a series of Monte Carlo (MC) simulations. Results show that the proposed method attains high-performance comparable to the CRLB.

The tolerable waiting time: A generalized Pareto distribution model with empirical investigation

In China, it is more common for pedestrians than vehicles to disobey traffic signals, resulting in a high risk of pedestrian-vehicle accidents. Pedestrian waiting time are the most critical indicator of the tendency to violate traffic signals. A statistical analysis based on 4027 field-collected samples showed that the length of time that pedestrians are prepared to wait depends on the type of pedestrian traffic signal. Compared to a countdown-type signal, pedestrians were more likely to violate conventional-type signals. Furthermore, pedestrians were willing to wait longer during peak hours than during off-peak hours. There were no significant differences between the waiting times of male and female travelers. To predict pedestrian waiting time, we propose a generalized Pareto distribution (GPD) model and calibrated it based on our field data. Monte Carlo simulations showed that the maximum likelihood estimation (MLE), Bayesian MLE (BMLE), and weighted nonlinear least squares (WNLS) models are the best methods for estimating the scale and shape parameters of the GPD model. Several empirical results were output from the models. For example, at countdown-type signals, the 85th quantile of the tolerable waiting time in off-peak and peak hours was 51.5 and 54.4 s, respectively; the respective values for males and females were 55.4 and 55.0 s. At conventional signals, the tolerable waiting time was approximately 42.5 s. These findings are useful for the planning, design, and operation of pedestrian facilities.

Fitting the generalized Pareto distribution to data based on transformations of order statistics

ABSTRACTGeneralized Pareto distribution (GPD) has been widely used to model exceedances over thresholds. In this article we propose a new method called weighted nonlinear least squares (WNLS) to estimate the parameters of the GPD. The WNLS estimators always exist and are simple to compute. Some asymptotic results of the proposed method are provided. The simulation results indicate that the proposed method performs well compared to existing methods in terms of mean squared error and bias. Its advantages are further illustrated through the analysis of two real data sets.

Weighted Nonlinear Least Squares Technique for Parameters Estimation of the NHPP Gompertz Model

With the problem of heteroscedasticity an alternative precise estimation method of the nonlinear least squares (NLS) technique is needed. Weighted nonlinear least squares estimation (WNLSE) technique is an alternative that may increase the accuracy of parameters estimation by assigning suitable weights to the time between failures data. In the present study, the traditional maximum likelihood (ML), nonlinear least squares (NLS), and weighted nonlinear least squares (WNLS) techniques are formulated to estimate the three parameters of the NHPP Gompertz model. Empirical weighting method is investigated in NHPP Gompertz model prediction process. Three real software failure data examples are provided to analyze the performance of the three considered methods of estimation. The results of this numerical study indicate the preferences to the WNLSE method with respect to the NHPP Gompertz model’s performance, also the value of the weighting factors which give the optimum solution differ according to the nature of software failure data.

Parameter estimation for three-parameter generalized Pareto distribution by weighted non linear least squares

ABSTRACTGeneralized Pareto distribution (GPD) is widely used to model exceedances over thresholds. In this paper, we propose a new method, called weighted non linear least squares (WNLS), to estimate the parameters of the three-parameter GPD. Some asymptotic results of the proposed method are provided. An extensive simulation is carried out to evaluate the finite sample behaviour of the proposed method and to compare the behaviour with other methods suggested in the literature. The simulation results show that WNLS outperforms other methods in general situations. Finally, the WNLS is applied to analysis the real-life data.

Adaptive Weighted Nonlinear Least Squares Method for Fluorodeoxyglucose Positron Emission Tomography Quantification

This study developed an adaptive weighted nonlinear least squares (AWNLS) method for solving the problem of high variability in the estimates of the microrate constants of fluorodeoxyglucose (FDG) kinetics caused by measurement noise. In the AWNLS method for adaptive quantitative analysis, the cost function is adjusted according to the characteristics of the tissue time-activity curve (TTAC). Specifically, the average of the early part of the TTAC was used to modify the cost function when fitting the FDG model to the TTAC. A computer simulation study applying different sets of parameter values and noise conditions was conducted. The accuracy and reliability of the parameter estimates obtained using AWNLS were compared with those of nonlinear least squares (NLS), weighted nonlinear least squares (WNLS), linear least squares (LLS), and generalized linear least squares (GLLS). The errors in k1–k3 obtained using NLS indicate this method’s poor precision in the presence of high noise levels. NLS and WNLS were sensitive to the initial values. Moreover, the results of k4 estimated using LLS and GLLS were inaccurate because of large bias. By contrast, the microrate constants (k1–k4), the FDG metabolic rate (K), and the volume of distribution (k1/k2) obtained using AWNLS were stable and accurate regardless of the noise level and initial values. The AWNLS method could estimate the FDG metabolic rate (K) and the microrate constants (k 1–k 4) of the FDG model accurately at various noise levels, irrespective of the initial values.

FracFit: A robust parameter estimation tool for fractional calculus models

AbstractAnomalous transport cannot be adequately described with classical Fickian advection‐dispersion equations (ADE) with constant coefficients. Rather, fractional calculus models may be used, which capture salient features of anomalous transport (e.g., skewness and power law tails). FracFit is a parameter estimation tool based on space‐fractional and time‐fractional models used by the hydrology community. Currently, four fractional models are supported: (1) space‐fractional advection‐dispersion equation (sFADE), (2) time‐fractional dispersion equation with drift (TFDE), (3) fractional mobile‐immobile (FMIM) equation, and (4) temporally tempered Lévy motion (TTLM). Model solutions using pulse initial conditions and continuous injections are evaluated using stable distributions or subordination integrals. Parameter estimates are extracted from measured breakthrough curves (BTCs) using a weighted nonlinear least squares (WNLS) algorithm. Optimal weights for BTCs for pulse initial conditions and continuous injections are presented. Two sample applications are analyzed: (1) pulse injection BTCs in the Selke River and (2) continuous injection laboratory experiments using natural organic matter. Model parameters are compared across models and goodness‐of‐fit metrics are presented, facilitating model evaluation.

A Bias-Reduced Nonlinear WLS Method for TDOA/FDOA-Based Source Localization

We address the source localization problem by using both time-difference-of-arrival (TDOA) and frequency-difference-of-arrival (FDOA) measurements. We solve this problem in two steps, and in each step, we formulate a nonlinear weighted least squares (WLS) problem followed by a bias reduction scheme. In the first step, we formulate a nonlinear WLS problem using TDOA measurements only and derive the bias of the WLS solution, which is then used to develop an unbiased WLS solution by subtracting the bias from the WLS solution. In the second step, we formulate another nonlinear WLS problem by combining the results in the first step and the FDOA measurements. To avoid the potential risk of local convergence, this WLS problem is reduced to an approximate WLS problem, for which the globally optimal solution can be obtained. The bias of the WLS solution is also derived and then subtracted from the WLS solution to reduce the bias. Simulation results show that the bias of the proposed method is reduced and that the Cramér-Rao lower bound accuracy is also achieved.

Overlapped peaks resolution for linear sweep polarography using Gaussian-like distribution

A resolution method based on Gaussian-like distribution for overlapped linear sweep polarographic peaks was proposed to simultaneously detect the polymetallic components, such as Zn(II) and Co(II), coexisting in the leaching solution of zinc hydrometallurgy. A Gaussian-like distribution was constructed as the sub-model of overlapped peaks by analyzing the characteristics of linear sweep polarographic curve. Then, the abscissas of each peak and trough were pinpointed through multi-resolution wavelet decomposition, the curve and its derivative curves were fitted by using nonlinear weighted least squares (NWLS). Finally, overlapped peaks were resolved into independent sub-peaks based on fitted reconstruction parameters. The experimental results show that the relative error of half-wave potential pinpointed by multi-resolution wavelet decomposition is less than 1% and the accuracy of Ip fitted by NWLS is higher than 96%. The proposed resolution method is effective for overlapped linear sweep polarographic peaks of Zn(II) and Co(II).

Global-two-stage filtering of clinical PET parametric maps: Application to [11C]-(R)-PK11195

IntroductionIn Positron Emission Tomography (PET) quantification of physiological parameters at the voxel level may result in unreliable estimates due to the high noise of voxel time activity curves. Global-Two-Stage (GTS), an estimation technique belonging to the group of “population approaches”, can be used to tackle this problem. GTS was previously tested on simulated PET data and yielded substantial improvements when compared to standard estimation approaches such as Weighted NonLinear Least Squares (WNLLS) and Basis Function Method (BFM). In this work GTS performance is assessed in a clinical context using the neuroinflammation marker [11C]-(R)-PK11195 applied to a cohort of Huntington's disease (HD) patients with and without symptoms. Materials and methodsParametric maps of binding potential (BPND) of 12 normal controls (NC), 9 symptomatic and 9 presymptomatic HD patients were generated by applying a modified reference tissue model that accounts for tracer vascular activity in both reference and target tissues (SRTMV). GTS was then applied to SRTMV maps and its performance compared with that of SRTMV. Three smoothed versions of SRTMV, obtained by filtering the original SRTMV maps with Gaussian filters of 3mm, 5mm and 7mm Full Width Half Maximum (FWHM), were also included in the comparison. Since striatal degeneration is the hallmark of HD, sensitivity was assessed for all methods by computing the mean of z-scores in caudate, putamen and globus pallidus in the voxel-by-voxel statistical comparison of BPND between HD and NC. ResultsApplication of GTS to parametric maps brought a substantial qualitative improvement to SRTMV maps to the extent that anatomical structures often became visible. In addition, most parameter estimates that were outside the physiological range with SRTMV were corrected by GTS. GTS yielded a 2.3-fold increase in sensitivity with respect to SRTMV for the symptomatic cohort (mean of striatal z-scores of 0.76 for SRTMV and 1.79 for GTS) and an even more substantial increase for the presymptomatic cohort (mean of striatal z-scores of 0.34 for SRTMV and 0.96 for GTS). The sensitivity of GTS was similar to the one obtained with a filter of 7mm FWHM applied to the initial SRTMV maps but GTS images were not characterized by the notable loss of resolution typical of smoothed maps. GTS, additionally, does not require to change/define settings according to the tracer and level of noise, whereas the choice of the FWHM value of the Gaussian filter normally employed in the smoothing procedure is typically arbitrary. ConclusionsGTS is a powerful and robust tool for improving the quality of parametric maps in PET. The method is particularly appealing in that it can be applied to any tracer and estimation method, provided that initial estimates of the parameter vector and of its covariance are available. Although the benefits of GTS are far from being exhaustively assessed, the significant improvements obtained both on real and simulated data suggest that it could become an important tool for dynamic PET in the future.

LIDAR density and linear interpolator effects on elevation estimates

Linear interpolation of irregularly spaced LIDAR elevation data sets is needed to develop realistic spatial models. We evaluated inverse distance weighting (IDW) and ordinary kriging (OK) interpolation techniques and the effects of LIDAR data density on the statistical validity of the linear interpolators. A series of 10 forested 1000‐ha LIDAR tiles on the Lower Coastal Plain of eastern North Carolina was used. An exploratory analysis of the spatial correlation structure of the LIDAR data set was performed. Weighted non‐linear least squares (WNLS) analysis was used to parameterize best‐fit theoretical semivariograms on the empirical data. Tile data were sequentially reduced through random selection of a predetermined percentage of the original LIDAR data set, resulting in data sets with 50%, 25%, 10%, 5% and 1% of their original densities. Cross‐validation and independent validation procedures were used to evaluate root mean square error (RMSE) and kriging standard error (SE) differences between interpolators and across density sequences. Review of errors indicated that LIDAR data sets could withstand substantial data reductions yet maintain adequate accuracy (30 cm RMSE; 50 cm SE) for elevation predictions. The results also indicated that simple interpolation approaches such as IDW could be sufficient for interpolating irregularly spaced LIDAR data sets.

The Matérn function as a general model for soil variograms

The variogram is important in pedometrics for describing and quantifying soil spatial variability. Therefore, it is essential to have a model that can describe various spatial processes and to use appropriate techniques for estimating its parameters. The Matérn model is a generalization of several theoretical variogram models that incorporates a smoothness parameter. We show the flexibility of the Matérn model using simulation and apply the Matérn model to some soil data in Australia. Parameters of the Matérn model were determined by restricted maximum likelihood (REML), and weighted nonlinear least-squares (WNLS) on the empirical variogram. The Matérn model is shown to be flexible and can be used to describe many isotropic spatial soil processes. The REML method fits the local spatial process correctly, however the drawback is the lengthy computation. Meanwhile WNLS fits only the shape of the calculated empirical variogram, and parameters estimated from WNLS can be misleading. From this study, the smoothness parameter of soil data from point measurement appears to be in the range of 0.25–0.50 and can be considered to be a rough spatial process.

Properties of estimators of count data model with endogenous switching

We examine properties of estimators of count data model with endogenous switching. The estimation of the count data model that accommodates endogenous switching can be accomplished by full information maximum likelihood (FIML). However, FIML estimation requires fully and correctly specified model and is computationally burdensome. Alternative estimation methods do not require fully specified model have been proposed. The typical methods are two-stage method of moments (TSM) and nonlinear weighted least-squares (NWLS). The properties of these estimators have never been studied so far. In this paper, we compared the finite sample properties of these estimators under correct and incorrect model specifications using Monte Carlo experiments. We find that FIML estimator has the smallest standard deviation and TSM estimator has the largest.

A sequential minimization algorithm based on the convexification approach

A sequential minimization algorithm for the numerical solution of inverseproblems of frequency sounding is presented. This algorithm is based on theconcept of convexification of a multiextremal objective function proposed recentlyby the present authors. A key point in the sequential minimization algorithm isthat, unlike conventional layer-stripping algorithms, it provides the stableapproximate solution via minimization of a finite sequence of strictlyconvex objective functions resulting from applying the nonlinear weightedleast squares method with Carleman’s weight functions (CWFs). Anotheradvantage of the proposed algorithm is that the starting vectors for thedescent methods of minimization are directly determined from the dataeliminating the uncertainty inherent to the local methods, such as thegradient or Newton-like methods. The one-dimensional inverse model ofmagnetotelluric frequency sounding is selected to demonstrate its computationalfeasibility. Based on the localizing property of CWFs, it is proven that thedistance between the approximate and ‘exact’ solutions is small if theapproximation error is small. The computational experiments with severalrealistic and synthetic marine shallow water configurations are presented todemonstrate the computational feasibility of the proposed algorithm.

Noise Cancellation with Static Mixtures of a Nonstationary Signal and Stationary Noise

We address the problem of cancelling a stationary noise component from its static mixtures with a nonstationary signal of interest. Two different approaches, both based on second-order statistics, are considered. The first is the blind source separation (BSS) approach which aims at estimating the mixing parameters via approximate joint diagonalization of estimated correlation matrices. Proper exploitation of the nonstationary nature of the desired signal, in contrast to the stationarity of the noise, allows parameterization of the joint diagonalization problem in terms of a nonlinear weighted least squares (WLS) problem. The second approach is a denoising approach, which translates into direct estimation of just one of the mixing coefficients via solution of a linear WLS problem, followed by the use of this coefficient to create a noise-only signal to be properly eliminated from the mixture. Under certain assumptions, the BSS approach is asymptotically optimal, yet computationally more intense, since it involves an iterative nonlinear WLS solution, whereas the second approach only requires a closed-form linear WLS solution. We analyze and compare the performance of the two approaches and provide some simulation results which confirm our analysis. Comparison to other methods is also provided.