Nonlinear Least-squares Research Articles

Effects of reducing redundant parameters in parameter optimization for symbolic regression using genetic programming

Gradient-based local optimization has been shown to improve results of genetic programming (GP) for symbolic regression (SR) – a machine learning method for symbolic equation learning. Correspondingly, several state-of-the-art GP implementations use iterative nonlinear least squares (NLS) algorithms for local optimization of parameters. An issue that has however mostly been ignored in SR and GP literature is overparameterization of SR expressions, and as a consequence, bad conditioning of NLS optimization problem. The aim of this study is to analyze the effects of overparameterization on the NLS results and convergence speed, whereby we use Operon as an example GP/SR implementation. First, we demonstrate that numeric rank approximation can be used to detect overparameterization using a set of six selected benchmark problems. In the second part, we analyze whether the NLS results or convergence speed can be improved by simplifying expressions to remove redundant parameters with equality saturation. This analysis is done with the much larger Feynman symbolic regression benchmark set after collecting all expressions visited by GP, as the simplification procedure is not fast enough to use it within GP fitness evaluation. We observe that Operon frequently visits overparameterized solutions but the number of redundant parameters is small on average. We analyzed the Pareto-optimal expressions of the first and last generation of GP, and found that for 70% to 80% of the simplified expressions, the success rate of reaching the optimum was better or equal than for the overparameterized form. The effect was smaller for the number of NLS iterations until convergence, where we found fewer or equal iterations for 51% to 63% of the expressions after simplification.

Synergistic and Antagonistic Effects of Mixed-Leaf Litter Decomposition on Nutrient Cycling.

Understanding decomposition patterns of mixed-leaf litter from agroforestry species is crucial, as leaf litter in ecosystems naturally occurs as mixtures rather than as separate individual species. We hypothesized that litter mixtures with larger trait divergence would lead to faster mass loss and more balanced nutrient release compared to single-species litter. Specifically, we expected mixtures containing nutrient-rich species to exhibit synergistic effects, resulting in faster decay rates and sustained nutrient release, while mixtures with nutrient-poor species would demonstrate antagonistic effects, slowing decomposition. We conducted a mesocosm experiment using a custom wooden setup filled with soil, and the litterbag method was used to test various leaf litter mixtures. The study involved leaf litter from six agroforestry tree species: three species from humid highland regions and three from semi-arid regions. Treatments included three single-species leaf litter mixtures, three two-species mixtures, and one three-species mixture, based on the sampling region. Species included Calliandra calothyrsus (Ca), Croton megalocarpus (Cr), Grevillea robusta (G), Alnus acuminata (A), Markhamia lutea (M), and Eucalyptus globulus (E). Decay rate constants (k) were estimated using non-linear least-squares regression and observed mass loss was compared to predicted values for mixed-species litter treatments to assess synergistic and antagonistic effects. A two-way linear mixed-effects model was employed to explain variation in mass loss. Results indicate positive non-additive effects for leaf litter mixtures including nutrient-rich species and negative non-additive effects for mixtures including nutrient-poor species. The mixture of Ca + Cr + G had positive non-additive or synergistic effects as it decomposed faster than its corresponding single-species litter. Leaf litters with higher lignin content, such as A + M + E and Ca + Cr + G, exhibited less lignin release compared to what would be expected based on individual litter types, demonstrating antagonistic effects. These findings highlight that both litter nutrient constituents and litter diversity play an important role in decomposition processes and therefore in the restoration of the degraded and nutrient-depleted soils of Rwanda.

A Comparison of Battery Equivalent Circuit Model Parameter Extraction Approaches Based on Electrochemical Impedance Spectroscopy

This paper presents three approaches to estimating the battery parameters of the electrical equivalent circuit model (ECM) based on electrochemical impedance spectroscopy (EIS); these approaches are referred to as (a) least squares (LS), (b) exhaustive search (ES), and (c) nonlinear least squares (NLS). The ES approach is assisted by the LS method for the rough determination of the lower and upper bound of the ECM parameters, and the NLS approach is incorporated with the Monte Carlo run such that different initial guesses can be assigned to improve the goodness of EIS fitting. The proposed approaches are validated using both simulated and real EIS data. Compared to the LS approach, the ES and NLS approaches show better fitting accuracy at various noise levels, whereas in both the validation using simulated EIS data and actual EIS data collected from LG 18650 and Molicel 21700 batteries, the NLS approach shows better fitting accuracy than that of LS and ES approaches. In all cases, compared with the ES approach, the computational time of the NLS approach is significantly faster, and compared with the LS approach, the NLS approach shows a minimal difference in computational time and considerably better fitting performance.

An ADCP Attitude Dynamic Errors Correction Method Based on Angular Velocity Tensor and Radius Vector Estimation

An acoustic Doppler current profiler (ADCP) installed on a platform produces rotational tangential velocity as a result of variations in the platform’s attitude, with both the tangential velocity and radial orientation varying between each pulse’s transmission and reception by the transducer. These factors introduce errors into the measurements of vessel velocity and flow velocity. In this study, we address the errors induced by dynamic factors related to variations in attitude and propose an ADCP attitude dynamic error correction method based on angular velocity tensor and radius vector estimation. This method utilizes a low-sampling-rate inclinometer and compass data and estimates the angular velocity tensor based on a physical model of vessel motion combined with nonlinear least-squares estimation. The angular velocity tensor is then used to estimate the transducers’ radius vectors. Finally, the radius vectors are employed to correct the instantaneous tangential velocity within the measured velocities of the vessel and flow. To verify the effectiveness of the proposed method, field tests were conducted in a water pool. The results demonstrate that the proposed method surpasses the attitude static correction approach. In comparison with the ASC method, the average relative error in vessel velocity during free-swaying movement decreased by 20.94%, while the relative standard deviation of the error was reduced by 17.38%.

Generalized, Three-Dimensional Wind Tunnel Wall Correction Using Nonlinear Least-Squares Optimization

A novel three-dimensional technique for correcting wind tunnel measurements for wall interference has been developed. The flow around the test body and its wake is approximated as a series of inviscid singularities, while the wind tunnel walls and model support are approximated using a panel method. A nonlinear least-squares approach with trust-region reflective optimization is then used to obtain the strength and locations of singularities together with the domain boundary panel strengths by fitting to measured wall pressures. The technique is demonstrated using computational fluid dynamics for the test case of a well-documented slender delta wing body with high blockage and validated against both experiments and legacy data. The nonlinear least-squares wall correction method is shown to demonstrate a significant improvement in wall corrections compared to the classical two-variable technique.

A Self-Calibration Method for Robot End-Effector Using Spherical Constraints

A self-calibration method utilizing spherical constraints is proposed for calibration of robot end-effectors. The method establishes a mathematical model to account for both the geometric errors of the robot and the deformation errors of the end-effector. A nonlinear least-squares parameter identification technique based on spherical constraints is employed to achieve autonomous calibration of the end-effector. Contrasted with methodologies relying on point plane or distance constraints, this novel technique delivers superior positioning accuracy, streamlined operational procedures and enhanced efficiency. Both simulation and experimental validation confirm that the self-calibration method using spherical constraints improves the positioning accuracy of the robot end-effector from 3 mm to 0.3 mm, showing the effectiveness of the method.

Expected constraints on Phobos interior from the MMX gravity and rotation observations

The origin of the Martian moons is still uncertain, and knowledge about their interior could provide support to some of its leading theories. In preparation for the JAXA Martian Moons eXploration (MMX) mission, we review our current knowledge on the interior of Phobos, and provide synthetic tests showing how the gravity and rotation determination could allow the detection of specific interior-structure properties. The inversion of the geodetic observables for the retrieval of the internal mass distribution of a set of synthetic interior models is performed via non-linear least-squares, where the interior parameterization is based on the level-set method. We additionally provide simple expressions allowing to relate some of these interior models to the geodetic observables of Phobos. The results, based on realistic measurement resolution and noise scenarios, show good retrievals for most of the models at the data resolutions expected from MMX. Specifically, we find the gravity information is realistically sufficient for the detection of mass anomalies below the Stickney crater, as well as large scale heterogeneous regions within plausible rubble-pile structures. Libration helps retrieve the more degenerate models for gravity, such as those with concentric layers or with density varying linearly with depth. The incremental improvement from further adding a hypothetical mean obliquity measurement is marginal. Finally, we apply the level-set inversion and the analytical formulas to estimate possible interior characteristics of the ‘real’ Phobos from the currently-available scarce geodetic observables. The level-set solutions for the real-data inversion generally converge to a higher mass concentration towards the surface in the equatorial region. Markov chain Monte Carlo estimations of parameters relative to a simple 2-layer model or a radial density distribution similarly hint at a lighter region inside of Phobos.

A nonlinear least-squares convexity enforcing 𝐶⁰ interior penalty method for the Monge–Ampère equation on strictly convex smooth planar domains

We construct a nonlinear least-squares finite element method for computing the smooth convex solutions of the Dirichlet boundary value problem of the Monge-Ampère equation on strictly convex smooth domains in R 2 {\mathbb {R}}^2 . It is based on an isoparametric C 0 C^0 finite element space with exotic degrees of freedom that can enforce the convexity of the approximate solutions. A priori and a posteriori error estimates together with corroborating numerical results are presented.

Harmonic Components Isolation in Vehicle Vibrations: Enhancing Quarter-Car Model Analysis with an Extended Kalman Filter Approach

<div>This article addresses the essential task of understanding vibrations produced by vehicles to enhance the design of authentic laboratory tests. The article focuses on two primary sources of vibrations: those arising from vehicle–road surface interaction, which is largely random, and those emanating from the drivetrain, characterized as a summation of harmonics with a time-varying fundamental frequency. The method involves the application of the extended Kalman filter (EKF) paired with robust nonlinear least-squares (NLS) initialization to isolate the harmonic components effectively. Through a comprehensive analysis involving mean-square-error (MSE) evaluation via Monte Carlo simulation, considering additive white Gaussian noise (AWGN) and a two-degrees-of-freedom quarter-car model’s simulation response to the road, the research demonstrates the EKF’s proficiency. The results indicate the EKF’s capability to accommodate AWGN with a signal-to-noise ratio (SNR) up to 0 dB and road-induced random background vibrations up to an SNR of −3 dB, maintaining an MSE order of approximately 10<sup>−3</sup>.</div>

NON-HOMOGENEOUS POISSON PROCESS OF A QUADRATIC TRANSMUTED LOG-LOGISTIC MODEL

In recent years, software reliability engineering has become a major topic due to rapid technological advancements, as researchers work to interpret and analyze software reliability data. Therefore, establishing new software reliability models is essential to possess reliable tools for diagnosing software systems. This paper proposes a new NHPP model based on the quadratic transmuted log-logistic (QTLL) distribution. The characteristics of the model were obtained and graphically illustrated. Three real software data sets were applied to evaluate the proposed model based on different criteria. The parameters of the NHPP QTLL were estimated using maximum likelihood estimation (MLE), and nonlinear least squares estimation (NLSE), and a comparative analysis of the two estimation methods has been performed to estimate the model parameters. Additionally, a study was carried out to compare the suggested model with other well-known NHPP models. The results indicate that NLSE outperforms MLE in parameter estimation. Furthermore, the NHPP QTLL model demonstrated superior performance across all evaluation assessments compared to other models. The graphical representation also shows that the proposed model is more effective in fitting fault data than existing models.

Revisiting the fitting of the Nelson–Siegel and Svensson models

The Nelson–Siegel and the Svensson models are two widely used models for the term structure of interest rates. These models are quite simple and intuitive, but fitting them to market data is numerically challenging and various difficulties have been reported. In this paper, we provide a novel mathematical analysis of the fitting problem based on parametric optimization. We formulate the fitting problem as a separable nonlinear least-squares problem, in which the linear parameters can be eliminated. We provide a thorough discussion on the conditioning of the inner part of the reformulated problem and show that many of the reported difficulties encountered when solving it are inherent to the problem formulation itself and cannot be tackled by choosing a particular optimization algorithm. Our stability analysis provides novel insights that we use to show that some of the ill-conditioning can be avoided, and that a suitably chosen penalty approach can be used to address the remaining ill-conditioning. Numerical results indicate that this approach has the expected impact while being independent of any choice of a particular optimization algorithm. We further establish smoothness properties of the reduced objective function, putting global optimization methods for the reduced problem on a sound mathematical basis.

Mirtazapine blood levels and antidepressant response

Objective Therapeutic drug monitoring (TDM) is an important tool for treatment optimisation. Its usefulness has recently been demonstrated for some first-line antidepressants; however, few studies have been reported on the relationship between blood levels of mirtazapine and its antidepressant effects. The aim of this study was to investigate the association between blood concentration of mirtazapine and antidepressant response. Methods 59 outpatients treated with mirtazapine for depression were recruited and followed up for three months in a naturalistic setting. Hamilton Depression Rating Scale-21 (HAMD-21) was administered at baseline, month 1, and month 3 to assess antidepressant response. Mirtazapine serum concentration was measured at steady state. Linear regression analysis and nonlinear least-squares regression were used to estimate association between serum concentration of mirtazapine and antidepressant response. Results Our results showed no overall association between serum concentration of mirtazapine and symptom improvement at month 1 and month 3. A marginally significantly higher serum concentration of mirtazapine was found in responders vs non-responders at month 3. Conclusions The study suggests that serum concentration of mirtazapine is not strongly associated with the antidepressant efficacy of mirtazapine. This is probably attributed to its pharmacodynamic profile, even though higher blood levels seem to be marginally more effective.

A Combined Sensor Design Applied to Large-Scale Measurement Systems.

The photoelectric sensing unit in a large-space measurement system primarily determines the measurement accuracy of the system. Aiming to resolve the problem whereby existing sensing units have difficulty accurately measuring the hidden points and free-form surfaces in large components, in this study, we designed a multi-node fusion of a combined sensor. Firstly, a multi-node fusion hidden-point measurement model and a solution model are established, and the measurement results converge after the number of nodes is simulated to be nine. Secondly, an adaptive front-end photoelectric conditioning circuit, including signal amplification, filtering, and adjustable level is designed, and the accuracy of the circuit function is verified. Then, a nonlinear least-squares calibration method is proposed by combining the constraints of the multi-position vector cones, and the internal parameters of the probe, in relation to the various detection nodes, are calibrated. Finally, a distributed system and laser tracking system are introduced to establish a fusion experimental validation platform, and the results show that the standard deviation and accuracy of the three-axis measurement of the test point of the combined sensor in the measurement area of 7000 mm × 7000 mm × 3000 mm are better than 0.026 mm and 0.24 mm, respectively, and the accuracy of the length measurement is within 0.28 mm. Further, the measurement accuracy of the hidden point of the aircraft hood and the free-form surface is better than 0.26 mm, which can meet most of the industrial measurement needs and expand the application field of large-space measurement systems.

New quantum assignments and analysis of high-resolution H212CO spectra in the range 3700–4450 cm-1

Four spectra of formaldehyde in natural isotopic abundance in the 3700–5200 cm-1 region were recorded at low temperature 160–166 K at Synchrotron SOLEIL for various pressures. Line positions and intensities were retrieved by non-linear least-squares curve-fitting procedures in the range 3700–4450 cm-1 and analyzed using ab initio based effective Hamiltonian and line intensities computed using new ab initio dipole moment surface. A new measured line list contains positions and intensities for 6177 features. Refined parameters of effective Hamiltonian were fitted to all assigned line positions with the RMS deviations of 0.001 cm-1. Updated line lists include intensity values based on ab initio variational calculations which were subsequently empirically optimized. Comparison of our theoretical simulation with previously available data as well as with high-resolution and low-resolution experimental spectra are reported.

One-Port Coaxial Line Sample Holder Characterisation Method of Dielectric Spectra.

A technique for solving the one-port closed coaxial transmission line sample holder scattering equation for complex permittivity inversion for lossy materials is presented. A non-linear least-squares procedure is used for the determination of parameters for the specification of the spectral functional form of the complex permittivity. The method allows for accurate retrieval of many low- and high-permittivity dielectric materials in the frequency range of 1 GHz to 3 GHz inserted into the coaxial cell. Using this method, the complex permittivity of a number of liquids and a Maltese soil known as Bajjad soil have been extracted by measurements using a short terminated coaxial transmission line sample holder. The proposed novel inversion method is mainly based on the reflection coefficient of the test material. The measured results of the complex permittivity of liquid dielectrics such as ethanol, methanol, and TX100 are validated and compared with previously published data obtained from measurements made by the National Physical Laboratory (NPL) using a two-port measurement setup made with the same commercial coaxial transmission line sample holder used in the one-port setup. Since the technique allows broadband measurements, it has been used to characterise the soil dielectric spectrum in the frequency range of 1-3 GHz, which is also compared with results from a two-port setup of the same coaxial line. The experimental results are a validation of the proposed approach for different types of materials.

A Deep Learning-Based Framework for Highly Accelerated Prostate MR Dispersion Imaging.

Dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) measures microvascular perfusion by capturing the temporal changes of an MRI contrast agent in a target tissue, and it provides valuable information for the diagnosis and prognosis of a wide range of tumors. Quantitative DCE-MRI analysis commonly relies on the nonlinear least square (NLLS) fitting of a pharmacokinetic (PK) model to concentration curves. However, the voxel-wise application of such nonlinear curve fitting is highly time-consuming. The arterial input function (AIF) needs to be utilized in quantitative DCE-MRI analysis. and in practice, a population-based arterial AIF is often used in PK modeling. The contribution of intravascular dispersion to the measured signal enhancement is assumed to be negligible. The MR dispersion imaging (MRDI) model was recently proposed to account for intravascular dispersion, enabling more accurate PK modeling. However, the complexity of the MRDI hinders its practical usability and makes quantitative PK modeling even more time-consuming. In this paper, we propose fast MR dispersion imaging (fMRDI) to effectively represent the intravascular dispersion and highly accelerated PK parameter estimation. We also propose a deep learning-based, two-stage framework to accelerate PK parameter estimation. We used a deep neural network (NN) to estimate PK parameters directly from enhancement curves. The estimation from NN was further refined using several steps of NLLS, which is significantly faster than performing NLLS from random initializations. A data synthesis module is proposed to generate synthetic training data for the NN. Two data-processing modules were introduced to improve the model's stability against noise and variations. Experiments on our in-house clinical prostate MRI dataset demonstrated that our method significantly reduces the processing time, produces a better distinction between normal and clinically significant prostate cancer (csPCa) lesions, and is more robust against noise than conventional DCE-MRI analysis methods.

Convergent least-squares optimization methods for variational data assimilation

Data assimilation combines prior (or background) information with observations to estimate the initial state of a dynamical system over a given time-window. A common application is in numerical weather prediction where a previous forecast and atmospheric observations are used to obtain the initial conditions for a numerical weather forecast. In four-dimensional variational data assimilation (4D-Var), the problem is formulated as a nonlinear least-squares problem, usually solved using a variant of the classical Gauss-Newton (GN) method. However, we show that GN may not converge if poorly initialized. In particular, we show that this may occur when there is greater uncertainty in the background information compared to the observations, or when a long time-window is used in 4D-Var allowing more observations. The difficulties GN encounters may lead to inaccurate initial state conditions for subsequent forecasts. To overcome this, we apply two convergent GN variants (line search and regularization) to the long time-window 4D-Var problem and investigate the cases where they locate a more accurate estimate compared to GN within a given budget of computational time and cost. We show that these methods are able to improve the estimate of the initial state, which may lead to a more accurate forecast. Highlights Poor initialization of Gauss-Newton method may result in failure to converge. Safeguarded Gauss-Newton improves initial state estimate within limited time/cost. Results using twin experiments with long time-window and chaotic Lorenz models. Apply state of the art least-squares convergence theory to data assimilation. Improvements to initial state estimate may lead to a more accurate forecast.

Parameter-input estimation of RC thermal models of buildings using unscented Kalman filter and nonlinear least square method

Effective building energy management (e.g. temperature control strategies) necessitates reliable and computationally efficient building thermal models. One type of them is the resistor–capacitor (RC) model. However, estimating model parameters and inputs (e.g. solar heat gain) simultaneously is challenging, especially when some of the temperature states are missing due to instrumentation limitations and/or sensor malfunctions. The present study utilizes unscented Kalman filter (UKF) and nonlinear least squares (NLSs) methods for parameters and input estimation of RC models with possible unavailable temperature states. The estimation procedure, mathematical operations and result analysis are presented in detail. To evaluate the capability of the method, two case studies were conducted. The first case study involved a simple, made-up RC model with known parameters, inputs and states, while the second case study used monitored data from a single detached house. The capability of the method was evaluated by comparing the estimated parameters, inputs and states to the corresponding true values in both study cases. The performance evaluation shows that the proposed method can effectively estimate RC model parameters and inputs, even with certain missing states. The proposed method can be employed for timely online updating of RC model parameters to improve response prediction.

DCE-Qnet: deep network quantification of dynamic contrast enhanced (DCE) MRI.

Quantification of dynamic contrast-enhanced (DCE)-MRI has the potential to provide valuable clinical information, but robust pharmacokinetic modeling remains a challenge for clinical adoption. A 7-layer neural network called DCE-Qnet was trained on simulated DCE-MRI signals derived from the Extended Tofts model with the Parker arterial input function. Network training incorporated B1 inhomogeneities to estimate perfusion (Ktrans, vp, ve), tissue T1 relaxation, proton density and bolus arrival time (BAT). The accuracy was tested in a digital phantom in comparison to a conventional nonlinear least-squares fitting (NLSQ). In vivo testing was conducted in ten healthy subjects. Regions of interest in the cervix and uterine myometrium were used to calculate the inter-subject variability. The clinical utility was demonstrated on a cervical cancer patient. Test-retest experiments were used to assess reproducibility of the parameter maps in the tumor. The DCE-Qnet reconstruction outperformed NLSQ in the phantom. The coefficient of variation (CV) in the healthy cervix varied between 5 and 51% depending on the parameter. Parameter values in the tumor agreed with previous studies despite differences in methodology. The CV in the tumor varied between 1 and 47%. The proposed approach provides comprehensive DCE-MRI quantification from a single acquisition. DCE-Qnet eliminates the need for separate T1 scan or BAT processing, leading to a reduction of 10 min per scan and more accurate quantification.

An improved accelerated 3-term conjugate gradient algorithm with second-order Hessian approximation for nonlinear least-squares optimization

An improved accelerated 3-term conjugate gradient algorithm with second-order Hessian approximation for nonlinear least-squares optimization