Nonlinear Least Squares Optimization Research Articles

An improved accelerated 3-term conjugate gradient algorithm with second-order Hessian approximation for nonlinear least-squares optimization

An improved accelerated 3-term conjugate gradient algorithm with second-order Hessian approximation for nonlinear least-squares optimization

An inverse problem of reconstructing the unknown coefficient in a third order time fractional pseudoparabolic equation

In this paper, we have considered the problem of reconstructing the time dependent potential term for the third order time fractional pseudoparabolic equation from an additional data at the left boundary of the space interval. This is very challenging and interesting inverse problem with many important applications in various fields of engineering, mechanics and physics. The existence of unique solution to the problem has been discussed by means of the contraction principle on a small time interval and the unique solvability theorem is proved. The stability results for the inverse problem have also been presented. However, since the governing equation is yet ill-posed (very slight errors in the additional input may cause relatively significant errors in the output potential), the regularization of the solution is needed. Therefore, to get a stable solution, a regularized objective function is to be minimized for retrieval of the unknown coefficient of the potential term. The proposed problem is discretized using the cubic B-spline (CB-spline) collocation technique and has been reshaped as a non-linear least-squares optimization of the Tikhonov regularization function. The stability analysis of the direct numerical scheme has also been presented. The MATLAB subroutine $lsqnonlin$ tool has been used to expedite the numerical computations. Both perturbed data and analytical are inverted and the numerical outcomes for two benchmark test examples are reported and discussed.

On Spectral Dai-Yuan Structured Technique for Solving Nonlinear Least-Squares Optimisation

Handling nonlinear least squares (NLS) problems as unconstrained optimisation problems has led to the development and adaptation of numerous numerical approaches. Calculating the Hessian matrix expensively for each iteration is one of the problems with the current numerical approaches for solving NLS problems. Some methods designed to rely solely on first-derivative information while ignoring second-order details tend to underperform when dealing with problems involving non-zero residuals. This paper suggests a modification to the Dai-Yuan type (DY) formula by deriving a spectral parameter for the proposed search direction. The sufficient descent results of the new formula are established using the standard assumptions under the strong Wolfe line search. Furthermore, experiments are conducted on some benchmark functions to demonstrate the computational efficiency of the proposed technique. The results highlight the effectiveness of the algorithm compared to existing methods.Keywords: Conjugate gradient, convergence theory, Hessian matrix, optimisation, Quasi-Newton, spectral, structured secant method

A Method to Evaluate Forchheimer Resistance Coefficients for Permeable Screens and Air Louvers Modelled as a Porous Medium

Porous medium models are commonly used in Computational Fluid Dynamics (CFD) to simulate flow through permeable screens of various types. However, the setup of these models is often limited to replicating a pressure drop in cases where fluid inflow is orthogonal to the screen. In this work, a porous medium formulation that employs a non-diagonal Forchheimer tensor is presented. This formulation is capable of reproducing both the pressure drop and flow deflection under varying inflow angles for complex screen geometries. A general method to determine the porous model coefficients valid for both diagonal and non-diagonal Forchheimer tensors is proposed. The coefficients are calculated using a nonlinear least-squares optimisation based on an analytical solution of a special case of the Navier–Stokes equations. The applicability of the proposed method is evaluated in four different scenarios supplemented by local CFD simulations of permeable screens: wire mesh, perforated screens, air louvers, and expanded mesh panels. The practical application of this method is demonstrated in the modelling of windbreaks and permeable double-skin facades, which typically employ the aforementioned types of porous screens.

Convergence properties of fine structure constant measurements using quasar absorption systems

ABSTRACT Searches for space–time variations of fundamental constants have entered an era of unprecedented precision. New, high-quality quasar spectra require increasingly refined analytical methods. In this paper, a continuation in a series to establish robust and unbiased methodologies, we explore how convergence criteria in non-linear least-squares optimization impact on quasar absorption system measurements of the fine structure constant α. Given previous claims for high-precision constraints, we critically examine the veracity of a so-called blinding approach, in which α is fixed at the terrestrial value during the model building process, releasing it as a free parameter only after the ‘final’ absorption system kinematic structure has been obtained. We show that this approach results in such small consecutive parameter steps during minimization that convergence is unlikely to be reached, even after as many as 1000 iterations. The fix is straightforward: α must be treated as a free parameter from the earliest possible stages of absorption system model building. The implication of the results presented here is that all previous measurements that have used initially fixed α should be reworked.

Predicting the Power Requirement of Agricultural Machinery Using ANN and Regression Models and the Optimization of Parameters Using an ANN–PSO Technique

Optimizing the design and operational parameters for tillage tools is crucial for improved performance. Recently, artificial intelligence approaches, like ANN with learning capabilities, have gained attention for cost-effective and timely problem solving. Soil-bin experiments were conducted and data were used to develop ANN and regression models using gang angle, velocity ratio, soil CI, and depth as input parameters, while tractor equivalent PTO (PTOeq) power was used as an output. Both models were trained with a randomly selected 90% of the data, reserving 10% for testing purposes. In regression, models were iteratively fitted using nonlinear least-squares optimization. The ANN model utilized a multilayer feed-forward network with a backpropagation algorithm. The comparative performance of both models was evaluated in terms of R2 and mean square error (MSE). The ANN model outperformed the regression model in the training, testing, and validation phases. A well-trained ANN model was integrated with the particle-swarm optimization (PSO) technique for optimization of the operational parameters. The optimized configuration featured a 36.6° gang angle, 0.50 MPa CI, 100 mm depth, and 3.90 velocity ratio for a predicted tractor PTOeq power of 3.36 kW against an actual value of 3.45 kW. ANN–PSO predicted the optimal parameters with a variation between the predicted and the actual tractor PTOeq power within ±6.85%.

Acquiring elastic properties of thin composite structure from vibrational testing data

Abstract The paper is devoted to a problem of acquiring elastic properties of a composite material from the vibration testing data with a simplified experimental acquisition scheme. The specimen is considered to abide by the linear elasticity laws and subject to viscoelastic damping. The boundary value problem for transverse movement of such a specimen in the frequency domain is formulated and solved with finite-element method. The correction method is suggested for the finite element matrices to account for the mass of the accelerometer. The problem of acquiring the elastic parameters is then formulated as a nonlinear least-square optimization problem. The usage of the automatic differentiation technique for stable and efficient computation of the gradient and hessian allows to use well-studied first and second order local optimization methods. We also explore the possibility of generating initial guesses for local minimization by heuristic global methods. The results of the numerical experiments on simulated data are analyzed in order to provide insights for the following real life experiments.

Batch Estimation of a Steady, Uniform, Flow-Field from Ground Velocity and Heading Measurements

This paper presents three batch estimation methods that use noisy ground velocity and heading measurements from a vehicle executing a circular orbit (or similar large heading change maneuver) to estimate the speed and direction of a steady, uniform, flow-field. The methods are based on a simple kinematic model of the vehicle's motion and use curve-fitting or nonlinear least-square optimization. A Monte Carlo simulation with randomized flow conditions is used to evaluate the batch estimation methods while varying the measurement noise of the data and the interval of unique heading traversed during the maneuver. The methods are also compared using experimental data obtained with a Bluefin-21 unmanned underwater vehicle performing a series of circular orbit maneuvers over a five hour period in a tide-driven flow.

An inverse problem of recovering the heat source coefficient in a fourth-order time-fractional pseudo-parabolic equation

In this paper, we have considered the problem of recovering the time dependent source term for the fourth order time fractional pseudoparabolic equation theoretically and numerically, for the first time, by considering an additional measurement at the left boundary of the space interval. This is very challenging and interesting inverse problem with many important applications in various fields of physics and mechanics. The existence of unique solution to the problem has been discussed by means of the contraction principle and Banach Fixed-point theorem on a small time interval and the unique solvability theorem is proved. The stability results for the inverse problem have also been presented. However, since the governing equation is yet ill-posed (very slight errors in the additional input may cause relatively significant errors in the output force), the regularization of the solution is needed. Therefore, to get a stable solution, a regularized objective function is to be minimized for retrieval of the unknown coefficient of the forcing term. The proposed problem is discretized using the quintic B-spline (QnB-spline) collocation technique and has been reshaped as a non-linear least-squares optimization of the Tikhonov regularization function. The stability analysis of the direct numerical scheme has also been presented. The MATLAB subroutine lsqnonlin tool has been used to expedite the numerical computations. Both analytical and perturbed data are inverted and the numerical outcomes for two benchmark test examples are reported and discussed.

Inverse problem of determining the coefficient in a sixth order Boussinesq equation with additional nonlocal integral condition

The work is devoted to the study of the solvability of the inverse boundary value problem with an unknown time depended coefficient for a sixth-order Boussinesq equation with an additional integral condition of the first kind is investigated. First, the given problem is reduced to an equivalent problem in a certain sense. Then, using the Fourier method the equivalent problem is reduced to solving the system of integral equations. The existence and uniqueness of a solution to the system of integral equation is proved by the contraction mapping principle. This solution is also the unique solution to the equivalent problem. Finally, by equivalence, the theorem of existence and uniqueness of a classical solution to the given problem is proved. The sixth order Boussinesq-Type problem is discretized using the Quintic B-spline (QB-spline) collocation technique and reshaped as non-linear least-squares optimization of the Tikhonov regularization function. This is numerically solved by means of the MATLAB subroutine lsqnonlin tool. Both perturbed data and analytical solution are inverted. Numerical outcomes are reported and discussed. In addition, the von Neumann stability analysis for the proposed numerical approach has also been discussed.

An inverse problem of identifying the time-dependent potential and source terms in a two-dimensional parabolic equation

In this article, simultaneous identification of the time-dependent lowest and source terms in a two-dimensional (2D) parabolic equation from knowledge of additional measurements is studied. Existence and uniqueness of the solution is proved by means of the contraction mapping on a small time interval. Since the governing equation is yet ill-posed (very slight errors in the time-average temperature input may cause relatively significant errors in the output potential and source terms), we need to regularize the solution. Therefore, regularization is needed for the retrieval of unknown terms. The 2D problem is discretized using the alternating direction explicit (ADE) method and reshaped as non-linear least-squares optimization of the Tikhonov regularization function. This is numerically solved by means of the MATLAB subroutine $lsqnonlin$ tool. Finally, we present a numerical example to demonstrate the accuracy and efficiency of the proposed method. Our numerical results show that the ADE is an efficient and unconditionally stable scheme for reconstructing the potential and source coefficients from minimal data which makes the solution of the inverse problem (IP) unique.

Advances in Wall Interference Corrections for Two-Dimensional Lifting Bodies

A novel approach to correcting the effects of wall interference has been developed for low-speed closed-wall wind tunnels using a nonlinear least-squares optimization process independent of the model geometry. Potential flow singularities are used to represent the test object, and the method of images is used to represent the tunnel walls. Trust-region reflective optimization enables the locations and strengths of singularities to be determined iteratively with only wall pressure measurements as independent inputs. Results are obtained from a range of two-dimensional and three-dimensional airfoil test cases using both computational fluid dynamics simulations and wind tunnel experiments, including a NACA 0012 with deployed spoiler, a NACA 0015, a NACA 4415, and a McDonnell Douglas 30P30N high-lift configuration. The nonlinear least-squares wall-correction method is shown to demonstrate a considerable improvement over the conventional two-variable technique.

An analytical approach to evaluate the impact of age demographics in a pandemic.

The time required to identify and confirm risk factors for new diseases and to design an appropriate treatment strategy is one of the most significant obstacles medical professionals face. Traditionally, this approach entails several clinical studies that may last several years, during which time strict preventative measures must be in place to contain the epidemic and limit the number of fatalities. Analytical tools may be used to direct and accelerate this process. This study introduces a six-state compartmental model to explain and assess the impact of age demographics by designing a dynamic, explainable analytics model of the SARS-CoV-2 coronavirus. An age-stratified mathematical model taking the form of a deterministic system of ordinary differential equations divides the population into different age groups to better understand and assess the impact of age on mortality. It also provides a more accurate and effective interpretation of the disease evolution, specifically in terms of the cumulative numbers of infected cases and deaths. The proposed Kermack-Mckendrick model is incorporated into a non-linear least-squares optimization curve-fitting problem whose optimized parameters are numerically obtained using the Levenberg-Marquard algorithm. The curve-fitting model's efficiency is proved by testing the age-stratified model's performance on three U.S. states: Connecticut, North Dakota, and South Dakota. Our results confirm that splitting the population into different age groups leads to better fitting and forecasting results overall as compared to those achieved by the traditional method, i.e., without age groups. By using comprehensive models that account for age, gender, and ethnicity, regional public health authorities may be able to avoid future epidemics from inflicting more fatalities and establish a public health policy that reduces the burden on the elderly population.

Initial-Pose Self-Calibration for Redundant Cable-Driven Parallel Robot Using Force Sensors Under Hybrid Joint-Space Control

This letter investigates initial-pose estimation (Cartesian position and orientation) for redundant cable-driven parallel robots (CDPRs). As the forward kinematics cannot be performed if the robot is equipped with incremental sensors, a calibration method using force sensors is proposed. The self-calibration problem is formulated as a non-linear least-squares optimization problem based on the hybrid joint-space control strategy. The redundant cables are tension-controlled; the other cables are maintained at a fixed length in the joint space. A position-determined cable adjustable force (P-CAF) performance index and the concept of calibratable space were proposed to maintain the quasi-static conditions. A simulated case study was used to validate the calibration process.

Forward kinematics of a cable-driven parallel robot with pose estimation error covariance bounds

This paper presents four forward kinematics algorithms for an over-constrained six degree-of-freedom (DOF) cable-driven parallel robot (CDPR) that in addition to computing the end-effector’s pose, also provide covariance bounds on the pose estimation error. The first two proposed methods are based on cable-length and cable-length-squared loop-closure equations and the use of unconstrained attitude parameterizations to describe the orientation of the CDPR end-effector. The second pair of methods involve constrained attitude parameterizations and are also based on cable-length and cable-length-squared loop-closure equations. Nonlinear least-squares optimization is used in each of these methods to iteratively compute the forward kinematics solution and determine covariance bounds on the pose estimation error. Attitude identities are used to obtain analytic expressions for the computations whenever possible. The forward kinematics algorithms are validated through Monte-Carlo simulations, where Euler-angle-sequence, quaternion, and DCM parameterizations of the end-effector attitude are implemented and the accuracy of the covariance bounds is demonstrated. It is also shown that the method based on the cable-length-squared loop-closure equations yields improved convergence properties compared to the cable-length loop-closure equations.

Time-series polarimetric bistatic scattering decomposition using comprehensive modified first-order radiative transfer model at C-band for vegetative terrain and validation

ABSTRACT The sensitivity of the bistatic scattered signal to both the soil and vegetation physical properties in microwave sensing of vegetation is subject to uncertainties. A multi-angular and fully polarimetric data acquisition from a bistatic system increases the number of observations. Thus, optimum bistatic system parameters for vegetation monitoring are necessary to develop an understating of microwave interactions with the surface and vegetation properties. In this study, C-band fully polarimetric bistatic scatterometer (BiSCAT) system was designed to measure the scattering response of vegetated terrain in the forward specular plane. The correlation analysis between the measured bistatic scattering coefficient () and in-situ soil/vegetation properties, such as plant water content (PWC) and soil moisture (), was used to find the optimum specular incidence angle of the BiSCAT system. The optimum parameters of the BiSCAT system were used as input to the modified first-order radiative transfer model (MRTM) for the simulation for vegetation. Kirchhoff’s approximate (KA) model for soil surface in forward specular plane was simplified for soil contribution within MRTM. Additionally, the temporal patterns of are modelled by employing the empirical formulation between the vegetation optical depth and PWC. The MRTM offers an understanding of co-polarized electromagnetic signal interaction with the temporal change in vegetation constituents and soil surface parameters. The model is limited to providing insights into co-polarized radar return from the target due to the incapability of yielding the cross-polarization factor from the KA model in the forward specular direction. The contributions of surface, vegetation, surface-vegetation, vegetation-surface and surface-vegetation-surface were quantified to better understand microwave’s interaction with the vegetation. The model is calibrated using a constrained non-linear least-square optimization algorithm. The performance indices of simulating yields good agreement with the BiSCAT measurements.

Observability-Aware Intrinsic and Extrinsic Calibration of LiDAR-IMU Systems

Accurate and reliable sensor calibration is essential to fuse LiDAR and inertial measurements, which are usually available in robotic applications. In this article, we propose a novel LiDAR-IMU calibration method within the continuous-time batch-optimization framework, where the intrinsics of both sensors and the spatial-temporal extrinsics between sensors are calibrated without using calibration infrastructure, such as fiducial tags. Compared to discrete-time approaches, the continuous-time formulation has natural advantages for fusing high-rate measurements from LiDAR and IMU sensors. To improve efficiency and address degenerate motions, the following two observability-aware modules are leveraged: first, The information-theoretic data selection policy selects <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">only</i> the most informative segments for calibration during data collection, which significantly improves the calibration efficiency by processing only the selected informative segments. Second, the observability-aware state update mechanism in nonlinear least-squares optimization updates <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">only</i> the identifiable directions in the state space with truncated singular value decomposition, which enables accurate calibration results even under degenerate cases where informative data segments are not available. The proposed LiDAR-IMU calibration approach has been validated extensively in both simulated and real-world experiments with different robot platforms, demonstrating its high accuracy and repeatability in commonly-seen human-made environments.

Symbolic regression for the interpretation of quantitative structure-property relationships

The interpretation of quantitative structure–activity or structure–property relationships is important in the field of chemoinformatics. Although multivariate linear regression models are typically interpretable, they do not generally have high predictive abilities. Symbolic regression (SR) combined with genetic programming (GP) is a well-established technique for generating the mathematical expressions that describe the relationships within a dataset. However, SR sometimes produces complicated expressions that are hard for humans to interpret. This paper proposes a method for generating simpler expressions by incorporating three filters into GP-based SR. The filters are further combined with nonlinear least-squares optimization to give filter-introduced GP (FIGP), which improves the predictive ability of SR models while retaining simple expressions. As a proof-of-concept, the quantitative estimate of drug-likeness and the synthetic accessibility score are predicted based on the chemical structures of compounds. Overall, FIGP generates less-complicated expressions than previous SR methods. In terms of predictive ability, FIGP is better than GP, but is outperformed by a support vector machine with a radial basis function kernel. Furthermore, quantitative structure–activity relationship models are constructed for three matching molecular series with biological targets. In the case of one target, the activity prediction models given by FIGP exhibit better predictive ability than multivariate linear regression and support vector regression with the radial basis function kernel, whereas for the remaining cases, FIGP is slightly less accurate than multivariate linear regression.

Parameter estimation of linear fractional-order system from laplace domain data

A novel parameter estimation method based on the Laplace transform and the response sensitivity method has been presented to recognize the parameters of linear fractional-order systems (FOS) rapidly. The proposed method consumes two orders of magnitude computational resources lower than the traditional time-domain method. Fractional-order operators are increasingly widely used in control and synchronization, epidemiology, viscoelastic material modelling and other emerging disciplines. It is difficult to measure the fractional-order α and system parameters directly in real engineering applications. This paper’s main work include: Firstly, a general linear fractional differential equation is transformed into an algebraic equation by the Laplace transform, and the parameter sensitivity analysis concerning the unknown parameters is also deduced. Then, the parameter estimation problem of the linear FOS is established as a nonlinear least-squares optimization in the Laplace domain, and the enhanced response sensitivity method is adopted to resolve this nonlinear minimum optimization equation iteratively. In addition, the Tikhonov regularization is employed to cope with the potential ill-posed situations, and the trust-region restriction is also introduced to improve the convergence. Finally, taking a differential system with two types of fractional-order operators, a multi-degree-of-freedom FOS with external excitation and an actual piezoelectric actuator model as examples, the specific implementation process is demonstrated in detail to test the robustness and validity of the proposed approach.

Investigating the effect of temporal, geographic, and management factors on US Holstein lactation curve parameters.

We fit the Wood's lactation model to an extensive database of test-day milk production records of US Holstein cows to obtain lactation-specific parameter estimates and investigated the effects of temporal, spatial, and management factors on lactation curve parameters and 305-d milk yield. Our approach included 2 steps as follows: (1) individual animal-parity parameter estimation with nonlinear least-squares optimization of the Wood's lactation curve parameters, and (2) mixed-effects model analysis of 8,595,413 sets of parameter estimates from individual lactation curves. Further, we conducted an analysis that included all parities and a separate analysis for first lactation heifers. Results showed that parity had the most significant effect on the scale (parameter a), the rate of decay (parameter c), and the 305-d milk yield. The month of calving had the largest effect on the rate of increase (parameter b) for models fit with data from all lactations. The calving month had the most significant effect on all lactation curve parameters for first lactation models. However, age at first calving, year, and milking frequency accounted for a higher proportion of the variance than month for first lactation 305-d milk yield. All parameter estimates and 305-d milk yield increased as parity increased; parameter a and 305-d milk yield rose, and parameters b and c decreased as year and milking frequency increased. Calving month estimates parameters a, b, c, and 305-d milk yield were the lowest values for September, May, June, and July, respectively. The results also indicated the random effects of herd and cow improved model fit. Lactation curve parameter estimates from the mixed-model analysis of individual lactation curve fits describe well US Holstein lactation curves according to temporal, spatial, and management factors.