Least-squares Parameter Estimation Research Articles

Control for Leader–Follower Consensus of Multi-agent Systems with Actuator Faults Using Decentralized Robust Fault-tolerant Control

The leader–follower consensus tracking is one of the important problems in multi-agent systems (MASs) which can be influenced by actuator faults. The main issue of the paper is fault occurrence, and the challenge is to design an appropriate decentralized fault-tolerant control (FTC) based on fault estimations to minimize the effects of actuator fault and maintain the tracking problem. In this paper, the Luenberger detection filter is adopted for the fault detection process and fault estimations are obtained by recursive least-square (RLS) parameter estimation method. The procedure utilized for FTC is fault hiding technique so that using virtual actuator methodology, the effects of faults are partially improved and a robust controller is combined with a virtual actuator to reduce the tracking error. The main contribution of this paper is designing a robust FTC which not only can compensate for the performance degradation, but also can make the system robust against fault estimation error uncertainties. It is also proved that the tracking error of faulty agents converges to zero asymptotically and the consensus tracking is preserved by using the proposed method. The effectiveness of the proposed method is verified through simulations.

Unbiased Least-Squares Modelling

In this paper we analyze the bias in a general linear least-squares parameter estimation problem, when it is caused by deterministic variables that have not been included in the model. We propose a method to substantially reduce this bias, under the hypothesis that some a-priori information on the magnitude of the modelled and unmodelled components of the model is known. We call this method Unbiased Least-Squares (ULS) parameter estimation and present here its essential properties and some numerical results on an applied example.

A theoretical and experimental exploration of the seismic dynamics of multi-span bridges

A generalized reduced-order model of a multi-span continuous bridge, on flexible discrete supports, that is subjected to multi-support seismic excitation is presented. This model highlights the key non-dimensional system parameters. Real spatiotemporal ground motion time-series (from the SMART-1 array, Taiwan) are used, as an alternative to employing artificial ground motion based on some spatial incoherence kernels. Benchmark experimental test data, using the multiple support excitation rig of a four-span bridge and SMART-1 array excitation, is used to validate/calibrate the proposed reduced-order model. An operational modal analysis is conducted to obtain least-square estimates of these key dynamic parameters using a Levenberg–Marquardt algorithm. The computationally efficient reduced-order model is then employed for a parametric study that explores the effect of spatial incoherence, bridge alignment and archetypal symmetrical and asymmetrical bridge geometries. A comparison of identical and multi-support excitation cases indicate the likely range of beneficial/adverse errors in neglecting the spatiotemporal nature of ground motions in design analyses.

Estimation and model selection in general spatial dynamic panel data models

Commonly used methods for estimating parameters of a spatial dynamic panel data model include the two-stage least squares, quasi-maximum likelihood, and generalized moments. In this paper, we present an approach that uses the eigenvalues and eigenvectors of a spatial weight matrix to directly construct consistent least-squares estimators of parameters of a general spatial dynamic panel data model. The proposed methodology is conceptually simple and efficient and can be easily implemented. We show that the proposed parameter estimators are consistent and asymptotically normally distributed under mild conditions. We demonstrate the superior performance of our approach via extensive simulation studies. We also provide a real data example.

Abstract P1-06-05: Image driven biophysical mathematical modeling of multicellular breast tumor spheroids

Abstract Introduction: Conventional 2D monolayer cell culture methods have distinct weaknesses in mimicking in vivo tumors as they ignore major phenotypic physical tumor microenvironment (TME) factors. 3D multicellular tumor spheroid (MTS) systems have been developed to study cancer cell growth in the presence of such TME factors, including cell-cell and cell-extracellular matrix (ECM) mechanical interactions. However, the development of analysis methodologies for these complex culture systems has considerably lagged. We hypothesize that a coupled experimental-computational framework using a microscopy image-driven biophysical mathematical model of MTS systems can accurately estimate phenotypic growth and biophysical TME properties, revealing significantly more mechanistic information than traditional morphometric analysis methods. Materials and Methods: MDA-MB-231 triple-negative breast cancer cells were used to generate spheroids. A single 3D MTS embedded in Collagen I ECM was used as a 3D culture tumor model system as they exhibit cell-cell and cell-ECM interactions that mimic in vivo tumors. Fluorescent fiducial microspheres were placed in the ECM to track displacement induced by the MTS. Time-lapse fluorescence microscopy images were taken of the MTS and microspheres for 72 hours on an automated fluorescent microscope every 12-hours. Images were compiled using a customized fully-automated tiling/stitching/image processing software in MATLAB. Time-lapse images of the MTS and microspheres are used in conjunction with subsequent mathematical modeling analysis for biophysical parameter estimation. We assess MTS using a mathematical model based on two coupled partial differential equations describing growth/motility and mechanical equilibrium in response to cellular-induced forces. Parameters describing cell proliferation rate (k), cell diffusion (D), and cellular mechanical forces (λ) are estimated based on fitting acquired microscopy images to the model via an iterative least-squares parameter estimation. Experiments and model-based analyses were performed in triplicate to define the reproducibility of biophysical parameter estimation. Results and Discussion: In this preliminary study, our modeling framework is able to estimate biophysical parameters of cell diffusion, proliferation rate, and cellular mechanical force in a 3D breast cancer cell culture system of invading MTS embedded in ECM with coefficients of variation across all parameters less than 15% on average. The model predictions accurately represent changes in cellular density and ECM deformation induced by cellular traction forces throughout the observed time series, demonstrating that this model is capable of representing MTS/ECM growth behavior and parameterizing the driving biophysical properties. Conclusions: These results indicate that our imaging-driven mathematical modeling framework for estimating biophysical model parameters has potential to reliably characterize 3D MTS growth and invasion. With reproducibility established, we plan to extend the method to investigate these phenotypic biophysical TME properties in a range of culture and treatment condition interventions. These biophysical properties could be utilized in developing anti-neoplastic drug sensitivity assays to improve cancer therapy by enabling patient-specific estimation/prediction of response to agents and dosing conditions. Acknowledgements: NIH-NCI K25CA204599 and P30CA012197 Citation Format: Haley Brooke Johnson, Emily E. Fannin, Alexandra Thomas, Jared A Weis. Image driven biophysical mathematical modeling of multicellular breast tumor spheroids [abstract]. In: Proceedings of the 2019 San Antonio Breast Cancer Symposium; 2019 Dec 10-14; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2020;80(4 Suppl):Abstract nr P1-06-05.

Estimating Regression Function of Multi-response Semiparametric Regression Model Using Smoothing Spline

The article describes a new estimation method of regression functions in a multi-response semiparametric regression model based on smoothing spline. The multi-response semiparametric regression model is a combined model between a parametric regression model and a nonparametric regression model, where there is a correlation between responses. The proposed estimation method enhances the flexibility of the multi-response semiparametric regression model by combining a goodness of fit function and a penalty function to calculate an estimation which not only considers the goodness of fitting of the model, but also the smoothness of the estimation model curve. The optimal trade-off between goodness and smoothness can be achieved by selecting the optimal smoothing parameters. The article discusses a theoretically proposed method for estimating this multi-response semiparametric regression model regression function of parametric and nonparametric components. We use the weighted least squares method to estimate the parametric component parameters, we determine the goodness of fit and penalty functions using the reproducing kernel Hilbert space method, and then take the result of penalized weighted least squares optimization to obtain an estimate of the nonparametric component. The new research results are a weighted least squares estimator of parameters of parametric components, and a weighted partial smoothing spline estimator of the nonparametric component. The result shows that the estimated multi-response semiparametric regression model is linear to the observation, and is a combination of the estimations of the parametric and nonparametric components. The research results of the estimation of this model can be applied to medical fields for predictive purposes.

Filterless Least-Squares Based Adaptive Stochastic Continuous-Time Nonlinear Control

Abstract In continuous-time system identification and adaptive control, the least-squares parameter estimation algorithm has always been used with regressor filtering, in order to avoid using time-derivatives of the measured state. Filtering adds to the dynamic order of the identifier and affects its performance. We solve the problem of filterless least-squares-based adaptive control for stochastic strict-feedback nonlinear systems with an unknown parameter in the drift term. The novel ingredient in our least-squares identification is that the update law for the parameter estimate is not a simple integrator but it also incorporates a feedthrough effect, namely, the parameter estimator is of relative degree zero (rather than one) relative to the update function. The feedthrough in the update law is a carefully designed nonlinear function, which incorporates the integration with respect to state (and not time) of the regressor function, the purpose of which is to eliminate the need for time-filtering of the regressor. Our backstepping design of the control law compensates the adverse effect of the noise (the Hessian nonlinear term, involving the diffusion nonlinearity, in the Lyapunov analysis) on the least-squares estimator. Such a controller also enables a construction of an single overall Lyapunov function, quadratic in the parameter error and quartic in the transformed state, to guarantee that the equilibrium at the origin of the closed-loop system is globally stable in probability and the states are regulated to zero almost surely.

Adaptive identification of anaerobic digestion process for biogas production management systems.

To achieve the goals of sustainable development, supplies of renewable energy must be increased and methods of stable production developed. This study focused on the anaerobic digestion process using biomass as a raw material, which represents a renewable energy resource which is robust to environmental change and can be adjusted to suit supply and demand. A state-space model of the process was built in this study, consisting of two differential equations and one algebraic equation. The parameters included in the model are dependent on the operating conditions of the process. Automatic estimation of parameters from the input and output data of the process enables easy use of the model under any operating conditions. An adaptive-identifier control system was introduced as the parameter-estimation system, which made it possible to obtain the least squares estimate of parameters. When accumulated biogas generation per day was predicted using the model, goodness-of-fit analysis indicated an accuracy of over 80% in all cases, validating the model and estimated parameters. Future tasks will involve implementation of model predictive control into anaerobic digestion processes with the model and parameter-estimation system developed in this study.

Estimation and variable selection for partially linear additive models with measurement errors

This article is concerned with statistical inference of partially linear additive regression models where the covariates in parametric component are measured with errors. Using polynomial spline approximations, we propose bias-corrected least squares estimators for parameters and establish the asymptotic normality, and show that the estimators of unknown functions achieve optimal nonparametric convergence rate. Moreover, we propose a variable selection procedure to identify significant regressors and derive the oracle property of penalized estimators. Finally, we propose two-stage local polynomial estimation for additive functions and show the corresponding asymptotical normality. Monte carlo studies and real data analysis illustrate the performance of our approaches.

Optimal Experimental Design for Parameter Identification of PEM Fuel Cell Models

Modeling has been an invaluable tool in analyzing and optimizing polymer electrolyte membrane (PEM) fuel cells for improved performance and durability. Moreover, computationally efficient models may be used as software sensors and provide critical information about the internal states of an operating fuel cell. While the utility of fuel cell models has been shown in a myriad of modeling works in the literature, model parameterization remains a relatively understudied subject. The fact that all fuel cell models rely on some level of empiricism, such as the uptake isotherms used for membrane water uptake, makes the problem of proper model parameterization even more significant for fuel cell models. To date, most of the models in the literature have been parameterized using extensive material characterization obtained from the literature [1]. There are several drawbacks to such an approach. First, it requires extensive resources to be able to characterize individual cell components. Second, such characterization is often invasive and requires taking the cell apart. Finally, parameterizing a model using ex-situ characterization data may reduce the model’s prediction capabilities when it is applied to operating fuel cells. Optimization-based approaches offer some remedy to many of these problems by enabling direct parameterization of the models using non-invasive measurements obtained on an operating fuel cell [2]. Although these techniques have been used occasionally to identify a few model parameters [3], [4], their application is rare, which is most likely due to the significant computational cost of many of the fuel cell models available in the literature. Nevertheless, recent progress in the battery research community highlights several opportunities for effective parameterization of electrochemical energy systems [5]–[7]. Building up on this literature, in this talk we present some results on optimal design of experiments for identifying the parameters of a recently developed model of PEM fuel cells [8], [9]. The optimal experiments help improve identifiability of the parameters of interest using non-invasive performance measurements. Furthermore, we present a methodology for systematic parameterization of such models. The results indicate the importance of optimally designing experiments for the purpose of parameter identification and highlight the utility of the systematic approach in effective model parameterization. We also discuss some issues regarding structural identifiability of fuel cell models and how additional measurements may help alleviate some of these issues. Acknowledgement: Financial support for this work was provided by Ford Motor Company. [1] A. Goshtasbi, P. Garcia-Salaberri, J. Chen, K. Talukdar, D. G. Snachez, and T. Ersal, “Through-the-Membrane Transient Phenomena in PEM Fuel Cells: A Modeling Study,” J. Electrochem. Soc., vol. 166, no. 7, 2019. [2] A. Goshtasbi, J. Chen, J. R. Waldecker, S. Hirano, and T. Ersal, “On Parameterizaing PEM Fuel Cell Models,” in Proceedings of the 2019 American Control Conference, 2019. [3] P. Dobson, C. Lei, T. Navessin, and M. Secanell, “Characterization of the PEM fuel cell catalyst layer microstructure by nonlinear least-squares parameter estimation,” J. Electrochem. Soc., vol. 159, no. 5, pp. B514–B523, 2012. [4] T. C. Yau, P. Sauriol, X. T. Bi, and J. Stumper, “Experimental determination of water transport in polymer electrolyte membrane fuel cells,” J. Electrochem. Soc., vol. 157, no. 9, pp. B1310–B1320, 2010. [5] L. Zhang et al., “Parameter sensitivity analysis of cylindrical lifepo4 battery performance using multi-physics modeling,” J. Electrochem. Soc., vol. 161, no. 5, pp. A762–A776, 2014. [6] S. Park, D. Kato, Z. Gima, R. Klein, and S. Moura, “Optimal Experimental Design for Parameterization of an Electrochemical Lithium-Ion Battery Model,” J. Electrochem. Soc., vol. 165, no. 7, pp. A1309–A1323, 2018. [7] A. M. Bizeray, J.-H. Kim, S. R. Duncan, and D. A. Howey, “Identifiability and parameter estimation of the single particle lithium-ion battery model,” IEEE Trans. Control Syst. Technol., no. 99, pp. 1–16, 2018. [8] A. Goshtasbi, B. L. Pence, and T. Ersal, “A real-time pseudo-2D bi-domain model of PEM fuel cells for automotive applications,” in ASME 2017 Dynamic Systems and Control Conference, 2017, p. V001T25A001-V001T25A001. [9] A. Goshtasbi, B. L. Pence, and T. Ersal, “Computationally efficient pseudo-2D non-isothermal modeling of polymer electrolyte membrane fuel cells with two-phase phenomena,” J. Electrochem. Soc., vol. 163, no. 13, pp. F1412–F1432, 2016.

Optimal experiment design in nonlinear parameter estimation with exact confidence regions

A model-based optimal experiment design (OED) of nonlinear systems is studied. OED represents a methodology for optimizing the geometry of the parametric joint-confidence regions (CRs), which are obtained in an a posteriori analysis of the least-squares parameter estimates. The optimal design is achieved by using the available (experimental) degrees of freedom such that more informative measurements are obtained. Unlike the commonly used approaches, which base the OED procedure upon the linearized CRs, we explore a path where we explicitly consider the exact CRs in the OED framework. We use a methodology for a finite parametrization of the exact CRs within the OED problem and we introduce a novel approximation technique of the exact CRs using inner- and outer-approximating ellipsoids as a computationally less demanding alternative. The employed techniques give the OED problem as a finite-dimensional mathematical program of bilevel nature. We use two small-scale illustrative case studies to study various OED criteria and compare the resulting optimal designs with the commonly used linearization-based approach. We also assess the performance of two simple heuristic numerical schemes for bilevel optimization within the studied problems.

Multiplicative coevolution regression models for longitudinal networks and nodal attributes

We introduce a simple and extendable coevolution model for the analysis of longitudinal network and nodal attribute data. The model features parameters that describe three phenomena: homophily, contagion and autocorrelation of the network and nodal attribute process. Homophily here describes how changes to the network may be associated with between-node similarities in terms of their nodal attributes. Contagion refers to how node-level attributes may change depending on the network. The model we present is based upon a pair of intertwined autoregressive processes. We obtain least-squares parameter estimates for continuous-valued fully-observed network and attribute data. We also provide methods for Bayesian inference in several other cases, including ordinal network and attribute data, and models involving latent nodal attributes. These model extensions are applied to an analysis of international relations data and to data from a study of teen delinquency and friendship networks.

A mechanical impedance-based measurement system for quantifying Parkinsonian rigidity

At present the parkinsonian rigidity assessment depends on subjective judgment of neurologists according to their experience. This study presents a parkinsonian rigidity quantification system based on the electromechanical driving device and mechanical impedance measurement method. The quantification system applies the electromechanical driving device to perform the rigidity clinical assessment tasks (flexion-extension movements) in Parkinson's disease (PD) patients, which captures their motion and biomechanical information synchronously. Qualified rigidity features were obtained through statistical analysis method such as least-squares parameter estimation. By comparing the judgments from both the parkinsonian rigidity quantification system and neurologists, correlation analysis was performed to find the optimal quantitative feature. Clinical experiments showed that the mechanical impedance has the best correlation (Pearson correlation coefficient r = 0.872, P < 0.001) with the clinical unified Parkinson's disease rating scale (UPDRS) rigidity score. Results confirmed that this measurement system is capable of quantifying parkinsonian rigidity with advantages of simple operation and effective assessment. In addition, the mechanical impedance can be adopted to help doctors to diagnose and monitor parkinsonian rigidity objectively and accurately.

Fast accurate multi-key weight measurement

We consider an industrial problem brought to the Mathematics in Industry New Zealand study group in 2016, where items pass briefly over load cells resulting in a noisy oscillatory signal, from which the mass of the item is to be computed. We compare results obtained using a single load cell for one piece of fruit, with results from passing over two load cells in tandem or in succession with fruit on multiple keys. We develop mathematical models to assist with the computation of total load mass, considering both deterministic and statistical approaches. The fitting of simple harmonic motion plus a step function exhibits the possibility of rapid estimates of load mass. We find that using multiple keys to measure the weight of a fruit provides more accurate results than using the single-key method. References minz-2004, Case Study Compac. http://www.minz.org.nz/assets/downloads/Case-Study-Compac.pdf Kesilmis, Z., and Baran, T. (2016). A Geometric Approach to Beam Type Load Cell Response for Fast Weighing. doi:10.1007/s12647-016-0168-2 K. Levenberg, A method for the solution of certain non-linear problems in least squares. Quarterly of applied mathematics , 2(1944), pp. 164–168. http://www.jstor.org/stable/43633451 D. W. Marquardt, An algorithm for least-squares estimation of nonlinear parameters. Journal of the society for Industrial and Applied Mathematics , 11(1963), pp. 431–441. doi:10.1137/0111030

Solution to Green–Ampt infiltration model using a two-step curve-fitting approach

Infiltration is an important hydrological process affecting the runoff, groundwater recharge and solute transport process. The Green–Ampt (GA) model, describing one-dimensional infiltration process, yields an implicit equation for estimation of cumulative infiltration with time. In this paper, an explicit approximation to the GA model is derived using two-step curve-fitting technique. The Marquardt’s algorithm is employed for least-squares estimation of nonlinear parameters. Performance of the proposed model is compared with the implicit GA model using numerical and published field experimental data. The quantitative statistical indicators namely, percent relative error (RE), maximum absolute percent relative error (MARE), percent bias (PB), and Nash–Sutcliffe modeling efficiency (E), are used to assess the performance of the proposed model. The solution of the proposed model matched very well with that of the implicit GA model with MARE ≤ 0.146%, PB ≤ 0.070%, and E value approximately equal to 1 for both the dimensionless numerical and dimensional field experimental infiltration estimates. Simulation results of the proposed model demonstrated the capability of the derived model in estimating infiltration rate and cumulative infiltration accurately and can be applied to solve variety of real-life hydrological problems.

The volume-dependent Forced Oscillation Technique

Abstract This paper describes a novel extension of the Forced Oscillation Technique (FOT), namely the volume-dependent Forced Oscillation Technique (v-FOT). This extension is based on the measurement of the respiratory impedance over the whole lung volume. A new measurement protocol was introduced: subjects breathe slowly from residual volume to total lung capacity, while oscillations are applied simultaneously. The respiratory impedance is computed by means of the windowed cosine fitting method. The respiratory impedance becomes frequency- and volume-dependent Z(jω,V). The novel impedance-volume (ZV) diagram was introduced for diagnostic purposes. Least-squares parameter estimation was performed for the extended-RIC and the constant phase models. However, both models failed in presenting the volume-dependency of Z. We proposed two new models: the volume-dependent extended RIC and the volume-dependent Mead model for a better fit of the data. The volume-dependent Mead model yields the smallest estimation error.

A Mathematical-Statistics Approach to the Least Squares Method

We consider a mathematical-statistics approach to least-squares parameter estimation in a linear multiple regression model. This approach has led to a detailed description of the basic premises for the emergence and application of the least-squares method, produced a number of general distributional and statistical formulas for the estimation of model parameters independently of a specific joint distribution of the random variables, provided a deeper understanding of the parameter estimation risks associated with model specification errors, and made it possible to identify the place and role of knowledge of the theoretical and empirical distributions of observation errors.

A Two-Stage Latent Variable Estimation Procedure for Time-Censored Accelerated Degradation Tests

Parallel constant-stress accelerated degradation testing (PCSADT) is widely used to assess the reliability of highly reliable products in a timely manner when the products’ degradation can be measured. Under a time-censored PCSADT, several groups of units are tested simultaneously, but under different stress levels, until a prespecified censoring time is reached. At this time, degradation values from the censored units, and failure times of the failed units are obtained. When the degradation follows a Wiener process where the parameters depend on the stress level through a life-stress model containing an unknown nuisance parameter, estimating this parameter often biases the maximum likelihood and least-squares estimators of the lifetime parameters. In this paper, we propose a two-stage procedure to address this problem. In the first stage, we transform the data under the different stress levels of a PCSADT so that the resulting data can be considered to have been obtained under normal stress. In the second stage, we introduce a latent variable for the unobserved degradation after the failure time for each failed unit to obtain a pseudodegradation value at the censoring time. We then use all degradation values (pseudo or observed) at the censoring time to develop latent variable estimators for all model parameters. Unlike other existing estimators, the proposed estimators are shown to be s-consistent, have closed-form expressions, and are easy to interpret. We use a real example of light-emitting diodes to illustrate the proposed method. In addition to proving s-consistencies, we conduct a simulation study to demonstrate that the proposed estimators also perform well in finite samples.

Inference on q-Weibull parameters

The q-Weibull distribution is a generalization of the Weibull distribution and could describe complex systems. We firstly point out how to derive the maximum likelihood estimates (MLEs) and least-squares estimates (LSEs) of the q-Weibull parameters. Next, three confidence intervals (CIs) for the q-Weibull parameters are constructed based on bootstrap methods and asymptotic normality of the MLEs. Explicit expressions for the Fisher information matrix necessary for the asymptotic CIs are derived. A Monte Carlo simulation study is conducted to compare the performances of the MLEs and LSEs as well as the different CIs. The simulation results show that the MLEs are superior to the LSEs in terms of both bias and mean squared error. The bootstrap CIs based on the MLEs are shown to have good coverage probabilities and average interval widths. Finally, a real data example is provided to illustrate the proposed methods.

Kinetic Modelling of the catalytic oxidation of 2-(methylmercapto)-benzothiazole under mild conditions

2-(Methylmercapto)-benzothiazole oxidation was performed over Copper modified zeolites. The microporous materials were synthesized by the hydrothermal crystallization method and later modified with metal incorporation by wet impregnation. The solid catalysts were characterized by means of X-ray diffraction, surface area determinations, inductively coupled plasma emission spectrophotometry, temperature programmed reduction and Fourier transformed infrared spectroscopy. Reaction parameters (Copper content, nature of the solvent, hydrogen peroxide concentration, reaction time, catalyst mass and reaction temperature) were evaluated to reach the optimum reaction conditions. Kinetic modelling and kinetic parameter estimation based on experimental kinetic data were included. Cu(II) species were confirmed as the active sites of the catalysts, being Y zeolite the optimal support. Thus, Fenton-like reaction mechanism with reactive oxygen species as hydroxyl and hydroperoxyl radicals (HO, HO2) was confirmed. Reactions rates constants values and activation energy were determined by non-linear least-square parameter estimation. Also, homogeneous catalytic system was evaluated in order to discard diffusional limitations.