Parameter Estimation Techniques Research Articles

Validating a Calibrated Model of a Groundwater Pump-And-Treat System Using Robust Multiple Regression

Validation of groundwater models is relatively challenging due to the need to reserve scarce water level data for calibration targets. In addition, traditional statistical validation metrics are unintuitive for non-technical audiences and do not directly identify model behaviors that require further refinement. We developed a novel model validation method that analyzes rate change events at pump-and-treat wells and statistically compares the water level responses at nearby monitoring wells between the data and model. The method takes advantage of events that occur alongside ambient pumping, unlike parameter estimation techniques that require independent drawdown or recovery events. The ability of the model to match well connections that are evident (or not evident) in the observations is characterized statistically, leading to four decision scenarios: model matches the observed connection (1) or lack thereof (2), model exhibits a connection that is not observed (3), or model over- or understates the observed connection (4). The method is applied to an FEHM-based groundwater flow and transport model that is shown to match 84.5% of the well connections analyzed. The method provides novel perspectives on the influence of calibration targets on the flow field and suggests that although the overall effect of drawdown targets was to improve the model, the choice of target well pairs creates flow pathways that may be inconsequential during normal operational conditions. The model adequately matches the flow over short spatial scales (<800 m) and over-represents the flow over greater distances (300–1200 m), suggesting the need for “null” drawdown targets in subsequent rounds of calibration.

Noise aware parameter estimation in bioprocesses: Using neural network surrogate models with nonuniform data sampling

AbstractThis article demonstrates a parameter estimation technique for bioprocesses that utilizes measurement noise in experimental data to determine credible intervals on parameter estimates, with this information of potential use in prediction, robust control, and optimization. To determine these estimates, the work implements Bayesian inference using nested sampling, presenting an approach to develop neural network‐ (NN) based surrogate models. To address challenges associated with nonuniform sampling of experimental measurements, an NN structure is proposed. The resultant surrogate model is utilized within a Nested Sampling Algorithm that samples possible parameter values from the parameter space and uses the NN to calculate model output for use in the likelihood function based on the joint probability distribution of the noise of output variables. This method is illustrated against simulated data, then with experimental data from a Sartorius fed‐batch bioprocess. Results demonstrate the feasibility of the proposed technique to enable rapid parameter estimation for bioprocesses.

Advanced parameter estimation for lithium-ion battery model using the information sharing group teaching optimization algorithm

Accurate battery modeling is crucial for optimizing the performance and safety of Lithium-ion batteries (LiBs), particularly in applications such as electric vehicles and smart grids. This paper introduces the Information Sharing Group Teaching Optimization Algorithm (ISGTOA), a novel human-based metaheuristic algorithm designed to estimate the 21 parameters of third-order Equivalent Circuit Model (ECM) for LiBs. Unlike existing methods, ISGTOA effectively manages the increased complexity of a model through advanced group teaching strategies and sophisticated information-sharing mechanisms inspired by classroom dynamics. We validated the effectiveness of ISGTOA using two distinct datasets—High Dynamic Profile (HDP) and Urban Dynamometer Driving Schedule (UDDS)—across various LiB chemistries (NCA and NMC) and temperature conditions (−5 °C, 25 °C and 45 °C). The results demonstrate that ISGTOA achieves superior parameter estimation accuracy and convergence speed compared to established and recent optimization methods such as TLBO, TLSBO, AISA, MTBO, and DTBO. Specifically, ISGTOA attained a minimum RMSE of 8.357 mV with rapid convergence and high stability in the HDP dataset and minimized RMSE to 10 mV within ten iterations at both 25 °C and 45 °C in the UDDS dataset. Additionally, ISGTOA exhibited high efficiency (up to 97.75 %) under varying thermal environments. This work not only introduces a novel algorithm for complex battery modeling but also sets the stage for future advancements in real-time battery management systems, emphasizing the importance of robust, versatile, and efficient parameter estimation techniques.

Model-Centric Integration of Uncertain Expert Knowledge into Importance Sampling-Based Parameter Estimation

This study presents a model-based parameter estimation method for integrating and validating uncertainty in expert knowledge and simulation models. The parameters of the models of complex systems are often unknown due to a lack of measurement data. The experience-based knowledge of experts can substitute missing information, which is usually imprecise. The novelty of the present paper is a method based on Monte Carlo (MC) simulation and importance sampling (IS) techniques for integrating uncertain expert knowledge into the system model. Uncertain knowledge about the model parameters is propagated through the system model by MC simulation in the form of a discrete sample, while IS helps to weight the sample elements regarding imprecise knowledge about the outputs in an iterative circle. Thereby, the consistency of expert judgments can be investigated as well. The contributions of this paper include an expert knowledge-based parameter estimation technique and a method for the evaluation of expert judgments according to the estimation results to eliminate incorrect ones. The applicability of the proposed method is introduced through a case study of a Hungarian operating waste separation system. The results verify that the assessments of experts can be efficiently integrated into system models, and their consistency can be evaluated.

Application of chaotic teaching-learning-based optimization technique for estimating unknown parameters of proton exchange membrane fuel cell model.

Proton exchange membrane fuel cells (PEMFC) possess features like high specific power density, low operating temperature, and low operating pressure and thus are most widely used. The performance of PEMFC highly depends on its output voltage which further affects its efficiency. This research paper aims to improve this output voltage by minimizing the losses through parameter estimation technique for three well-known commercial PEMFC stacks, namely, 250W, BCS 500W, and Ballard Mark V. The multiobjective function framed for optimization serves two goals. One is to extract the unknown parameters of FC, and second is to achieve the minimum sum of squared errors (SSE) that occur due to the difference between experimental voltage and estimated voltage. Chaotic teaching-learning-based optimization (CTLBO) algorithm is used for optimization process. SSE values obtained for three commercial PEMFC stacks 250W, BCS 500W, and Ballard Mark V are 7.02267, 4.00150, and 0.8100, respectively. The applicability of this technique is further checked for different operating conditions of each model. The I-P (current-power) curve and I-V (current-voltage) curve obtained using estimated data closely matched the experimental data that showed the practical relevance and efficacy of the algorithm. A comparison is done among the SSE results obtained using CTLBO to other competitive algorithms mentioned in the literature. CTLBO showed its superiority over other algorithms in estimating the unknown parameters of PEMFC stack parameters.

Deep optimal experimental design for parameter estimation problems

Abstract Optimal experimental design is a well studied field in applied science and engineering. Techniques for estimating such a design are commonly used within the framework of parameter estimation. Nonetheless, in recent years parameter estimation techniques are changing rapidly with the introduction of deep learning techniques to replace traditional estimation methods. This in turn requires the adaptation of optimal experimental design that is associated with these new techniques. In this paper we investigate a new experimental design methodology that uses deep learning. We show that the training of a network as a Likelihood Free Estimator can be used to significantly simplify the design process and circumvent the need for the computationally expensive bi-level optimization problem that is inherent in optimal experimental design for non-linear systems. Furthermore, deep design improves the quality of the recovery process for parameter estimation problems. As proof of concept we apply our methodology to two different systems of Ordinary Differential Equations.

Maximizing similarity: using correlation coefficients to calibrate kinetic parameters in population balance models

Crystallization plays a crucial role as a separation and purification technique, particularly in the chemical and pharmaceutical industries. By adjusting process parameters, the productivity, product quality, and efficiency of downstream processes can be improved. In complex processes, model-based design becomes invaluable. Population Balance Models (PBMs) have successfully aided the chemical industry in achieving more effective production processes for decades. These models can utilize various input data sources to identify the dominant mechanisms and calibrate model parameters. While inline particle monitoring tools serve as excellent qualitative descriptors, challenges arise from experimentation and data interpretation, hindering their direct application in the kinetic parameter estimation of PBMs. In this study, we present a novel approach that utilizes information from inline particle monitoring tools for the kinetic parameter estimation of PBMs, bypassing the associated obstacles. Our pioneering approach relies on offline product size data and the correlation-based utilization of inline particle monitoring information. The paper compares this novel strategy with two parameter estimation techniques: the classical method using solute concentration and product size data and a somewhat naïve approach, which assumes that the inline particle monitoring data can directly be compared with the simulations. The prediction capabilities are evaluated through two in-silico case studies. The results indicate that the precision and predictive capability of the correlation-based technique are comparable to the classical approach, both for noisy data and for a system undergoing significant agglomeration and deagglomeration. The use of Pearson’s correlation coefficient yields the best results in the novel cases. These findings in in-silico datasets provide a foundation and motivation for the practical application of this idea, unleashing the so-far hidden model development potential of such measurements.

Efficient parameter estimation in biochemical pathways: Overcoming data limitations with constrained regularization and fuzzy inference

In analytical modeling for biochemical pathways, precisely determining unknown parameters is paramount. Traditional methods, reliant on experimental time course data, often encounter roadblocks — limited accessibility and variable quality — that can significantly impact the algorithm’s performance. In this study, we address these hurdles by unveiling a groundbreaking parameter estimation technique, Constrained Regularized Fuzzy Inferred Extended Kalman Filter (CRFIEKF). This innovative approach eliminates the need for experimental time-course measurements and capitalizes on the existing imprecise relationships among the molecules within the network. Our proposed framework integrates a Fuzzy Inference System (FIS) block to encapsulate these approximated relationships. To fine-tune the estimated parameter values, we employ Tikhonov regularization. The selection of Tikhonov regularization and Gaussian membership functions was based on the Mean Squared Error (MSE) values observed during the parameter estimation process, contrasting our results with those of previous studies.We rigorously tested the proposed approach across various pathways, from the glycolytic processes in mammalian erythrocytes and yeast cells to the intricate JAK/STAT and Ras signaling pathways. The results were impressive, showing a significant similarity (p-value < 0.001) to the outcomes of specific prior experiments. The dynamics of the biochemical networks normalized within the [0, 1] range mirrored the transient behavior (MSE < 0.5) of both in vivo and in silico results from previous studies. In conclusion, our findings highlight the effectiveness of CRFIEKF in estimating the kinetic parameter values without prior knowledge of experimental data within a biochemical pathway in the state-space model. The proposed method underscores its potential as a game-changer in biochemical pathway analysis.

Optimizing parameter learning and calibration in an integrated hydrological model: Impact of observation length and information

Integrated hydrological modeling is gaining popularity due to its mechanistic representation of the surface and subsurface processes. However, estimating the parameters of such process-based models can be computationally expensive if careful consideration is not given to the length of streamflow observations used during model calibration. Here we evaluate the influence of the calibration period, the role of streamflow information content, and the gage location in parameter learning and calibration of a fully integrated hydrological model, the Advanced Terrestrial Simulator (ATS). We conducted the study at the Upper Neversink River Watershed within the Delaware River Basin, where streamflow observations are available at 11 gages with varying record lengths. We leveraged a recently proposed knowledge-informed deep learning technique for parameter estimation. To assess the impact of observation period and gage location, model parameters were learned on scenarios using different chunks of streamflow observations, including (1) using only one year or consecutive multiple years of streamflow observations at the watershed outlet and (2) using one shared year of observations at each of the sub-catchment gages, with the period from 1991-10-01 through 1999-09-30 as the overall calibration period. Using the estimated parameters, ATS was rerun for each scenario and evaluated on a subsequent period from 1999-10-01 through 2002-09-30. Results show that the basin outlet discharge prediction is mostly improved when using at least four years of observations for parameter estimation. Further, the performance of the calibrated ATS run correlates with the information content of the observed streamflows, suggesting that the information-theoretic metrics could be indicators for selecting the observation period for parameter estimation. Finally, we find that observations from an informative gage can be used in learning parameters to predict the streamflow at a nearby gage, which would potentially lower the computational expense by reducing the watershed domain used in calibration. Our success underscores the potential of using information theory to achieve robust model parameter estimation on a reduced computational budget.

Analyzing parametric influences driving age-associated changes in absorption using a PBPK-GSA approach

The advanced age population may be susceptible to an increased risk of adverse effects due to increased drug exposure after oral dosing. Factors such as high-interindividual variability and lack of data has led to poor characterization of absorption's role in pharmacokinetic changes in this population. Physiologically based pharmacokinetic (PBPK) models are increasingly being used during the drug development process, as their unique qualities are advantageous in atypical scenarios such as drug-drug interactions or special populations such as older people. Along with relying on various sources of data, auxiliary tools including parameter estimation and sensitivity analysis techniques are employed to support model development and other applications. However, sensitivity analyses have mostly been limited to localized techniques in the majority of reported PBPK models using them. This is disadvantageous, since local sensitivity analyses are unsuitable for risk analysis, which require assessment of parametric interactions and proper coverage of the input space to better estimate and subsequently mitigate the effects of the phenomenon of interest. For this reason, this study seeks to integrate a global sensitivity analysis screening method with PBPK models based in PK-Sim® to characterize the consequences of potential changes in absorption that are often associated with advanced age. The Elementary Effects (Morris) method and visualization of the results are implemented in R and three model drugs representing Biopharmaceutical Classification System classes I-III that are expected to exhibit some sensitivity to three age-associated hypotheses were successfully tested.

STATISTICAL INFERENCE ON SINE-EXPONENTIAL DISTRIBUTION PARAMETER

The Sine-Exponential (Sine-E) distribution is a probability distribution that combines the periodic behavior of the sine function with the decay characteristic of the exponential function. This study addresses the problem of identifying the most accurate and reliable estimation method for the parameter of the Sine-E distribution. The objective is to evaluate various parameter estimation techniques, including Maximum Likelihood Estimation (MLE), Least Squares Estimation (LSE), Weighted Least Squares Estimation (WLSE), Maximum Product of Spacing Estimation (MPSE), Cramer-von-Mises Estimation (CVME), and Anderson-Darling Estimation (ADE), using Mean Square Error (MSE) as the criterion for determining the technique with the minimum error. The study’s findings reveal that as sample size increases, the parameter estimates for all techniques converge to the true parameter value, with decreases in bias, MSE, and mean relative estimates. Among the techniques evaluated, the MPSE method consistently provides estimates closest to the true parameter value and exhibits the least bias and lowest MSE across small, moderate, and large sample sizes, making it the best estimator for the Sine-E distribution.

A Bayesian Tensor Decomposition Method for Joint Estimation of Channel and Interference Parameters.

Bayesian tensor decomposition has been widely applied in channel parameter estimations, particularly in cases with the presence of interference. However, the types of interference are not considered in Bayesian tensor decomposition, making it difficult to accurately estimate the interference parameters. In this paper, we present a robust tensor variational method using a CANDECOMP/PARAFAC (CP)-based additive interference model for multiple input-multiple output (MIMO) with orthogonal frequency division multiplexing (OFDM) systems. A more realistic interference model compared to traditional colored noise is considered in terms of co-channel interference (CCI) and front-end interference (FEI). In contrast to conventional algorithms that filter out interference, the proposed method jointly estimates the channel and interference parameters in the time-frequency domain. Simulation results validate the correctness of the proposed method by the evidence lower bound (ELBO) and reveal the fact that the proposed method outperforms traditional information-theoretic methods, tensor decomposition models, and robust model based on CP (RCP) in terms of estimation accuracy. Further, the interference parameter estimation technique has profound implications for anti-interference applications and dynamic spectrum allocation.

Nonstationary longitudinal autoregressive mixed model for count data with measurement error in covariates: Estimation and asymptotics

Several methods have been proposed in the literature for computing unbiased and efficient estimates of the parameters of generalized linear models when the covariates are measured with error. However, to our knowledge, no documented research on computational techniques for parameter estimation currently exist in the literature when the data is a longitudinal count data influenced by an unobservable latent variable and observable covariates that are measured with error. In this paper, we propose a nonstationary conditionally Poisson mixed model for such data and develop unbiased estimating equations with iterative methods for computing estimates of the effect of the covariates, variance of the latent variable, and the correlation index parameter. The performance of the iterative methods is examined through extensive simulation studies. The results show that the methods performed well when the magnitude of the measurement error is not so large as to dominate or mask the effect of the true covariates. Using observed longitudinal count data on the number of patents awarded to 168 firms in the United States from 1974 to 1979 along with associated covariate information on the type of firm, log of the book value of capital in 1972 and research and development (R &amp; D) expenditures we have demonstrated how the methods proposed in this paper can be applied to a real data. In addition, we derive the influence function of the estimator of the covariate effect and discuss the asymptotic properties of the estimator. Journal of Statistical Research 2024, Vol. 58, No. 1, pp. 151-180.

Bias Reduction of Maximum Likelihood Estimation in the Inverse Xgamma Distribution

A distribution for modeling survival data that accounts for flexibility in modeling data with upside-down bathtub-shaped hazard rate functions is the inverse Xgamma distribution (IXG). The maximum likelihood method (MLE) is the most often used technique for parameter estimation of the IXG distribution. Conversely, the MLE is infamously biased for small sample sizes. This motivates us to produce almost unbiased estimators for IXG parameter. More precisely, we minimize MLE biases to the second degree of bias using two techniques for bias correction: bootstrap and analytical approaches. Two actual data applications and Monte Carlo simulations are used to compare the performances of these methods.

A Modeling Study on the Effect of Interstate Mobility Restrictions on the SARS-CoV-2 Pandemic.

Mobility is a crucial element in comprehending the possible expansion of the transmission chain in an epidemic. In the initial phases, strategies for containing cases can be directly linked to population mobility restrictions, especially when only non-pharmaceutical measures are available. During the pandemic of COVID-19 in Brazil, mobility limitation measures were strongly opposed by a large portion of the population. Hypothetically, if the population had supported such measures, the sharp rise in the number of cases could have been suppressed. In this context, computational modeling offers systematic methods for analyzing scenarios about the development of the epidemiological situation taking into account specific conditions. In this study, we examine the impacts of interstate mobility in Brazil. To do so, we develop a metapopulational model that considers both intra and intercompartmental dynamics, utilizing graph theory. We use a parameter estimation technique that allows us to infer the effective reproduction number in each state and estimate the time-varying transmission rate. This makes it possible to investigate scenarios related to mobility and quantify the effect of people moving between states and how certain measures to limit movement might reduce the impact of the pandemic. Our results demonstrate a clear association between the number of cases and mobility, which is heightened when states are closer to each other. This serves as a proof of concept and shows how reducing mobility in more heavily trafficked areas can be more effective.

Bayesian Optimization for State and Parameter Estimation of Dynamic Networks with Binary Space.

This paper focuses on joint state and parameter estimation in partially observed Boolean dynamical systems (POBDS), a hidden Markov model tailored for modeling complex networks with binary state variables. The majority of current techniques for parameter estimation rely on computationally expensive gradient-based methods, which become intractable in most practical applications with large size of networks. We propose a gradient-free approach that uses Gaussian processes to model the expensive log-likelihood function and utilizes Bayesian optimization for efficient likelihood search over parameter space. Joint state estimation is also achieved alongside parameter estimation using the Boolean Kalman filter. The performance of the proposed method is demonstrated using gene regulatory networks observed through synthetic gene-expression data. The numerical results demonstrate the scalability and effectiveness of the proposed method in the joint estimation of the model parameters and genes' states.

Optimized Parameter Estimation and Integrating Neural Network Forecasting of Dynamic Plant-Livestock Model for Early Warning in Agro-Environment Control Systems

The research utilizes the Lotka-Volterra prey-predator model to study Plant-Herbivore dynamics, focusing on the relationship between traditional livestock farming and vegetation conditions. Advanced methods are developed to improve the precision and efficiency of parameter estimation in these models. Neural networks are incorporated to enhance forecasting abilities, and an extension of the Plant-Herbivore models includes Botswana's climate and livestock variables. Efficient parameter space exploration is achieved using the Runge-Kutta method along with Multistart and the local solver $fmincon$ in MATLAB. This method improves parameter estimation accuracy. To address the impact of homogeneity assumptions in the data, estimate aggregation through weighting and time conversion is applied. Furthermore, the study investigates the use of nonlinear least squares to further refine the process, allowing for the identification of parameters that best fit observed livestock data, even with non-linearity. By using optimized parameter estimation techniques along with normalized nonlinear least squares, the cumulative error was reduced from an initial 1563.4521 to a final value of 0.0038, well within the specified thresholds (1.0, 0.1, and 0.01). Comparisons between Autoregressive Integrated Moving Average (ARIMA) and Neural Network Auto-Regressive (NNAR) models showed that NNAR models outperformed ARIMA models, with lower variance estimates (0.000004 - 0.000562) compared to ARIMA (0.103 - 0.155). NNAR models displayed Mean Error (ME) values ranging from -0.0012 to 0.0140, indicating a close match between forecasts and actual values with minor deviations. As a result, NNAR forecasting was used for predicting soil moisture, death, and harvest rates, which were integrated into the extended Plant-Herbivore model. This integration enabled the estimation of livestock production trajectories for 2021-2022, along with corresponding interpretations. The study also assessed the uncertainty propagation from NNAR forecasts onto the Plant-Herbivore dynamic model, revealing an increase in uncertainty with longer lead times.

Modeling and Forecasting Short-Term Electricity Demand for Libyan Electric Network

Electrical load forecasting is a method of guessing the future of the electricity demand. Load forecast is an essential factor for power system planners and demand controllers. This is to ensure that would be an adequate amount of the electricity meeting the increasing demands. Therefore, an accurate load forecasting model can effectively lead to reduction of power system cost, better budget planning, and maintenance scheduling. The future load on a power system is predicted by extrapolating a predetermined relationship between the load and its influential variables. Determination of this relationship involves modeling and estimation the coefficient of the model through the use of an efficient parameter estimation technique. This paper presents a short term forecasting model for Libyan electric network using multi parameter regression method. The proposed method shows a result with small deviation between the actual and expected load, which means Multi parameter regression emerged as a suitable model for forecasting electricity demand in Libya.

Inference for the Parameters of a Zero-Inflated Poisson Predictive Model

In the insurance sector, Zero-Inflated models are commonly used due to the unique nature of insurance data, which often contain both genuine zeros (meaning no claims made) and potential claims. Although active developments in modeling excess zero data have occurred, the use of Bayesian techniques for parameter estimation in Zero-Inflated Poisson models has not been widely explored. This research aims to introduce a new Bayesian approach for estimating the parameters of the Zero-Inflated Poisson model. The method involves employing Gamma and Beta prior distributions to derive closed formulas for Bayes estimators and predictive density. Additionally, we propose a data-driven approach for selecting hyper-parameter values that produce highly accurate Bayes estimates. Simulation studies confirm that, for small and moderate sample sizes, the Bayesian method outperforms the maximum likelihood (ML) method in terms of accuracy. To illustrate the ML and Bayesian methods proposed in the article, a real dataset is analyzed.

Recurrent neural networks for aerodynamic parameter estimation with Lyapunov stability analysis

ABSTRACT Aircraft parameter estimation is crucial for designing accurate and reliable flight dynamic models, essential for integration into flight simulators and developing effective control laws. These parameters, encompassing aerodynamic characteristics, play a pivotal role in enhancing aircraft performance, safety and overall operational efficiency. This paper explores aircraft parameter estimation using two Recurrent Neural Network (RNN) approaches: (i) the weight-bias method (RNN-WB) and (ii) the gradient-based method (RNN-Gradient). RNN-WB is based on the Hopfield neural network and relies on pre-computation of weight and bias information, applying non-linearity to the states of the network. RNN-Gradient computes direct gradients and applies non-linearity to the error. Simulated flight data is employed in MATLAB implementations to assess estimation results, considering both performance metrics and computation time. The estimated parameters were also compared with estimates obtained from Unscented Kalman Filter (UKF). A novel methodology is introduced to establish the asymptotic stability of RNN-WB and RNN-Gradient methods, utilizing the normalized Lyapunov energy functional. The asymptotic stability ensures the reliability and robustness of these techniques for parameter estimation. The RNN-Gradient shows a 20% performance improvement over RNN-WB. Satisfactory simulation results from both algorithms provide a promising foundation for their potential applications to real flight test data.