Consistent Parameter Estimates Research Articles

Comparing AI versus Optimization Workflows for Simulation-Based Inference of Spatial-Stochastic Systems

Abstract Model parameter inference is a universal problem across science. This challenge is particularly pronounced in developmental biology, where faithful mechanistic descriptions require spatial-stochastic models with numerous parameters, yet quantitative empirical data often lack sufficient granularity due to experimental limitations. Parameterizing such complex models therefore necessitates methods that elaborate on classical Bayesian inference by incorporating notions of optimality and goal-orientation through low-dimensional objective functions that quantitatively encapsulate target system behavior. In this study, we contrast two such inference workflows and apply them to biophysically inspired spatial-stochastic models. Technically, both workflows employ simulation-based inference (SBI) methods: the first leverages a modern deep-learning technique known as sequential neural posterior estimation (SNPE), while the second relies on a classical optimization technique called simulated annealing (SA). We evaluate these workflows by inferring the parameters of two complementary models for the inner cell mass (ICM) lineage differentiation in the blastocyst-stage mouse embryo. This developmental biology system serves as a paradigmatic example of a highly robust and reproducible cell-fate proportioning process that self-organizes under strongly stochastic conditions, such as intrinsic biochemical noise and cell-cell signaling delays. Our results reveal that while both methods provide consistent model parameter estimates, the modern SBI workflow yields significantly richer inferred distributions at an equivalent computational cost. We identify the computational scenarios that favor the modern SBI method over its classical counterpart, and propose a plausible strategy to exploit the complementary strengths of both workflows for enhanced parameter space exploration.

Interaction model estimation-based robotic force-position coordinated optimization for rigid-soft heterogeneous contact tasks

Inspired by Model Predictive Interaction Control (MPIC), this paper proposes differential models for estimating contact geometric parameters and normal-friction forces and formulates an optimal control problem with multiple constraints to allow robots to perform rigid-soft heterogeneous contact tasks. Within the MPIC, robot dynamics are linearized, and Extended Kalman Filters are used for the online estimation of geometry-aware parameters. Meanwhile, a geometry-aware Hertz contact model is introduced for the online estimation of contact forces. We then implement the force-position coordinate optimization by incorporating the contact parameters and interaction force constraints into a gradient-based optimization MPC. Experimental validations were designed for two contact modes: “single-point contact” and “continuous contact”, involving materials with four different Young’s moduli and tested in human arm “relaxation-contraction” task. Results indicate that our framework ensures consistent geometry-aware parameter estimation and maintains reliable force interaction to guarantee safety. Our method reduces the maximum impact force by 50% and decreases the average force error by 42%. The proposed framework has potential applications in medical and industrial tasks involving the manipulation of rigid, soft, and deformable objects.

Parameter estimation from single patient, single time-point sequencing data of recurrent tumors.

In this study, we develop consistent estimators for key parameters that govern the dynamics of tumor cell populations when subjected to pharmacological treatments. While these treatments often lead to an initial reduction in the abundance of drug-sensitive cells, a population of drug-resistant cells frequently emerges over time, resulting in cancer recurrence. Samples from recurrent tumors present as an invaluable data source that can offer crucial insights into the ability of cancer cells to adapt and withstand treatment interventions. To effectively utilize the data obtained from recurrent tumors, we derive several large number limit theorems, specifically focusing on the metrics that quantify the clonal diversity of cancer cell populations at the time of cancer recurrence. These theorems then serve as the foundation for constructing our estimators. A distinguishing feature of our approach is that our estimators only require a single time-point sequencing data from a single tumor, thereby enhancing the practicality of our approach and enabling the understanding of cancer recurrence at the individual level.

Bayesian estimation of spatially varying mortality risk using tagged animal data

Abstract The survival of animals is impacted by landscapes of spatially varying mortality factors including habitat type, predation risk or harvest risk, among others. Characterization of these spatial mortality processes is important for managing animal populations and their habitats, yet this information has proved challenging to capture. Advances in telemetry now make it possible to monitor tagged individuals' locations through time for a wide range of taxa, providing opportunity to assess movement and mortality simultaneously through spatial time‐to‐event data. Existing time‐to‐event modelling frameworks have largely ignored spatially varying mortality processes or have treated an animal's location as fixed at a regional level. Here we present a general spatial survival model for analysing time‐to‐event data arising from animal telemetry. Our model has a flexible Cox regression form and can estimate the effects of conventional non‐spatial risk factors (size, seasonality, etc.), spatial covariates (e.g. habitat type) and spatial variation in risk not explained by covariates on mortality. We show how to fit our model using Bayesian methods and demonstrate applications of our model with several simulated examples and two animal telemetry case studies. Our model produced consistent and unbiased parameter estimates throughout simulations with a variety of spatial and non‐spatial hazards. In the first case study, data from 147 tagged caribou in British Columbia, Canada, revealed a spatially heterogeneous mortality landscape with caribou survival varying by elevation, likely in response to space use by predators. Our second case study involved a dataset of 390 acoustically tagged Atlantic cod in a southern Norwegian fjord where a marine protected area (MPA) was established during the study. We found that the MPA led to a shift from mostly fishing mortality to mostly natural mortality within the fjord and that these risks had markedly different spatial hazard patterns. Our spatial time‐to‐event model can make use of data from a variety of widely used telemetry technologies to characterize landscapes of mortality risks for different taxa. Our work provides new opportunities to inform the spatial ecology and management of fish and wildlife populations.

A robust M-estimator for Gaussian ARMA time series based on the Whittle approximation

In the frequency domain, the Whittle approach using the classical periodogram is traditionally used to estimate the parameters of a stationary time series. The M-periodogram has recently become an alternative tool for analyzing the dependence of time series. It is particularly useful for dealing with outliers and heavy-tailed noise. This paper proposes an alternative to the standard Whittle approach, the M-Whittle estimator, built from the M-periodogram. We show that the proposed method is a consistent estimator of the true parameters of an ARMA process. A finite sample investigation is carried out to assess the performance of the estimator in the scenarios of contaminated and uncontaminated time series. As expected, for the uncontaminated data, the M-Whittle estimator performs similarly to the classical Whittle approach. However, the superiority of the first method is clear in terms of root mean squared error when the series has additive outliers. Two applications are considered to illustrate the methodologies in real data contexts. Regardless of whether or not the data are contaminated by additive outliers, the results presented here strongly motivate using the M-Whittle estimator in practical problems.

Calibrating high-dimensional rock creep constitutive models for geological disaster prevention: An application of data assimilation methods

The study of rock creep phenomena is of paramount importance due to its potential to trigger geological disasters, such as landslides. To predict and prevent such disasters, creep constitutive models are widely employed to comprehend the time-dependent deformation of rocks. These models encompass various mechanical parameters that describe the intricate stress-strain behaviors. Nevertheless, significant challenges persist in achieving accurate and consistent parameter estimation and state prediction. In this study, we introduce three advanced data assimilation (DA) methods, including one Markov chain Monte Carlo method, DREAM(KZS), and two ensemble smoother methods, ESMDA and ILUES. This marks the first application of such methods for calibrating rock creep models in the scenario of geological disaster prevention. We conducted numerical simulations under both low- and high-dimensional conditions to assess the performance of these DA methods. For the single partition model, all three DA methods demonstrated promising results. In the high-dimensional case, DREAM(KZS) displayed inefficiency, while both ESMDA and ILUES proved to be still effective. ESMDA offered improved data matching but tended to underestimate parameter uncertainties, whereas ILUES excelled in addressing the issue of equifinality. In a real-world case focusing on characterizing creep deformation at the Mogu tilting deformation body near the Lianghekou Dam, China, we employed all three DA methods, and they collectively demonstrated satisfactory performance. Particularly noteworthy is the enhanced performance of the DREAM(KZS) method during the accelerated creep phase, even in the presence of limited data. The findings of this research bear significant importance in reducing uncertainties associated with model parameters in the realm of rock mechanics, thereby advancing our capabilities in predicting and preventing disasters.

Statistical analysis of multivariate binary time series based on a neural network model

This article is devoted to the statistical analysis of multivariate binary time series. For solving this problem a parsimonious neural network model of Markov’s ergodic chain of order s was determined. Consistent statistical estimators for model parameters and estimation algorithms of parameters and forecasting algorithms of future states of time series were developed. The results of computer experiments on simulated and real data are presented.

A Python tool for parameter estimation of “black box” macro- and micro-kinetic models with Bayesian optimization – petBOA

We develop an open-source Python-based Parameter Estimation Tool utilizing Bayesian Optimization (petBOA) with a unique wrapper interface for gradient-free parameter estimation of expensive black-box kinetic models. We provide examples for Python macrokinetic and microkinetic modeling (MKM) tools, such as Cantera and OpenMKM. petBOA leverages surrogate Gaussian processes to approximate and minimize the objective function designed for parameter estimation. Bayesian Optimization (BO) is implemented using the open-source BoTorch toolkit. petBOA employs local and global sensitivity analyses to identify important parameters optimized against experimental data, and leverages pMuTT for consistent kinetic and thermodynamic parameters while perturbing species binding energies within the typical error of conventional DFT exchange-correlation functionals (20-30 kJ/mol). The source code and documentation are hosted on GitHub. Program summaryProgram title: petBOACPC Library link to program files:https://doi.org/10.17632/hwwvksbb75.1Developer's repository link: https://github.com/VlachosGroup/petBOALicensing provisions: MIT licenseProgramming language: PythonExternal routines: NEXTorch, PyTorch, GPyTorch, BoTorch, Matplotlib, PyDOE2, NumPy, SciPy, pandas, pMuTT, SALib, docker.Nature of the problem: An open-source, gradient-free parameter estimation of black-box microkinetic modeling tools, such as OpenMKM is lacking.Solution method: petBOA is a Python-based tool that utilizes Bayesian Optimization and offers a unique wrapper interface for expensive black-box kinetic models. It leverages the pMuTT library for consistent kinetic and thermodynamic parameter estimation and employs both local and global sensitivity analyses to identify crucial parameters.

Facility Location Problem: Modeling Joint Disruptions Using Subordination

We study the facility location problem with disruptions where the objective is to choose a set of locations that minimizes the sum of expected servicing and setup costs. Disruptions can affect multiple locations simultaneously and are caused by multiple factors like geography, supply chain characteristics, politics, and ownership. Accounting for the various factors when modeling disruptions is challenging due to a large number of required parameters, the lack of calibration methodologies, the sparsity of disruption data, and the number of scenarios to be considered in the optimization. Because of these reasons, existing models neglect dependence or prespecify the dependence structures. Using partially subordinated Markov chains, we present a comprehensive approach that starts from disruption data, models dependencies, calibrates the disruption model, and optimizes location choices. We construct a metric and a calibration algorithm that learns from the data the strength of dependence, the number of necessary factors (subordinators), and the locations each subordinator affects. We prove that our calibration approach yields consistent estimates of the model parameters. Then, we introduce a variant of the standard approach to the underlying optimization problem, which leverages partially subordinated Markov chains to solve it quickly and precisely. Finally, we demonstrate the efficacy of our approach using twelve different disruption data sets. Our calibrated parameters are robust, and our optimization algorithm performs better than the simulation-based algorithm. The solutions from our model for disruptions have lower costs than those from other disruption models. Our approach allows for better modeling of disruptions from historical data and can be adapted to other problems in logistics, like the hub location, capacitated facility location, and so on., with joint disruptions. Supplemental Material: The online appendix is available at https://doi.org/10.1287/trsc.2023.0103 .

Parameter estimation of PEM fuel cells using metaheuristic algorithms

Developing an accurate model for precise parameter estimation is crucial for the design of polymer electrolyte membrane fuel cells (PEMFC). While many studies can meet the evaluation criteria for parameter estimation, there is a significant physical inconsistency when applying the Generalized Steady State of Electrochemical Model (GSSEM) prediction in the optimization framework. This study aims to address this inconsistency by adding necessary constraints to the optimization model. It employs the Artificial Hummingbird Algorithm (AHA) and Lévy flight improvement to find unknown parameters by minimizing the sum of squared errors (SSE). The proposed method has been rigorously tested with seven PEMFC cases, demonstrating its robustness and superior accuracy compared to alternative approaches. Notably, the cathode charge transfer coefficient exhibits zero relative percent difference (RPD) between the two sources, confirming its physical consistency. Furthermore, the proposed constrained optimization model lays the groundwork for achieving consistent parameter estimation results for PEMFC when utilizing GSSEM.

Instrumental Variable Method for Regularized Estimation in Generalized Linear Measurement Error Models

Regularized regression methods have attracted much attention in the literature, mainly due to its application in high-dimensional variable selection problems. Most existing regularization methods assume that the predictors are directly observed and precisely measured. It is well known that in a low-dimensional regression model if some covariates are measured with error, then the naive estimators that ignore the measurement error are biased and inconsistent. However, the impact of measurement error in regularized estimation procedures is not clear. For example, it is known that the ordinary least squares estimate of the regression coefficient in a linear model is attenuated towards zero and, on the other hand, the variance of the observed surrogate predictor is inflated. Therefore, it is unclear how the interaction of these two factors affects the selection outcome. To correct for the measurement error effects, some researchers assume that the measurement error covariance matrix is known or can be estimated using external data. In this paper, we propose the regularized instrumental variable method for generalized linear measurement error models. We show that the proposed approach yields a consistent variable selection procedure and root-n consistent parameter estimators. Extensive finite sample simulation studies show that the proposed method performs satisfactorily in both linear and generalized linear models. A real data example is provided to further demonstrate the usage of the method.

Influence of Apparatus Scale on Geogrid Monotonic and Cyclic/Post-Cyclic Pullout Behavior in Cohesive Soils

Geosynthetics have increasingly been applied to geotechnical engineering works due to their numerous advantages, including cost-effectiveness and their significant role in sustainable development. When geosynthetics are used as reinforcement in earth structures, such as embankments, retaining walls and bridge abutments, soil–geosynthetic interface shear behavior is a critical parameter involved in the design. This paper presents a series of monotonic and cyclic/post-cyclic pullout tests carried out to examine the apparatus scale effect on the pullout response of a geogrid embedded in two different soils. To assess the small-scale equipment feasibility, comparisons were made between pullout test parameters derived from small- and large-scale equipment. The test results indicate that, under a low confining stress of 25 kPa, using a smaller-sized apparatus results in lower values of geogrid pullout resistance and maximum mobilized shear stress, but higher values of confined tensile stiffness at low strains. On the other hand, as the confining stress increases (i.e., 50 kPa and 100 kPa), the difference between the results becomes less significant and similar trends are observed regardless of the equipment type. Adopting small-scale equipment enables obtaining soil–reinforcement interaction parameters using test procedures that are less time-consuming than those associated with large-scale pullout tests. However, proper scale effect correction factors may be considered for more consistent estimates of the interface strength parameters under low normal stress values.

A Comparison of Limited Information Estimation Methods for the Two-Parameter Normal-Ogive Model with Locally Dependent Items

The two-parameter normal-ogive (2PNO) model is one of the most popular item response theory (IRT) models for analyzing dichotomous items. Consistent parameter estimation of the 2PNO model using marginal maximum likelihood estimation relies on the local independence assumption. However, the assumption of local independence might be violated in practice. Likelihood-based estimation of the local dependence structure is often computationally demanding. Moreover, many IRT models that model local dependence do not have a marginal interpretation of item parameters. In this article, limited information estimation methods are reviewed that allow the convenient and straightforward handling of local dependence in estimating the 2PNO model. In detail, pairwise likelihood, weighted least squares, and normal-ogive harmonic analysis robust method (NOHARM) estimation are compared with marginal maximum likelihood estimation that ignores local dependence. A simulation study revealed that item parameters can be consistently estimated with limited information methods. At the same time, marginal maximum likelihood estimation resulted in biased item parameter estimates in the presence of local dependence. From a practical perspective, there were only minor differences regarding the statistical quality of item parameter estimates of the different estimation methods. Differences between the estimation methods are also compared for two empirical datasets.

A temporal statistical assessment of the effectiveness of bicyclist safety helmets in mitigating injury severities in vehicle/bicyclist crashes

This study estimates mixed logit models taking into account heterogeneity in means and partially constrained parameters in order to explore possible shifts within parameters over time to study factors influencing bicyclist injury severity outcomes. Separate statistical models are estimated for two bicyclist helmet-wearing scenarios (helmet and non-helmet) using a comprehensive dataset from Florida covering a three-year period to assess COVID-19 effects from the 1st of January 2019 to the 31st of December 2021. This research evaluates several factors influencing helmeted and non-helmeted bicyclist injury severity, encompassing the attributes of drivers and cyclists, the environment and weather, the features of the roads and their temporal aspects, and the different types of vehicles. The performed analysis further enhances model robustness by assessing the temporal stability and transferability across different contexts through likelihood ratio tests, alongside an in-depth examination of the temporal consistency of explanatory variables via marginal effects analysis, confirming significant variations between non-helmeted and helmeted bicyclist models and revealing temporal shifts in factors affecting injury severity during the study period. Findings from the model estimations identify several significant variables with consistent parameter estimates across years. Stop signs, cycling with traffic, and dark, unlit conditions increase severe injury risk in non-helmet models, while the stop sign indicator consistently reduces severe injury risk in helmet models. Statistically significant random parameters are identified across different years and helmet-wearing scenarios, including the male driver indicator, which exhibits varying effects on injury severity. Out-of-sample prediction analysis suggests helmets reduce severe injury probability but may increase minor injuries and decrease no-injury accidents, indicating potential risk compensation behavior among helmeted bicyclists. Although helmets offer protection against severe injuries for bicyclists, it is crucial to adopt a comprehensive safety approach, particularly given the evolving demographics of bicyclists amid the COVID-19 outbreak. This entails considering factors like bicyclist and driver behavior, environmental conditions, and infrastructure enhancements. Policymakers, road safety professionals, and advocacy groups should collaborate to develop holistic strategies to address the determinants of bicycle crash severity outcomes and enhance safety measures for bicyclists across diverse road environments.

On the approximation of high-order binary Markov chains by parsimonious models

Abstract We consider two parsimonious models of binary high-order Markov chains and discover their ability to approximate arbitrary high-order Markov chains. Two types of global measures for approximation accuracy are introduced, theoretical and experimental results are obtained for these measures and for the considered parsimonious models. New consistent statistical parameter estimator is constructed for parsimonious model based on two-layer artificial neural network.

A bias-correction modeling method of Hammerstein–Wiener systems with polynomial nonlinearities using noisy measurements

Modeling of Hammerstein–Wiener nonlinear systems has received a lot of attention in the signal processing community. However, all existing model identification methods may fail to provide a consistent parameter estimate for Errors-In-Variables (EIV) Hammerstein–Wiener systems, where both input–output data are contaminated by measurement white noises. In this paper, a bias-correction Least-Squares (LS) algorithm for consistent identification of EIV Hammerstein–Wiener systems with polynomial nonlinearities using noisy measurements is proposed. Firstly, the analytic expression for the estimated bias of the LS algorithm using noisy measurements for EIV Hammerstein–Wiener systems with polynomial nonlinearities is derived, which is caused by the correlation between the input–output signals and measurement noises. Secondly, a consistent estimation method for the bias-correcting term, including a recursive step and a cross-validation step based on the available noisy measurements only, is then proposed to estimate the unknown terms of noises variances and noise-free measurements in the estimated bias. The effectiveness of the proposed algorithm is demonstrated through a simulated example and a robot arm system.

Pairwise Likelihood Estimation of the 2PL Model with Locally Dependent Item Responses

The local independence assumption is crucial for the consistent estimation of item parameters in item response theory models. This article explores a pairwise likelihood estimation approach for the two-parameter logistic (2PL) model that treats the local dependence structure as a nuisance in the optimization function. Hence, item parameters can be consistently estimated without explicit modeling assumptions of the dependence structure. Two simulation studies demonstrate that the proposed pairwise likelihood estimation approach allows nearly unbiased and consistent item parameter estimation. Our proposed method performs similarly to the marginal maximum likelihood and pairwise likelihood estimation approaches, which also estimate the parameters for the local dependence structure.

A new and unified method for regression analysis of interval-censored failure time data under semiparametric transformation models with missing covariates.

This paper discusses regression analysis of interval-censored failure time data arising from semiparametric transformation models in the presence of missing covariates. Although some methods have been developed for the problem, they either apply only to limited situations or may have some computational issues. Corresponding to these, we propose a new and unified two-step inference procedure that can be easily implemented using the existing or standard software. The proposed method makes use of a set of working models to extract partial information from incomplete observations and yields a consistent estimator of regression parameters assuming missing at random. An extensive simulation study is conducted and indicates that it performs well in practical situations. Finally, we apply the proposed approach to an Alzheimer's Disease study that motivated this study.

Compare the marginal effects for environmental exposure and biomonitoring data with repeated measurements and values below the limit of detection.

Environmental exposure and biomonitoring data with repeated measurements from environmental and occupational studies are commonly right-skewed and in the presence of limits of detection (LOD). However, existing model has not been discussed for small-sample properties and highly skewed data with non-detects and repeated measurements. Marginal modeling provides an alternative to analyzing longitudinal and cluster data, in which the parameter interpretations are with respect to marginal or population-averaged means. We outlined the theories of three marginal models, i.e., generalized estimating equations (GEE), quadratic inference functions (QIF), and generalized method of moments (GMM). With these approaches, we proposed to incorporate the fill-in methods, including single and multiple value imputation techniques, such that any measurements less than the limit of detection are assigned values. We demonstrated that the GEE method works well in terms of estimating the regression parameters in small sample sizes, while the QIF and GMM outperform in large-sample settings, as parameter estimates are consistent and have relatively smaller mean squared error. No specific fill-in method can be deemed superior as each has its own merits. Marginal modeling is firstly employed to analyze repeated measures data with non-detects, in which only the mean structure needs to be correctly provided to obtain consistent parameter estimates. After replacing non-detects through substitution methods and utilizing small-sample bias corrections, in a simulation study we found that the estimating approaches used in the marginal models have corresponding advantages under a wide range of sample sizes. We also applied the models to longitudinal and cluster working examples.

Parametric modal regression with error in covariates.

An inference procedure is proposed to provide consistent estimators of parameters in a modal regression model with a covariate prone to measurement error. A score-based diagnostic tool exploiting parametric bootstrap is developed to assess adequacy of parametric assumptions imposed on the regression model. The proposed estimation method and diagnostic tool are applied to synthetic data generated from simulation experiments and data from real-world applications to demonstrate their implementation and performance. These empirical examples illustrate the importance of adequately accounting for measurement error in the error-prone covariate when inferring the association between a response and covariates based on a modal regression model that is especially suitable for skewed and heavy-tailed responsedata.