Inference Of Model Parameters Research Articles

Tricks with metrics: combining statistics for improved inference in regression analysis

. In maximum likelihood methods, the three classical tests statistics are often unreliable for inference in small samples, even under the correct model specification. In this article, I discuss how likelihood ratio and Wald test statistics can be combined to obtain highly improved parameter inference in regression models with small samples. I consider modifications obtained from both the Barndorff-Nielsen and the Lugannani and Rice likelihood approximations, and I show how they can produce highly accurate parameter inference in a general (possibly non linear) regression model with possibly non spherical disturbances. I discuss the underlying theory and provide Monte Carlo simulations demonstrating the superior accuracy of the proposed procedures over the first-order classical likelihood methods (i.e., the signed log-likelihood ratio test and the Wald test). An empirical application to a regression model of mobile money (“M-pesa”) adoption in Kenya is provided as an illustration of the usefulness of these methods in practice.

Enhanced network inference from sparse incomplete time series through automatically adapted L1\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$L_1$$\\end{document} regularization

Reconstructing dynamics of complex systems from sparse, incomplete time series data is a challenging problem with applications in various domains. Here, we develop an iterative heuristic method to infer the underlying network structure and parameters governed by Ising dynamics from incomplete spin configurations based on sparse and small-sized samples. Our method iterates between imputing missing spin states given current coupling strengths and re-estimating couplings from completed spin state data. Central to our approach is the novel application of adaptive l1\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$l_1$$\\end{document} regularization on updating coupling strengths, which features an automatic adjustment of the regularization strength throughout the iterative inference process. By doing so, we aim at preventing over-fitting and enforcing the sparsity of couplings without access to ground truth parameters. We demonstrate that this approach accurately recovers parameters and imputes missing spins even with substantial missing data and short time series, providing improvements in the inference of Ising model parameters even for relatively small sample sizes.

Limited information-processing capacity in vision explains number psychophysics.

Humans and other animals are able to perceive and represent a number of objects present in a scene, a core cognitive ability thought to underlie the development of mathematics. However, the perceptual mechanisms that underpin this capacity remain poorly understood. Here, we show that our visual sense of number derives from a visual system designed to efficiently encode the location of objects in scenes. Using a mathematical model, we demonstrate that an efficient but information-limited encoding of objects' locations can explain many key aspects of number psychophysics, including subitizing, Weber's law, underestimation, and effects of exposure time. In two experiments (N = 100 each), we find that this model of visual encoding captures human performance in both a change-localization task and a number estimation task. In a third experiment (N = 100), we find that individual differences in change-localization performance are highly predictive of differences in number estimation, both in terms of overall performance and inferred model parameters, with participants having numerically indistinguishable inferred information capacities across tasks. Our results therefore indicate that key psychophysical features of numerical cognition do not arise from separate modules or capacities specific to number, but rather as by-products of lower level constraints on perception. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Increasing the power of weak lensing data with multipole-based intrinsic alignment estimators

Abstract It has long been known that galaxy shapes align coherently with the large-scale density field. Characterizing this effect is essential to interpreting measurements of weak gravitational lensing, the deflection of light from distant galaxies by matter overdensities along the line of sight, as it also produces coherent galaxy alignments that we wish to interpret in terms of a cosmological model. Existing direct measurements of intrinsic alignments using galaxy samples with high-quality shape and redshift measurements typically use well-understood but sub-optimal projected estimators, which do not make good use of the information in the data when comparing those estimators to theoretical models. We demonstrate a more optimal estimator, based on a multipole expansion of the correlation functions or power spectra, for direct measurements of galaxy intrinsic alignments. We show that even using the lowest order multipole alone increases the significance of inferred model parameters using simulated and real data, without any additional modeling complexity. We apply this estimator to measurements of parameters of the non-linear alignment model using data from the Sloan Digital Sky survey, demonstrating consistent results with a factor of ∼2 greater precision in parameter fits to intrinsic alignments models. This result is functionally equivalent to quadrupling the survey area, but without the attendant costs – thereby demonstrating the value in using this new estimator in current and future intrinsic alignments measurements using spectroscopic galaxy samples.

Understanding the impact of numerical solvers on inference for differential equation models.

Most ordinary differential equation (ODE) models used to describe biological or physical systems must be solved approximately using numerical methods. Perniciously, even those solvers that seem sufficiently accurate for the forward problem, i.e. for obtaining an accurate simulation, might not be sufficiently accurate for the inverse problem, i.e. for inferring the model parameters from data. We show that for both fixed step and adaptive step ODE solvers, solving the forward problem with insufficient accuracy can distort likelihood surfaces, which might become jagged, causing inference algorithms to get stuck in local 'phantom' optima. We demonstrate that biases in inference arising from numerical approximation of ODEs are potentially most severe in systems involving low noise and rapid nonlinear dynamics. We reanalyse an ODE changepoint model previously fit to the COVID-19 outbreak in Germany and show the effect of the step size on simulation and inference results. We then fit a more complicated rainfallrun-off model to hydrological data and illustrate the importance of tuning solver tolerances to avoid distorted likelihood surfaces. Our results indicate that, when performing inference for ODE model parameters, adaptive step size solver tolerances must be set cautiously and likelihood surfaces should be inspected for characteristic signs of numerical issues.

Parameter estimation and forecasting with quantified uncertainty for ordinary differential equation models using QuantDiffForecast: A MATLAB toolbox and tutorial.

Mathematical models based on systems of ordinary differential equations (ODEs) are frequently applied in various scientific fields to assess hypotheses, estimate key model parameters, and generate predictions about the system's state. To support their application, we present a comprehensive, easy-to-use, and flexible MATLAB toolbox, QuantDiffForecast, and associated tutorial to estimate parameters and generate short-term forecasts with quantified uncertainty from dynamical models based on systems of ODEs. We provide software ( https://github.com/gchowell/paramEstimation_forecasting_ODEmodels/) and detailed guidance on estimating parameters and forecasting time-series trajectories that are characterized using ODEs with quantified uncertainty through a parametric bootstrapping approach. It includes functions that allow the user to infer model parameters and assess forecasting performance for different ODE models specified by the user, using different estimation methods and error structures in the data. The tutorial is intended for a diverse audience, including students training in dynamic systems, and will be broadly applicable to estimate parameters and generate forecasts from models based on ODEs. The functions included in the toolbox are illustrated using epidemic models with varying levels of complexity applied to data from the 1918 influenza pandemic in San Francisco. A tutorial video that demonstrates the functionality of the toolbox is included.

Bayesian model calibration for block copolymer self-assembly: Likelihood-free inference and expected information gain computation via measure transport

We consider the Bayesian calibration of models describing the phenomenon of block copolymer (BCP) self-assembly, which is to infer model parameters and their uncertainty from noisy image data produced by microscopy or X-ray scattering characterization of BCP structures. To account for the long-range disorder in BCP structures, we introduce auxiliary variables in the model to represent this aleatory uncertainty. These variables, however, result in an integrated likelihood function for high-dimensional image data that is generally intractable to evaluate. We tackle this Bayesian inference problem using a likelihood-free inference (LFI) approach based on measure transport together with the construction of summary statistics for the image data. We show that expected information gains (EIG) from observed data about the model parameters can be computed with no significant additional cost. Lastly, we present a numerical case study using the Ohta–Kawasaki model for diblock copolymer thin film self-assembly and top-down microscopy characterization. We introduce several domain-specific energy and Fourier-based summary statistics for calibration and quantify their informativeness using EIG. The effect of various data corruptions, summary statistics, and experimental designs on the calibration results are studied using the proposed LFI method.

Estimation of Intensity‐Duration‐Area‐Frequency Relationships Based on the Full Range of Non‐Zero Precipitation From Radar‐Reanalysis Data

AbstractIntensity‐Duration‐Area‐Frequency (IDAF) models provide the mathematical link between precipitation intensities (I), durations (D), areas (A), and frequency of occurrence (F). They play a critical role in hydrological design, areal rainfall hazard quantification, storm characterization, and early warning system development. IDAF models extend the conventional Intensity‐Duration‐Frequency models by accounting for the spatial extent of precipitation (i.e., the area). In this study, we develop IDAF models using the entire non‐zero precipitation intensities, not only the extremes. We use the extended generalized Pareto distribution (EGPD) to model the precipitation intensities. To build the IDAF models, we adopt a data‐driven approach that allows the linkage of EGPD parameters with duration and area, based on empirically determined parametric relationships. The inference of model parameters is done using a global maximum likelihood estimation, and uncertainties are assessed by the bootstrap method. The study area is Switzerland, a topographically complex region of 42,000 km2 with regional precipitation variability and clear seasonality. The study utilizes 17 years of data from CombiPrecip, a radar‐reanalysis product developed by geostatistically merging radar and rain gauge data in an operational setting. We build the IDAF models for the spatiotemporal range of 1–72 hr and 1 to 1,089 km2 at each pixel in the study area. To the best of our knowledge, our study is the first attempt to use the EGPD in IDAF curve modeling. It discusses the use and limitations of CombiPrecip in extreme value analysis and highlights the challenges of modeling areal precipitation in a complex topographical environment.

Online Statistical Inference for Stochastic Optimization via Kiefer-Wolfowitz Methods

This article investigates the problem of online statistical inference of model parameters in stochastic optimization problems via the Kiefer-Wolfowitz algorithm with random search directions. We first present the asymptotic distribution for the Polyak-Ruppert-averaging type Kiefer-Wolfowitz (AKW) estimators, whose asymptotic covariance matrices depend on the distribution of search directions and the function-value query complexity. The distributional result reflects the tradeoff between statistical efficiency and function query complexity. We further analyze the choice of random search directions to minimize certain summary statistics of the asymptotic covariance matrix. Based on the asymptotic distribution, we conduct online statistical inference by providing two construction procedures of valid confidence intervals. Supplementary materials for this article are available online.

Restricted Latent Class Models for Nominal Response Data: Identifiability and Estimation.

Restricted latent class models (RLCMs) provide an important framework for diagnosing and classifying respondents on a collection of multivariate binary responses. Recent research made significant advances in theory for establishing identifiability conditions for RLCMs with binary and polytomous response data. Multiclass data, which are unordered nominal response data, are also widely collected in the social sciences and psychometrics via forced-choice inventories and multiple choice tests. We establish new identifiability conditions for parameters of RLCMs for multiclass data and discuss the implications for substantive applications. The new identifiability conditions are applicable to a wealth of RLCMs for polytomous and nominal response data. We propose a Bayesian framework for inferring model parameters, assess parameter recovery in a Monte Carlo simulation study, and present an application of the model to a real dataset.

4D nucleome equation predicts gene expression controlled by long-range enhancer-promoter interaction.

Recent experimental evidence strongly supports that three-dimensional (3D) long-range enhancer-promoter (E-P) interactions have important influences on gene-expression dynamics, but it is unclear how the interaction information is translated into gene expression over time (4D). To address this question, we developed a general theoretical framework (named as a 4D nucleome equation), which integrates E-P interactions on chromatin and biochemical reactions of gene transcription. With this equation, we first present the distribution of mRNA counts as a function of the E-P genomic distance and then reveal a power-law scaling of the expression level in this distance. Interestingly, we find that long-range E-P interactions can induce bimodal and trimodal mRNA distributions. The 4D nucleome equation also allows for model selection and parameter inference. When this equation is applied to the mouse embryonic stem cell smRNA-FISH data and the E-P genomic-distance data, the predicted E-P contact probability and mRNA distribution are in good agreement with experimental results. Further statistical inference indicates that the E-P interactions prefer to modulate the mRNA level by controlling promoter activation and transcription initiation rates. Our model and results provide quantitative insights into both spatiotemporal gene-expression determinants (i.e., long-range E-P interactions) and cellular fates during development.

Improving estimates of waning immunity rates in stochastic SIRS models with a hierarchical framework

As most disease causing pathogens require transmission from an infectious individual to a susceptible individual, continued persistence of the pathogen within the population requires the replenishment of susceptibles through births, immigration, or waning immunity.Consider the introduction of an unknown infectious disease into a fully susceptible population where it is not known how long immunity is conferred once an individual recovers from infection. If, initially, the prevalence of disease increases (that is, the infection takes off), the number of infectives will usually decrease to a low level after the first major outbreak. During this post-outbreak period, the disease dynamics may be influenced by stochastic effects and there is a non-zero probability that the epidemic will die out. Die out in this period following the first major outbreak is known as an epidemic fade-out. If the disease does not die out, the susceptible population may be replenished by the waning of immunity, and a second wave may start.In this study, we investigate if the rate of waning immunity (and other epidemiological parameters) can be reliably estimated from multiple outbreak data, in which some outbreaks display epidemic fade-out and others do not. We generated synthetic outbreak data from independent simulations of stochastic SIRS models in multiple communities. Some outbreaks faded-out and some did not. We conducted Bayesian parameter estimation under two alternative approaches: independently on each outbreak and under a hierarchical framework. When conducting independent estimation, the waning immunity rate was poorly estimated and biased towards zero when an epidemic fade-out was observed. However, under a hierarchical approach, we obtained more accurate and precise posterior estimates for the rate of waning immunity and other epidemiological parameters. The greatest improvement in estimates was obtained for those communities in which epidemic fade-out was observed.Our findings demonstrate the feasibility and value of adopting a Bayesian hierarchical approach for parameter inference for stochastic epidemic models.

Free-field characterization of locally reacting sound absorbers using Bayesian inference with sequential frequency transfer

The in situ absorption characteristics of sound-absorbing samples are generally estimated inversely from sound field measurements above the sample’s surface. However, the entire measurement process is subject to numerous sources of uncertainty, which cannot be accounted for using deterministic inference methods. This paper introduces a Bayesian framework, including sequential frequency transfer to infer point estimates and uncertainty ranges of the frequency-dependent surface admittance, the corresponding sound absorption coefficient and the associated measurement and model uncertainty. The inference is conducted in two phases: in the initial phase, the absorption characteristics and the measurement and model uncertainty at a predefined initial frequency are inferred based on a weakly informative prior. Extensive prior predictive checks are conducted to find the best trade-off between a weak level of informativeness of the prior and physically meaningful predictions of the sound field above the absorber. In the sequential phase, the posterior mean at an analyzed frequency serves as the prior mean at the subsequent frequency. The framework is applied to simulated as well as real free-field measurements of the specific impedance above two different rock wool samples. The measurements are conducted with a combined pressure-particle velocity probe and the samples are assumed to exhibit local reaction behavior. For the investigated samples, a minimum number of four individual measurements of the specific impedance already allows for an accurate estimation of the frequency-dependent absorption characteristics and the associated measurement and model uncertainty, particularly for frequencies above 600 Hz. Due to the proposed sequential frequency transfer, the computational effort of the sampling-based Bayesian inference at a discrete frequency is significantly reduced. This paves the way for efficient inference of frequency-dependent model parameters while overcoming the curse of dimensionality associated with the inference of stochastic processes. The proposed framework lays the foundation for accurate and efficient inference of frequency-dependent parameters in models beyond the field of acoustics.

A Class of Hierarchical Multivariate Wiener Processes for Modeling Dependent Degradation Data

In engineering practice, many products exhibit multiple and dependent degrading performance characteristics (PCs). It is common to observe that these PCs’ initial measurements are nonconstant and sometimes correlated with the subsequent degradation rate, which typically varies from one unit to another. To accommodate the unit-wise heterogeneity, PC-wise dependency, and “initiation-growth” correlation, this article proposes a broad class of multi-dimensional degradation models under a framework of hierarchical multivariate Wiener processes. These models incorporate dual multi-normally distributed random effects concerning the initial values and degradation rates. To infer model parameters, expectation-maximization (EM) algorithms and several tools for model validation and selection are developed. Various simulation studies are carried out to assess the performance of the inference method and to compare different models. Two case studies are conducted to demonstrate the applicability of the proposed methodology. The online supplementary materials of this article contain derivations of EM estimators, additional numerical results, and R codes.

Reduction of apparent temporal variations of tidal parameters by a proper local response model

We describe a new harmonic tidal analysis method, which constrains the solution to be near a reference model. This regularization stabilizes the linear regression, allowing us to infer model parameters for each tidal harmonic. This overcomes the need to create a priori groupings of harmonics. The inversion is done iteratively by adjusting the reference model to reduce the data misfit. The frequency dependence of the solution is thus data-driven. We find models for the different spherical degrees independently. Our procedure allows narrow-band variations of the tidal admittance. We test the hypothesis that some of the temporal variations of tidal parameters found in previous studies were caused by inappropriate body tide models in combination with a priori wave grouping. We determine a local response model from 11.5 years of data recorded by the superconducting gravimeter SG056 at Black Forest Observatory (BFO, Schiltach). Using this as an a priori model in a non-regularized moving window analysis of wave groups composed from summed harmonics, we find that periodic variations of groups M1, K1, mu 2, N2, L2, and S2 are reduced by up to a factor of 7 compared to earlier studies. Some variations previously seen in the M2 group are captured as well.

Bayesian model calibration for diblock copolymer thin film self-assembly using power spectrum of microscopy data and machine learning surrogate

Identifying parameters of computational models from experimental data, or model calibration, is fundamental for assessing and improving the predictability and reliability of computer simulations. In this work, we propose a method for Bayesian calibration of models that predict morphological patterns of diblock copolymer (Di-BCP) thin film self-assembly while accounting for various sources of uncertainties in pattern formation and data acquisition. This method extracts the azimuthally-averaged power spectrum (AAPS) of the top-down microscopy characterization of Di-BCP thin film patterns as summary statistics for Bayesian inference of model parameters via the pseudo-marginal method. We derive the analytical and approximate form of a conditional likelihood for the AAPS of image data. We demonstrate that AAPS-based image data reduction retains the mutual information, particularly on important length scales, between image data and model parameters while being relatively agnostic to the aleatoric uncertainties associated with the random long-range disorder of Di-BCP patterns. Additionally, we propose a phase-informed prior distribution for Bayesian model calibration. Furthermore, reducing image data to AAPS enables us to efficiently build surrogate models to accelerate the proposed Bayesian model calibration procedure. We present the formulation and training of two multi-layer perceptrons for approximating the parameter-to-spectrum map, which enables fast integrated likelihood evaluations. We validate the proposed Bayesian model calibration method through numerical examples, for which the neural network surrogate delivers a fivefold reduction of the number of model simulations performed for a single calibration task.

Large deviations for the Pearson family of ergodic diffusion processes involving a quadratic diffusion coefficient and a linear force

The Pearson family of ergodic diffusions with a quadratic diffusion coefficient and a linear force is characterized by explicit dynamics of their integer moments and by explicit relaxation of spectral properties towards their steady state. Besides the Ornstein–Uhlenbeck process with a Gaussian steady state, other representative examples of the Pearson family are the square root or the Cox–Ingersoll–Ross process converging towards the gamma distribution, the Jacobi process converging towards the beta distribution, the reciprocal gamma process (corresponding to an exponential functional of the Brownian motion) that converges towards the inverse-gamma distribution, the Fisher–Snedecor process and the Student process. The last three steady states display heavy tails. The goal of the present paper is to analyze the large deviation properties of these various diffusion processes in a unified framework. We first consider level 1 concerning time-averaged observables over a large time window T. We write the first rescaled cumulants for generic observables and identify specific observables whose large deviations can be explicitly computed from the dominant eigenvalue of the appropriate deformed generator. The explicit large deviations at level 2 concerning the time-averaged density are then used to analyze the statistical inference of model parameters from data on a very long stochastic trajectory in order to obtain the explicit rate function for the two inferred parameters of the Pearson linear force.

A step towards digital operations—A novel grey-box approach for modelling the heat dynamics of ultra-low temperature freezing chambers

Ultra-low temperature (ULT) freezers store perishable bio-contents and have high energy consumption, which highlight a demand for reliable methods for intelligent surveillance and smart energy management. This study introduces a novel grey-box modelling approach based on stochastic differential equations to describe the heat dynamics of the ULT freezing chambers. The proposed modelling approach only requires temperature data measured by the embedded sensors and uses data from the regular operation periods for model identification. The model encompasses three states: chamber temperature, envelope temperature, and local evaporator temperature. Special attention is given to the local evaporator temperature state, which is modelled as a time-variant system, to characterise the time delay and dynamic variations in cooling intensity.The model has three states, of which a time-variant model with nonlinear input for the local evaporator temperature state is specifically established to adapt to the variation of the cooling intensity at the position of the embedded chamber control probe. Two ULT freezers with different operational patterns are modelled. The unknown model parameters are estimated using the maximum likelihood method. The results demonstrate that the models can accurately predict the chamber temperature measured by the control probe (RMSE <0.19°C) and are promising to be applied for forecasting future states. In addition, the model for local evaporator temperature can effectively adapt to different operational patterns and provide insight into the local evaporation cooling supply status. The proposed approach greatly promotes the practical feasibility of grey-box modelling of the heat dynamics for ULT freezers and is a step forward in future digital operationscan serve several potential digital applications. A major limitation of the modelling approach is the low identifiability, which can potentially be addressed by inferring model parameters based on relative parameter changes.

Unlabelled landmark matching via Bayesian data selection, and application to cell matching across imaging modalities

We consider the problem of landmark matching between two unlabelled point sets, in particular where the number of points in each cloud may differ, and where points in each cloud may not have a corresponding match. We invoke a Bayesian framework to identify the transformation of coordinates that maps one cloud to the other, alongside correspondence of the points. This problem necessitates a novel methodology for Bayesian data selection, simultaneous inference of model parameters, and selection of the data which leads to the best fit of the model to the majority of the data. We apply this to a problem in developmental biology where the landmarks correspond to segmented cell centres, where potential death or division of cells can lead to discrepancies between the point-sets from each image. We validate the efficacy of our approach using in silico tests and a microinjected fluorescent marker experiment. Subsequently we apply our approach to the matching of cells between real time imaging and immunostaining experiments, facilitating the combination of single-cell data between imaging modalities. Furthermore our approach to Bayesian data selection is broadly applicable across data science, and has the potential to change the way we think about fitting models to data.

Topic-Clustering Model with Temporal Distribution for Public Opinion Topic Analysis of Geospatial Social Media Data

Analysis of the spatiotemporal distribution of online public opinion topics can help understand the hotspots of public concern. The topic model is employed widely in public opinion topic clustering for social media data. In order to handle topic-clustering of low-quality geospatial social media data, such as microblog data, with short text and timeliness characteristics, this study proposed a Dirichlet multinomial mixture over time (DMMOT) model to cluster microblog topic for public opinion analysis. The DMMOT model assumes that a single document belongs to a single topic, in line with the characteristics of a short text, and it introduces the probability distribution of “topic-time” in the process of topic generation. The model parameter inference process was presented in detail by exploring the Gibbs sampling method. Results generated using the DMMOT model in case study show that the “topic-word” distribution is semantically aggregated within various topics, and “topic-time” distribution clustered within a time window under each topic. Furthermore, the characteristics of the trend of each topic over time are basically consistent with the corresponding trend of topic in reality in terms of content. These indicate that the DMMOT model improves topic clustering for short text to some extent. Furthermore, the DMMOT model performed well in both temporal and spatial analysis of public opinion topics based on microblog data.