Problem Of Parameter Inference Research Articles

FitMultiCell: simulating and parameterizing computational models of multi-scale and multi-cellular processes.

Biological tissues are dynamic and highly organized. Multi-scale models are helpful tools to analyse and understand the processes determining tissue dynamics. These models usually depend on parameters that need to be inferred from experimental data to achieve a quantitative understanding, to predict the response to perturbations, and to evaluate competing hypotheses. However, even advanced inference approaches such as approximate Bayesian computation (ABC) are difficult to apply due to the computational complexity of the simulation of multi-scale models. Thus, there is a need for a scalable pipeline for modeling, simulating, and parameterizing multi-scale models of multi-cellular processes. Here, we present FitMultiCell, a computationally efficient and user-friendly open-source pipeline that can handle the full workflow of modeling, simulating, and parameterizing for multi-scale models of multi-cellular processes. The pipeline is modular and integrates the modeling and simulation tool Morpheus and the statistical inference tool pyABC. The easy integration of high-performance infrastructure allows to scale to computationally expensive problems. The introduction of a novel standard for the formulation of parameter inference problems for multi-scale models additionally ensures reproducibility and reusability. By applying the pipeline to multiple biological problems, we demonstrate its broad applicability, which will benefit in particular image-based systems biology. FitMultiCell is available open-source at https://gitlab.com/fitmulticell/fit.

Cooperative Identification of Multi-Agent Systems in Presence of Integral Action: Insights from a Two-Agent Framework

This article addresses the parameter inference problem on dynamical systems in a cooperative setting. The dynamical systems contain integral action, which is of practical interest within cyber-physical systems. Given this dynamical characteristic of the problem, existing statistical results on system Identification are not straightforwardly applicable and hence adaptations are required. To this end, a formulation of the problem is presented and results highlighting the challenges of the given setup are discussed. Insights from a two-agent framework are used to formulate results and the next research steps on the topic, where the benefit of performing open-loop Identification on the variance of the parameter is highlighted and bias issues are discussed.

Accelerating multimodal gravitational waveforms from precessing compact binaries with artificial neural networks

Gravitational waves from the coalescences of black hole and neutron stars afford us the unique opportunity to determine the sources' properties, such as their masses and spins, with unprecedented accuracy. To do so, however, theoretical models of the emitted signal that are i) extremely accurate and ii) computationally highly efficient are necessary. The inclusion of more detailed physics such as higher-order multipoles and relativistic spin-induced orbital precession increases the complexity and hence also computational cost of waveform models, which presents a severe bottleneck to the parameter inference problem. A popular method to generate waveforms more efficiently is to build a fast surrogate model of a slower one. In this paper, we show that traditional surrogate modelling methods combined with artificial neural networks can be used to build a computationally highly efficient while still accurate emulation of multipolar time-domain waveform models of precessing binary black holes. We apply this method to the state-of-the-art waveform model SEOBNRv4PHM and find significant computational improvements: On a traditional CPU, the typical generation of a single waveform using our neural network surrogate SEOBNN_v4PHM_4dq2 takes 18ms for a binary black hole with a total mass of $44 M_{\odot}$ when generated from 20Hz. In comparison to SEOBNRv4PHM itself, this amounts to an improvement in computational efficiency by two orders of magnitude. Utilising additional GPU acceleration, we find that this speed-up can be increased further with the generation of batches of waveforms simultaneously. Even without additional GPU acceleration, this dramatic decrease in waveform generation cost can reduce the inference timescale from weeks to hours.

Understanding the romanization spreading on historical interregional networks in Northern Tunisia

Spreading processes are important drivers of change in social systems. To understand the mechanisms of spreading it is fundamental to have information about the underlying contact network and the dynamical parameters of the process. However, in many real-wold examples, this information is not known and needs to be inferred from data. State-of-the-art spreading inference methods have mostly been applied to modern social systems, as they rely on availability of very detailed data. In this paper we study the inference challenges for historical spreading processes, for which only very fragmented information is available. To cope with this problem, we extend existing network models by formulating a model on a mesoscale with temporal spreading rate. Furthermore, we formulate the respective parameter inference problem for the extended model. We apply our approach to the romanization process of Northern Tunisia, a scarce dataset, and study properties of the inferred time-evolving interregional networks. As a result, we show that (1) optimal solutions consist of very different network structures and spreading rate functions; and that (2) these diverse solutions produce very similar spreading patterns. Finally, we discuss how inferred dominant interregional connections are related to available archaeological traces. Historical networks resulting from our approach can help understanding complex processes of cultural change in ancient times.

ISALT: Inference-based schemes adaptive to large time-stepping for locally Lipschitz ergodic systems

<p style='text-indent:20px;'>Efficient simulation of SDEs is essential in many applications, particularly for ergodic systems that demand efficient simulation of both short-time dynamics and large-time statistics. However, locally Lipschitz SDEs often require special treatments such as implicit schemes with small time-steps to accurately simulate the ergodic measures. We introduce a framework to construct inference-based schemes adaptive to large time-steps (ISALT) from data, achieving a reduction in time by several orders of magnitudes. The key is the statistical learning of an approximation to the infinite-dimensional discrete-time flow map. We explore the use of numerical schemes (such as the Euler-Maruyama, the hybrid RK4, and an implicit scheme) to derive informed basis functions, leading to a parameter inference problem. We introduce a scalable algorithm to estimate the parameters by least squares, and we prove the convergence of the estimators as data size increases.</p><p style='text-indent:20px;'>We test the ISALT on three non-globally Lipschitz SDEs: the 1D double-well potential, a 2D multiscale gradient system, and the 3D stochastic Lorenz equation with a degenerate noise. Numerical results show that ISALT can tolerate time-step magnitudes larger than plain numerical schemes. It reaches optimal accuracy in reproducing the invariant measure when the time-step is medium-large.</p>

Cosmological parameter estimation and inference using deep summaries

The ability to obtain reliable point estimates of model parameters is of crucial importance in many fields of physics. This is often a difficult task given that the observed data can have a very high number of dimensions. In order to address this problem, we propose a novel approach to construct parameter estimators with a quantifiable bias using an order expansion of highly compressed deep summary statistics of the observed data. These summary statistics are learned automatically using an information maximising loss. Given an observation, we further show how one can use the constructed estimators to obtain approximate Bayes computation (ABC) posterior estimates and their corresponding uncertainties that can be used for parameter inference using Gaussian process regression even if the likelihood is not tractable. We validate our method with an application to the problem of cosmological parameter inference of weak lensing mass maps. We show in that case that the constructed estimators are unbiased and have an almost optimal variance, while the posterior distribution obtained with the Gaussian process regression is close to the true posterior and performs better or equally well than comparable methods.

Generative adversarial networks for construction of virtual populations of mechanistic models: simulations to study Omecamtiv Mecarbil action

Biophysical models are increasingly used to gain mechanistic insights by fitting and reproducing experimental and clinical data. The inherent variability in the recorded datasets, however, presents a key challenge. In this study, we present a novel approach, which integrates mechanistic modeling and machine learning to analyze in vitro cardiac mechanics data and solve the inverse problem of model parameter inference. We designed a novel generative adversarial network (GAN) and employed it to construct virtual populations of cardiac ventricular myocyte models in order to study the action of Omecamtiv Mecarbil (OM), a positive cardiac inotrope. Populations of models were calibrated from mechanically unloaded myocyte shortening recordings obtained in experiments on rat myocytes in the presence and absence of OM. The GAN was able to infer model parameters while incorporating prior information about which model parameters OM targets. The generated populations of models reproduced variations in myocyte contraction recorded during in vitro experiments and provided improved understanding of OM’s mechanism of action. Inverse mapping of the experimental data using our approach suggests a novel action of OM, whereby it modifies interactions between myosin and tropomyosin proteins. To validate our approach, the inferred model parameters were used to replicate other in vitro experimental protocols, such as skinned preparations demonstrating an increase in calcium sensitivity and a decrease in the Hill coefficient of the force–calcium (F–Ca) curve under OM action. Our approach thereby facilitated the identification of the mechanistic underpinnings of experimental observations and the exploration of different hypotheses regarding variability in this complex biological system.

An improved approximate Bayesian computation scheme for parameter inference based on a recalibration post-processing method

Approximate Bayesian Computation algorithms (ABC), bypassing the intractable likelihood function, are commonly used to estimate the posterior distributions of parameters in practice. However ABC algorithms also have some inherent problems in the application, for example, when the algorithm maintains operating efficiency, it is difficult to obtain a more accurate approximation of the target distribution. To overcome these defects, an improved ABC algorithm is introduced in this article. The novel algorithm based on the ABC Sequential Monte Carlo (ABC-SMC) and a new recalibration post-processing methods is obtained to estimate the parameters of the biodynamic models, which is also called the ABC-SMCR algorithm. Through this article, it will be shown that the new algorithm is promising in processing the parameter inference problem. This algorithm not only offers high computational efficiency but also improves the quality estimate of the posterior distributions. To demonstrate its strengths, two examples are given. The first illustrative example on the basis of simulated data mainly demonstrates that it can be used to obtain accurate parameter inference of the Lotka-Volterra model. The second illustration based on actual flu outbreak data are given to show the computational efficiency of this algorithm in parameter estimation of the epidemic model.

Surrogate-based sequential Bayesian experimental design using non-stationary Gaussian Processes

Inferring arbitrary quantities of interest (QoI) using limited computational or, in realistic scenarios, financial budgets, is a challenging problem that requires sophisticated strategies for the optimal allocation of the available resources. Bayesian optimal experimental design identifies the optimal set of design locations for the purpose of solving a parameter inference problem and the optimality criterion is typically associated with maximizing the worth of information in the experimental measurements. Sequential design strategies further identify the optimal design in a sequential manner, starting from a initial budget and iteratively selecting new optimal points until either an accuracy threshold is reached, or a cost limit is exceeded. In this paper, we present a generic sequential Bayesian experimental design framework that relies on maximizing an information theoretic design criterion, namely the Expected Information Gain, in order to infer QoIs formed as nonlinear operators acting on black-box functions. Our framework relies on modeling the underlying response function using non-stationary Gaussian Processes, thus enabling efficient sampling from the QoI in order to provide Monte Carlo estimators for the design criterion. We demonstrate the performance of our method on an engineering problem of steel wire manufacturing and compare it with two classic approaches: uncertainty sampling and expected improvement.

Inference of protoneutron star properties from gravitational-wave data in core-collapse supernovae

The eventual detection of gravitational waves from core-collapse supernovae (CCSN) will help improve our current understanding of the explosion mechanism of massive stars. The stochastic nature of the late post-bounce gravitational wave signal due to the non-linear dynamics of the matter involved and the large number of degrees of freedom of the phenomenon make the source parameter inference problem very challenging. In this paper we take a step towards that goal and present a parameter estimation approach which is based on the gravitational waves associated with oscillations of proto-neutron stars (PNS). Numerical simulations of CCSN have shown that buoyancy-driven g-modes are responsible for a significant fraction of the gravitational wave signal and their time-frequency evolution is linked to the physical properties of the compact remnant through universal relations, as demonstrated in [1]. We use a set of 1D CCSN simulations to build a model that relates the evolution of the PNS properties with the frequency of the dominant g-mode, which is extracted from the gravitational-wave data using a new algorithm we have developed for our study. The model is used to infer the time evolution of a combination of the mass and the radius of the PNS. The performance of the method is estimated employing simulations of 2D CCSN waveforms covering a progenitor mass range between 11 and 40 solar masses and different equations of state. Considering signals embedded in Gaussian gravitational wave detector noise, we show that it is possible to infer PNS properties for a galactic source using Advanced LIGO and Advanced Virgo data at design sensitivities. Third generation detectors such as Einstein Telescope and Cosmic Explorer will allow to test distances of ${\cal O}(100\, {\rm kpc})$.

A new approach to observational cosmology using the scattering transform

ABSTRACT Parameter estimation with non-Gaussian stochastic fields is a common challenge in astrophysics and cosmology. In this paper, we advocate performing this task using the scattering transform, a statistical tool sharing ideas with convolutional neural networks (CNNs) but requiring neither training nor tuning. It generates a compact set of coefficients, which can be used as robust summary statistics for non-Gaussian information. It is especially suited for fields presenting localized structures and hierarchical clustering, such as the cosmological density field. To demonstrate its power, we apply this estimator to a cosmological parameter inference problem in the context of weak lensing. On simulated convergence maps with realistic noise, the scattering transform outperforms classic estimators and is on a par with the state-of-the-art CNN. It retains advantages of traditional statistical descriptors, has provable stability properties, allows to check for systematics, and importantly, the scattering coefficients are interpretable. It is a powerful and attractive estimator for observational cosmology and the study of physical fields in general.

Identifiability of a Binomial Synapse.

Synapses are highly stochastic transmission units. A classical model describing this stochastic transmission is called the binomial model, and its underlying parameters can be estimated from postsynaptic responses to evoked stimuli. The accuracy of parameter estimates obtained via such a model-based approach depends on the identifiability of the model. A model is said to be structurally identifiable if its parameters can be uniquely inferred from the distribution of its outputs. However, this theoretical property does not necessarily imply practical identifiability. For instance, if the number of observations is low or if the recording noise is high, the model's parameters can only be loosely estimated. Structural identifiability, which is an intrinsic property of a model, has been widely characterized; but practical identifiability, which is a property of both the model and the experimental protocol, is usually only qualitatively assessed. Here, we propose a formal definition for the practical identifiability domain of a statistical model. For a given experimental protocol, this domain corresponds to the set of parameters for which the model is correctly identified as the ground truth compared to a simpler alternative model. Considering a model selection problem instead of a parameter inference problem allows to derive a non-arbitrary criterion for practical identifiability. We apply our definition to the study of neurotransmitter release at a chemical synapse. Our contribution to the analysis of synaptic stochasticity is three-fold: firstly, we propose a quantitative criterion for the practical identifiability of a statistical model, and compute the identifiability domains of different variants of the binomial release model (uni or multi-quantal, with or without short-term plasticity); secondly, we extend the Bayesian Information Criterion (BIC), a classically used tool for model selection, to models with correlated data (which is the case for most models of chemical synapses); finally, we show that our approach allows to perform data free model selection, i.e., to verify if a model used to fit data was indeed identifiable even without access to the data, but having only access to the fitted parameters.

Likelihood-Free Inference by Ratio Estimation

We consider the problem of parametric statistical inference when likelihood computations are prohibitively expensive but sampling from the model is possible. Several so-called likelihood-free methods have been developed to perform inference in the absence of a likelihood function. The popular synthetic likelihood approach infers the parameters by modelling summary statistics of the data by a Gaussian probability distribution. In another popular approach called approximate Bayesian computation, the inference is performed by identifying parameter values for which the summary statistics of the simulated data are close to those of the observed data. Synthetic likelihood is easier to use as no measure of “closeness” is required but the Gaussianity assumption is often limiting. Moreover, both approaches require judiciously chosen summary statistics. We here present an alternative inference approach that is as easy to use as synthetic likelihood but not as restricted in its assumptions, and that, in a natural way, enables automatic selection of relevant summary statistic from a large set of candidates. The basic idea is to frame the problem of estimating the posterior as a problem of estimating the ratio between the data generating distribution and the marginal distribution. This problem can be solved by logistic regression, and including regularising penalty terms enables automatic selection of the summary statistics relevant to the inference task. We illustrate the general theory on canonical examples and employ it to perform inference for challenging stochastic nonlinear dynamical systems and high-dimensional summary statistics.

Parametric inference for diffusions observed at stopping times

In this paper we study the problem of parametric inference for multidimensional diffusions based on observations at random stopping times. We work in the asymptotic framework of high frequency data over a fixed horizon. Previous works on the subject (such as [10, 17, 19, 5] among others) consider only deterministic, strongly predictable or random, independent of the process, observation times, and do not cover our setting. Under mild assumptions we construct a consistent sequence of estimators, for a large class of stopping time observation grids (studied in [20, 23]). Further we carry out the asymptotic analysis of the estimation error and establish a Central Limit Theorem (CLT) with a mixed Gaussian limit. In addition, in the case of a 1-dimensional parameter, for any sequence of estimators verifying CLT conditions without bias, we prove a uniform a.s. lower bound on the asymptotic variance, and show that this bound is sharp.

A geostatistical approach to estimating the parameters of a 3D Cox-Boolean discrete fracture network from 1D and 2D sampling observations

This work addresses the problem of parameter inference in the Cox-Boolean discrete fracture network model, in which the fractures are represented by flat discs with random orientations and diameters that are centered at the points of a doubly stochastic Poisson process or Cox process. Since the distributions of the fracture orientations and fracture diameters can be estimated from information of borehole and areal surveys, the focus is given to the distribution of the potential field of the Cox process, which measures the average number of fracture centers per unit volume. To this end, a relationship between the distributions of fracture diameter, fracture intensity (mean fractured surface per unit volume) and potential field over a large block support is established, and relationships between the moments of these distributions are derived.In practice, the moments of the fracture intensity at a point support can be inferred from borehole information and upscaled to larger block supports, using geostatistical tools. The moments of the potential field over these block supports can be deduced with the previous relationships, which in turn allows estimation of the distribution and spatial continuity of the point-support potential field under a given parametric model. The validity and applicability of the concepts are demonstrated on synthetic case studies and a workflow to fit a Cox-Boolean discrete fracture network model on the basis of information of borehole and areal surveys is provided.

Transport Map Accelerated Markov Chain Monte Carlo

We introduce a new framework for efficient sampling from complex probability distributions, using a combination of optimal transport maps and the Metropolis-Hastings rule. The core idea is to use continuous transportation to transform typical Metropolis proposal mechanisms (e.g., random walks, Langevin methods) into non-Gaussian proposal distributions that can more effectively explore the target density. Our approach adaptively constructs a lower triangular transport map-an approximation of the Knothe-Rosenblatt rearrangement-using information from previous MCMC states, via the solution of an optimization problem. This optimization problem is convex regardless of the form of the target distribution. It is solved efficiently using a Newton method that requires no gradient information from the target probability distribution; the target distribution is instead represented via samples. Sequential updates enable efficient and parallelizable adaptation of the map even for large numbers of samples. We show that this approach uses inexact or truncated maps to produce an adaptive MCMC algorithm that is ergodic for the exact target distribution. Numerical demonstrations on a range of parameter inference problems show order-of-magnitude speedups over standard MCMC techniques, measured by the number of effectively independent samples produced per target density evaluation and per unit of wallclock time.

Identification, Data Combination and the Risk of Disclosure

It is commonplace that the data needed for econometric inference are not contained in a single source. In this paper we analyze the problem of parametric inference from combined individual-level data when data combination is based on personal and demographic identi fiers such as name, age, or address. Our main question is the identi fication of the econometric model based on the combined data when the data do not contain exact individual identifi ers and no parametric assumptions are imposed on the joint distribution of information that is common across the combined dataset. We demonstrate the conditions on the observable marginal distributions of data in individual datasets that can and cannot guarantee identi fication of the parameters of interest. We also note that the data combination procedure is essential in the semiparametric setting such as ours. Provided that the (non-parametric) data combination procedure can only be de ned in finite samples, we introduce a new notion of identi fication based on the concept of limits of statistical experiments. Our results apply to the setting where the individual data used for inferences are sensitive and their combination may lead to a substantial increase in the data sensitivity or lead to a de-anonymization of the previously anonymized information. We demonstrate that the point identi fication of an econometric model from combined data is incompatible with restrictions on the risk of individual disclosure. If the data combination procedure guarantees a bound on the risk of individual disclosure, then the information available from the combined dataset allows one to identify the parameter of interest only partially, and the size of the identi cation region is inversely related to the upper bound guarantee for the disclosure risk. This result is new in the context of data combination as we notice that the quality of links that need to be used in the combined data to assure point identi cation may be much higher than the average link quality in the entire dataset, and thus point inference requires the use of the most sensitive subset of the data. Our results provide important insights into the ongoing discourse on the empirical analysis of merged administrative records as well as discussions on the disclosive nature of policies implemented by the data-driven companies (such as Internet services companies and medical companies using individual patient records for policy decisions).

Interactive design of experiments: A priori global versus sequential optimization, revised under changing states of knowledge

AbstractPredicting hydro(geo)logical or environmental systems is subject to high levels of uncertainties, especially if appropriate data for model calibration are lacking. For subsurface systems, where data acquisition is cost intensive and time demanding, it is especially important to collect only those data that provide the largest amount of relevant information. The high expenses call for optimal experimental design, which is widely recognized for maximizing the efficiency of experiments. In model‐based design of experiments, the analysis of the design efficiency and the resulting optimal design are based on the initial state of knowledge about the modeled system. Joint optimization of multimeasurement designs is a well‐known challenge, and the usefulness of global optimization approaches is widely recognized in this context. However, we will show that the benefit for such global optimization becomes questionable when measurement data become available sequentially. Instead, the optimization effort should be invested within an interactive design approach. Today's fast telecommunication, global connectivity, and high‐performance computing allow to consider such interactive coupling. This study will use a synthetic case study to compare the standard en‐bloc global optimization approach to two interactive design approaches. The approaches are implemented in a Bayesian framework and are compared based on their complexity and overall performance. The key conclusion confirms a previously untested presumption: for models that trigger nonlinear parameter inference problems, interaction (which may come at a loss of global optimization) is more beneficial than global optimization based on the initial state of knowledge (which typically implies the impossibility of interactivity).

Bayesian inference for Markov jump processes with informative observations.

In this paper we consider the problem of parameter inference for Markov jump process (MJP) representations of stochastic kinetic models. Since transition probabilities are intractable for most processes of interest yet forward simulation is straightforward, Bayesian inference typically proceeds through computationally intensive methods such as (particle) MCMC. Such methods ostensibly require the ability to simulate trajectories from the conditioned jump process. When observations are highly informative, use of the forward simulator is likely to be inefficient and may even preclude an exact (simulation based) analysis. We therefore propose three methods for improving the efficiency of simulating conditioned jump processes. A conditioned hazard is derived based on an approximation to the jump process, and used to generate end-point conditioned trajectories for use inside an importance sampling algorithm. We also adapt a recently proposed sequential Monte Carlo scheme to our problem. Essentially, trajectories are reweighted at a set of intermediate time points, with more weight assigned to trajectories that are consistent with the next observation. We consider two implementations of this approach, based on two continuous approximations of the MJP. We compare these constructs for a simple tractable jump process before using them to perform inference for a Lotka-Volterra system. The best performing construct is used to infer the parameters governing a simple model of motility regulation in Bacillus subtilis.

On optimality of kernels for approximate Bayesian computation using sequential Monte Carlo

Approximate Bayesian computation (ABC) has gained popularity over the past few years for the analysis of complex models arising in population genetics, epidemiology and system biology. Sequential Monte Carlo (SMC) approaches have become work-horses in ABC. Here we discuss how to construct the perturbation kernels that are required in ABC SMC approaches, in order to construct a sequence of distributions that start out from a suitably defined prior and converge towards the unknown posterior. We derive optimality criteria for different kernels, which are based on the Kullback-Leibler divergence between a distribution and the distribution of the perturbed particles. We will show that for many complicated posterior distributions, locally adapted kernels tend to show the best performance. We find that the added moderate cost of adapting kernel functions is easily regained in terms of the higher acceptance rate. We demonstrate the computational efficiency gains in a range of toy examples which illustrate some of the challenges faced in real-world applications of ABC, before turning to two demanding parameter inference problems in molecular biology, which highlight the huge increases in efficiency that can be gained from choice of optimal kernels. We conclude with a general discussion of the rational choice of perturbation kernels in ABC SMC settings.