Calibration Of Agent-based Model Research Articles

A Pipeline for Calibrating Agent-Based Models of Microbial Populations: From Image Collection to Model Parameterization

We present a framework for calibration of agent-based models of bacilliform (rod-shaped) bacterial populations based on time-lapse microscopy at single-cell resolution. Our approach draws on pattern-oriented modelling for feature selection, followed by sensitivity analysis and Bayesian inference to arrive at a model calibration that aligns with experimental observations. We illustrate this pipeline by calibrating a model of microcolony formation against observations of monolayer E. coli population growth.

On the Epidemiological Evolution of Colistin-ResistantAcinetobacter Baumanniiin the City of Valencia: An Agent-Based Modelling Approach

Antibiotic resistance is one of the greatest public health threats today, mainly due to the non-rational use of antibiotics.Acinetobacter baumanniiis an example of a microorganism with high antibiotic resistance that has developed rapidly in recent years. Consequently, only a few lastresort antibiotics, such as colistin, are currently effective against it. In this work, we propose a random agent-based computational model to describe the evolution of colistin-resistantA. baumanniiin the population of Valencia (Spain) and to predict its impact both on the whole population and by age groups. The agent- based model uses a synthetic population of individuals with a vector of characteristics or state variables. These variables change over time based on a series of random events with certain conditional probabilities. The synthetic population statistical features and the probabilities have been found in demographic and hospital databases. One of these probabilities, the probability of infection by a resistant strain, has been modeled using random differential equations. The model takes into account antibiotic consumption as the primary driving force of variation and assumes non-rewersibility of resistance as the worst-case scenario. The agent-based model calibration and the selection of a real-world representative set of solutions have been carried out using the Partide Swarm Optimization evolutionary algorithm. This approach takes into account the inherent stochasticity of the model and the uncertainty of the data. Finally, projections of the incidence and absolute cases of colistin-resistantA. baumanniihave been performed. Our results suggest that, if the same consumption pattern continues, the ervolution of the colistin-resistant strain proportion will be exponential, exceeding 50% in 2025. Additionally, the results reveal that, despite the low incidence in Valencian hospitals, the impact on people over 60 years old will be more significant in terms of the number of cases. Based on these findings, it can be deduced that colistin will cense to be an effective antibiotic in the coming years, negatively impacting the human population, especially the most advanced age groups.

Bridging the Micro and Macro: Calibration of Agent-Based Model Using Mean-Field Dynamics

Calibration of agent-based models (ABM) is an essential stage when they are applied to reproduce the actual behaviors of distributed systems. Unlike traditional methods that suffer from the repeated trial and error and slow convergence of iteration, this article proposes a new ABM calibration approach by establishing a link between agent microbehavioral parameters and systemic macro-observations. With the assumption that the agent behavior can be formulated as a high-order Markovian process, the new approach starts with a search for an optimal transfer probability through a macrostate transfer equation. Then, each agent's microparameter values are computed using mean-field approximation, where his complex dependencies with others are approximated by an expected aggregate state. To compress the agent state space, principal component analysis is also introduced to avoid high dimensions of the macrostate transfer equation. The proposed method is validated in two scenarios: 1) population evolution and 2) urban travel demand analysis. Experimental results demonstrate that compared with the machine-learning surrogate and evolutionary optimization, our method can achieve higher accuracies with much lower computational complexities.

Calibration of Agent-Based Models by Means of Meta-Modeling and Nonparametric Regression

Calibration of Agent-Based Models by Means of Meta-Modeling and Nonparametric Regression

Multimodal Evolutionary Algorithms for Easing the Complexity of Agent-Based Model Calibration

Multimodal Evolutionary Algorithms for Easing the Complexity of Agent-Based Model Calibration

Coral reefs optimization algorithms for agent-based model calibration

Calibrating agent-based models involves estimating multiple parameter values. This can be performed automatically using automatic calibration but its success depends on the optimization method’s ability for exploring the parameter search space. This paper proposes to carry out this process using coral reefs optimization algorithms, a new branch of competitive bio-inspired metaheuristics that, beyond its novel metaphor, has shown its good behavior in other optimization problems. The performance of these metaheuristics for model calibration is evaluated by conducting an exhaustive experimentation against well-established and recent evolutionary algorithms, including their hybridization with local search procedures. The study analyzes the calibration accuracy of the metaheuristics using an integer coding scheme over a benchmark of 12 problem instances of an agent-based model with an increasing number of decision variables. The outstanding performance of the memetic coral reefs optimization is reported after performing statistical tests to the results.

Evolutionary Multiobjective Optimization for Automatic Agent-Based Model Calibration: A Comparative Study

Complex problems can be analyzed by using model simulation but its use is not straight-forward since modelers must carefully calibrate and validate their models before using them. This is specially relevant for models considering multiple outputs as its calibration requires handling different criteria jointly. This can be achieved using automated calibration and evolutionary multiobjective optimization methods which are the state of the art in multiobjective optimization as they can find a set of representative Pareto solutions under these restrictions and in a single run. However, selecting the best algorithm for performing automated calibration can be overwhelming. We propose to deal with this issue by conducting an exhaustive analysis of the performance of several evolutionary multiobjective optimization algorithms when calibrating several instances of an agent-based model for marketing with multiple outputs. We analyze the calibration results using multiobjective performance indicators and attainment surfaces, including a statistical test for studying the significance of the indicator values, and benchmarking their performance with respect to a classical mathematical method. The results of our experimentation reflect that those algorithms based on decomposition perform significantly better than the remaining methods in most instances. Besides, we also identify how different properties of the problem instances (i.e., the shape of the feasible region, the shape of the Pareto front, and the increased dimensionality) erode the behavior of the algorithms to different degrees.

A comparison of economic agent-based model calibration methods

Despite significant expansion in recent years, the literature on quantitative and data-driven approaches to economic agent-based model validation and calibration consists primarily of studies that have focused on the introduction of new calibration methods that are neither benchmarked against existing alternatives nor rigorously tested in terms of the quality of the estimates they produce. In response, we compare a number of prominent agent-based model calibration methods, both established and novel, through a series of computational experiments in an attempt to determine the respective strengths and weaknesses of each approach. Overall, we find that a simple, likelihood-based approach to Bayesian estimation consistently outperforms several members of the more popular class of simulated minimum distance methods and results in reasonable parameter estimates in many contexts, with a degradation in performance observed only when considering a large-scale model and attempting to fit a substantial number of its parameters.

A multicriteria integral framework for agent-based model calibration using evolutionary multiobjective optimization and network-based visualization

Automated calibration methods are a common approach to agent-based model calibration as they can estimate those parameters which cannot be set because of the lack of information. The modeler requires to validate the model by checking the parameter values before the model can be used and this task is very challenging when the model considers two or more conflicting outputs. We propose a multicriteria integral framework to assist the modeler in the calibration and validation of agent-based models that combines evolutionary multiobjective optimization with network-based visualization, which we believe is the first integral approach to model calibration. On the one hand, evolutionary multiobjective optimization provides several sets of calibration solutions (i.e., parameter values) with different trade-offs for the considered objectives in a single run. On the other hand, network-based visualization is used to better understand the decision space and the set of solutions from the obtained Pareto set approximation. To illustrate our proposal, we face the calibration of three agent-based model examples for marketing which consider two conflicting criteria: the awareness of the brand and its word-of-mouth volume. The final analysis of the calibrated solutions shows how our proposed framework eases the analysis of Pareto sets with high cardinality and helps with the identification of flexible solutions (i.e., those having close values in the design space).

A Comparison of Economic Agent-Based Model Calibration Methods

Interest in agent-based models of financial markets and the wider economy has increased consistently over the last few decades, in no small part due to their ability to reproduce a number of empirically-observed stylised facts that are not easily recovered by more traditional modelling approaches. Nevertheless, the agent-based modelling paradigm faces mounting criticism, focused particularly on the rigour of current validation and calibration practices, most of which remain qualitative and stylised fact-driven. While the literature on quantitative and data-driven approaches has seen significant expansion in recent years, most studies have focused on the introduction of new calibration methods that are neither benchmarked against existing alternatives nor rigorously tested in terms of the quality of the estimates they produce. We therefore compare a number of prominent ABM calibration methods, both established and novel, through a series of computational experiments in an attempt to determine the respective strengths and weaknesses of each approach and the overall quality of the resultant parameter estimates. We find that Bayesian estimation, though less popular in the literature, consistently outperforms frequentist, objective function-based approaches and results in reasonable parameter estimates in many contexts. Despite this, we also find that agent-based model calibration techniques require further development in order to definitively calibrate large-scale models. We therefore make suggestions for future research.

Markov chain analysis in agent-based model calibration by classical and simulated minimum distance

Agent-based models are nowadays widely used; however, their calibration on real data still remains an open issue which prevents to exploit completely their potentiality. Rarely such a kind of models can be studied analytically; more often they are studied by simulation. Among the problems encountered in ABM calibration, the choice of the criteria to fit can appear arbitrary. Markov chain analysis can come through to identify a standard procedure able to face this issue. Indeed, Izquierdo et al. (J Artif Soc Soc Simul 12(16):1–6, 2009) show that many computer simulation models can be represented as Markov chains. Exploiting such an idea classical minimum distance and its simulated counterpart, i.e., simulated minimum distance, are discussed theoretically and applied to Kirman model, which can be reformulated as a Markov chain. Comparison with approximate Bayesian computation is also addressed.

Agent-based model calibration using machine learning surrogates

Efficiently calibrating agent-based models (ABMs) to real data is an open challenge. This paper explicitly tackles parameter space exploration and calibration of ABMs by combining machine-learning and intelligent iterative sampling. The proposed approach “learns” a fast surrogate meta-model using a limited number of ABM evaluations and approximates the nonlinear relationship between ABM inputs (initial conditions and parameters) and outputs. Performance is evaluated on the Brock and Hommes (1998) asset pricing model and the “Islands” endogenous growth model Fagiolo and Dosi (2003). Results demonstrate that machine learning surrogates obtained using the proposed iterative learning procedure provide a quite accurate proxy of the true model and dramatically reduce the computation time necessary for large scale parameter space exploration and calibration.

Empirically grounded agent-based models of innovation diffusion: a critical review

Innovation diffusion has been studied extensively in a variety of disciplines, including sociology, economics, marketing, ecology, and computer science. Traditional literature on innovation diffusion has been dominated by models of aggregate behavior and trends. However, the agent-based modeling (ABM) paradigm is gaining popularity as it captures agent heterogeneity and enables fine-grained modeling of interactions mediated by social and geographic networks. While most ABM work on innovation diffusion is theoretical, empirically grounded models are increasingly important, particularly in guiding policy decisions. We present a critical review of empirically grounded agent-based models of innovation diffusion, developing a categorization of this research based on types of agent models as well as applications. By connecting the modeling methodologies in the fields of information and innovation diffusion, we suggest that the maximum likelihood estimation framework widely used in the former is a promising paradigm for calibration of agent-based models for innovation diffusion. Although many advances have been made to standardize ABM methodology, we identify four major issues in model calibration and validation, and suggest potential solutions.

Agent-Based Model Calibration Using Machine Learning Surrogates

Efficiently calibrating agent-based models (ABMs) to real data is an open challenge. This paper explicitly tackles parameter space exploration and calibration of ABMs by combining machine-learning and intelligent iterative sampling. The proposed approach learns a fast surrogate meta-model using a limited number of ABM evaluations and approximates the nonlinear relationship between ABM inputs (initial conditions and parameters) and outputs. Performance is evaluated on the (Brock and Hommes, 1998) asset pricing model and the Islands endogenous growth model (Fagiolo and Dosi, 2003). Results demonstrate that machine learning surrogates obtained using the proposed iterative learning procedure provide a quite accurate proxy of the true model and dramatically reduce the computation time necessary for large scale parameter space exploration and calibration.

Assessing historical reliability of the agent-based model of the global energy system

This study looks at the historical reliability of the agent-based model of the global energy system. We present a mathematical framework for the agent-based model calibration and sensitivity analysis based on historical observations. Simulation consistency with the historical record is measured as a distance between two vectors of data points and inference on parameter values is done from the probability distribution of this stochastic estimate. Proposed methodology is applied to the model of the global energy system. Some model properties and limitations followed from calibration results are discussed.

Dynamic calibration of agent-based models using data assimilation.

A widespread approach to investigating the dynamical behaviour of complex social systems is via agent-based models (ABMs). In this paper, we describe how such models can be dynamically calibrated using the ensemble Kalman filter (EnKF), a standard method of data assimilation. Our goal is twofold. First, we want to present the EnKF in a simple setting for the benefit of ABM practitioners who are unfamiliar with it. Second, we want to illustrate to data assimilation experts the value of using such methods in the context of ABMs of complex social systems and the new challenges these types of model present. We work towards these goals within the context of a simple question of practical value: how many people are there in Leeds (or any other major city) right now? We build a hierarchy of exemplar models that we use to demonstrate how to apply the EnKF and calibrate these using open data of footfall counts in Leeds.

Effects of Research Policy in Biotechnology: An Empirical Agent-Based Model of Knowledge Generation

Over the recent past, we can observe increasing interest in the ex-ante impact assessment of research policy, mainly related to the growing importance of accountability and limited budgets. However, existing methods often lack quantitative scenarios that go beyond extrapolations of current trends. This study addresses this research gap by proposing an empirical agent-based model (ABM) of knowledge generation in a system of researching firms. With our emphasis on the empirical calibration of ABMs, we intend to conduct scenario simulations applicable to real world contexts – in this study illustrated by means of an ABM on the Austrian biotechnology sector. In our model, effects of public research policy on the knowledge-related system output – measured by the patent portfolio – are under scrutiny. By this, the study contributes to the literature on ABMs in several aspects: Building on an existing concept of knowledge representation, we advance the model of individual and collective knowledge generation in firms by conceptualising policy intervention and corresponding output indicators. Furthermore, we go beyond symbolic ABMs of knowledge production by using empirical patent data as knowledge representations, adopt an elaborate empirical initialisation and calibration strategy using company data, and utilise econometric techniques to generate a sector-specific fitness function that determines the model output. With this model, we are able to conduct scenario analyses on effects of different public research funding schemes in the field of biotechnology. The results demonstrate that an empirically calibrated and transparent model design increases credibility and robustness of the ABM approach in the context of ex-ante impact assessment of public research policy.

Derivation of spatiotemporal data for cyclists (from video) to enable agent-based model calibration

Cycling, as a mode share of urban travel, is widely desired to be increased. To support the delivery of improved infrastructure, robust modelling is desired and modelling such agents demands real-world calibration. Data to enable this is expensive to obtain by conventional means.This paper presents and demonstrates a process for the analysis of video to provide such data. This analysis yields spatiotemporal data for experimentally-observed cyclists from which velocity information (amongst other things) can be derived. With further refinement, this process can be used in the analysis of existing and future highway video data.

A FRAMEWORK FOR THE CALIBRATION OF SOCIAL SIMULATION MODELS

Simulation with agent-based models is increasingly used in the study of complex socio-technical systems and in social simulation in general. This paradigm offers a number of attractive features, namely the possibility of modeling emergent phenomena within large populations. As a consequence, often the quantity in need of calibration may be a distribution over the population whose relation with the parameters of the model is analytically intractable. Nevertheless, we can simulate. In this paper we present a simulation-based framework for the calibration of agent-based models with distributional output based on indirect inference. We illustrate our method step by step on a model of norm emergence in an online community of peer production, using data from three large Wikipedia communities. Model fit and diagnostics are discussed.