Sensitivity analysis of agent-based models: a new protocol

Emanuele Borgonovo,Nicolaj Siggelkow,Leonardo Rizzo,Jan Rivkin,Marco Pangallo

doi:10.1007/s10588-021-09358-5

Abstract

Agent-based models (ABMs) are increasingly used in the management sciences. Though useful, ABMs are often critiqued: it is hard to discern why they produce the results they do and whether other assumptions would yield similar results. To help researchers address such critiques, we propose a systematic approach to conducting sensitivity analyses of ABMs. Our approach deals with a feature that can complicate sensitivity analyses: most ABMs include important non-parametric elements, while most sensitivity analysis methods are designed for parametric elements only. The approach moves from charting out the elements of an ABM through identifying the goal of the sensitivity analysis to specifying a method for the analysis. We focus on four common goals of sensitivity analysis: determining whether results are robust, which elements have the greatest impact on outcomes, how elements interact to shape outcomes, and which direction outcomes move when elements change. For the first three goals, we suggest a combination of randomized finite change indices calculation through a factorial design. For direction of change, we propose a modification of individual conditional expectation (ICE) plots to account for the stochastic nature of the ABM response. We illustrate our approach using the Garbage Can Model, a classic ABM that examines how organizations make decisions.

Highlights

Simulation modeling has become increasingly important in studying organizational behavior (Carley 2002; Harrison et al 2007)
We propose a modification of individual conditional expectation (ICE) plots to account for the stochastic nature of the Agent-based models (ABMs) response
We propose to visualize this information in Stochastic Individual Conditional Expectation (S-ICE) plots that take into account the stochastic nature of the ABM response

Summary

Introduction

Simulation modeling has become increasingly important in studying organizational behavior (Carley 2002; Harrison et al 2007). Among the several simulation approaches available, agent-based models (ABMs) play a special role (Prietula and Carley 1994). Since early classics such as the Garbage Can Model. (Cohen et al 1972), researchers have increasingly used agent-based simulation techniques to address relevant organizational, strategic and operational questions (Prietula et al 1998; Luo et al 2018; Barnes et al 2020). We can broadly define ABMs as computational models in which aggregate outcomes emerge from agents’ properties, behaviors and interactions, without the imposition of any top-down constraint. This makes ABMs extremely flexible, as it is relatively easy to vary the building blocks of an ABM. Lacking a closed-form expression that links inputs to outputs, agent-based modelers often struggle to assess whether the results of their ABMs are robust and their conclusions are valid

Objectives

Methods

Results

Discussion

Conclusion