Modeler in a box: how can large language models aid in the simulation modeling process?

Abstract

We examine the potential of prompting a large language model-based chatbot, ChatGPT, to generate functional simulation model code from a prose-based narrative. The simple narrative describes how the mode of transportation for elementary school students changed due to the COVID-19 pandemic and related factors, including a lack of available bus drivers, lack of mask enforcement on buses, and inclement weather. We document the process of providing this narrative to ChatGPT and further prompting the chatbot to generate model code to represent the narrative and to make it executable. We test ChatGPT’s ability to use prose descriptions of a phenomenon to generate simulation models using three paradigms: discrete event system, system dynamics, and agent-based modeling. Our findings reveal that ChatGPT could not produce concise or executable models, facing higher difficulty when asked to do so in programming languages it was less familiar with. This analysis underscores the strengths and limitations of the current state of this technology for modeling and simulation. Furthermore, we propose how future advancements in Large Language Models may aid the simulation modeling process, increasing transparency and participation in multidisciplinary team efforts.