Modeling Knowledge Using a Crowd Of Experts

Abstract

This paper describes a series of methodological enhancements made to the model-building method in DESIM (Descriptive to Executable Simulation Modeling), which is a participatory modeling technique that converts qualitative models drawn from subject-matter experts into quantitative models that can be used for decision support. The knowledge elicitation methods of three studies are reviewed and evaluated in terms of the quality of the models they produced and their ability to generate actionable information for decision makers. The results, which can be applied across other participatory modeling techniques, demonstrate the practical impacts of different knowledge elicitation strategies for modeling expert knowledge, including the time and cost of acquiring qualified experts and the level of hands-on involvement of experts in model building.