What do the pictures mean? Guidelines for experimental evaluation of representation fidelity in diagrammatical conceptual modeling techniques

Abstract

It is important to articulate the objectives and underlying assumptions behind a growing body of experimental research in conceptual modeling. We provide four guidelines for developing materials for experiments that evaluate conceptual modeling techniques, under the assumption that a primary purpose of conceptual modeling is to facilitate communication between analysts and users in validating domain knowledge during systems development. These guidelines assist in developing experimental materials that support meaningful tests of domain semantics. We present empirical evidence indicating the value of two of the guidelines. We also evaluate selected recent experiments on conceptual modeling with respect to the guidelines.