Computer-based assessment of Complex Problem Solving: concept, implementation, and application

Abstract

Complex Problem Solving (CPS) skills are essential to successfully deal with environments that change dynamically and involve a large number of interconnected and partially unknown causal influences. The increasing importance of such skills in the 21st century requires appropriate assessment and intervention methods, which in turn rely on adequate item construction, delivery, and scoring. The lack of assessment tools, however, has slowed down research on and understanding of CPS. This paper first presents the MicroDYN framework for assessing CPS, which is based on linear structural equation systems with input and output variables and opaque relations among them. Second, a versatile assessment platform, the CBA Item Builder, which allows the authoring, delivery, and scoring of CPS tasks for scientific and educational purposes is introduced. Third, we demonstrate the potential of such a tool for research by reporting an experimental study illustrating the effect of domain specific content knowledge on performance in CPS tasks both on an overall performance and on a process level. The importance of accessible and versatile technical platforms not only for assessment and research but also for intervention and learning are discussed with a particular focus on educational contexts.