Assessing the quality of scientific explanations with networks

Abstract

ABSTRACT This article introduces a network approach to describe the quality of written scientific explanations. Existing approaches evaluate explanations mainly on the level of sentences or as a whole but not on the elementary level of single terms. Moreover, evaluation of explanations is often based on highly inferential scoring techniques. We addressed both issues by converting the elementary structure of terms in explanations into networks (so-called element maps) and analysing these with mathematical measures, thus extracting the size and complexity of an explanation, adequacy, coherence, and use of key terms. A total of 65 explanations of experts and students were analysed quantitatively and qualitatively. Differences between expert and student maps’ measures can be interpreted meaningfully against the background of existing research findings. Thus, we argue that our approach using network analysis provides a precise, fine-grained, and low-inferential tool that complements and refines existing approaches. Element maps have the potential to improve teaching and research by precisely revealing the strengths and weaknesses of explanations.