Development and Evaluation of a Digital Expert Elicitation Method Aimed at Fostering Elementary School Teachers' Diagnostic Competence

Kimberley Lek,Rens Van De Schoot

doi:10.3389/feduc.2018.00082

Abstract

Expert elicitation - an approach to systematically consult experts and quantify their insights - has been succesfully applied in fields as risk assessment, health and environmental research. Unfortunately, it has never been used within the Educational sciences, while it offers ample opportunities for educational practice, especially when used to foster the accuracy of teacher judgments; generally referred to as their ‘diagnostic competence’. The current paper is the first to explore expert elicitation in an educational context and has two major goals. The first goal is to develop a digital expert elicitation method suitable to be used by elementary school teachers for self-reflection purposes. The second goal is to extensively test the expert elicitation method, using a test panel of 24 primary school teachers for 503 pupils in total. Results regarding the development of the elicitation method and its reliability, construct validity, face validity and feasibility are discussed as well as ideas how this elicitation method can be a valuable self-reflection instrument for teachers. The results are promising: all measures of reliability, feasibility, face validity and construct validity show positive results and teachers are enthusiastic about the possibilities of the method.

Highlights

EXPERT ELICITATION Expert JudgmentsThe general goal of expert elicitation is to capture and quantify experts’ implicit and intagible judgments
The first goal is to develop a digital expert elicitation method suitable to be used by elementary school teachers for self-reflection purposes
An easy and intuitive expert elicitation method is discussed which is suitable to be used by elementary school teachers

Summary

Introduction

The general goal of expert elicitation is to capture and quantify experts’ implicit and intagible judgments. Applied to the context of elementary school teaching, our purpose is to make explicit teachers’ implicit judgments of their pupils ability. These judgments are a reflection of the teacher’s diagnostic competence; his or her ability to judge achievement characteristics of his or her pupils correctly (Artelt and Rausch, 2014). Judgments with regard to other areas such as language development can be elicited, the advantage of mathematics is that it is a relatively unambiguous area. As Artelt and Rausch (2014) put it: “Given that mathematical skill is mainly acquired in school and that the curriculum is well-defined according to age, teachers quite likely have a shared understanding of what constitutes mathematical proficiency” (p. 35)

Methods

Results

Conclusion