Construction and analysis of educational assessments using knowledge maps with weight appraisal of concepts

Abstract

The rapid advance of information and communication technologies (ICT) has important impacts on teaching and learning, as well as on the educational assessment. Teachers may create assessments utilizing some developed assessment software or test authoring tools. However, problems could occur, such as neglecting key concepts in the curriculum or having disproportionate course topics distribution, when teachers create assessments or test items. This study proposes a novel approach, which uses knowledge map with appraisal of concept weights and other ICTs, and implements an assessment system KMAAS to help primary school teachers in Taiwan, or elsewhere, create educational assessments properly. When compiling an assessment, KMAAS ensures that teachers can include all important course concepts intended for assessing and maintain correct balance between course concepts among test items. It does so first by analyzing course material of the assessment range and displaying a concept-weight-annotated knowledge map which concretize and visualize the importance of and the relationships among concepts in the range. It then analyzes the test sheet which is being complied and displays another similar real-time updated knowledge map containing balance between course concepts among the test items. Teachers may cross-refer to these maps to help them adjust concept balances and even select appropriate test items from test banks. The system has being evaluated in both the accuracy of learning concepts extraction and the degree of user satisfaction, as measured by questionnaires given to the teachers who tested the system. The promising results confirm the feasibility of this system in helping teachers compile their educational assessments easily and precisely. Other results of the formative evaluations on techniques have being used to improve the system in order to make it more effective and efficient. The methodology and technologies KMAAS employed are all well developed and are domain independent, which makes it highly flexible to transfer to other course subject domain too.