THE APPLICATION OF THE ANALYTIC HIERARCHY PROCESS FOR EVALUATING CREATIVE PRODUCTS IN SCIENCE CLASS AND ITS MODIFICATION FOR EDUCATIONAL EVALUATION

Abstract

To design and to manufacture in science learning are increasingly important in science education. Yet, evaluation techniques in school for evaluating students’ creative products are apparently left behind. With the aim of developing an evaluation method to evaluate creative products in science and technology class, this study constructed a set of criteria with data collected from teachers and students. The analytic hierarchy process (AHP), a multiple criteria decision-making tool for single rater, was selected for the purpose of weighting and evaluating students’ products. However, the traditional AHP used one rater’s pair-wise comparisons; its subjectivity and complexity limit its applications in school. For solving this problem, this study developed an advanced technique, called direct-rating AHP (DR-AHP), to extend the applicability of the traditional AHP. The DR-AHP is used to obtain weights or preferences for criteria/alternatives by a process of directly ranking criteria/alternatives by single/multi rater(s), checking consistency, and developing a rank vector matrix. The DR-AHP was implemented in obtaining criteria weights of a hierarchy framework for creative products evaluation by a group of science educators (N = 13) and field-tested in ranking creative products by another group of science teachers (N = 9). Results showed its superiority in objectivity and efficiency over traditional ways of evaluation. The results also demonstrate how the AHP and DR-AHP are capable of helping evaluators systematically construct criteria and/or to evaluate students’ creative products for classroom instruction as well as during many other activities.