ASSESSING STUDENTS’ HIGHER ORDER THINKING SKILLS USING MULTIDIMENSIONAL ITEM RESPONSE THEORY

Abstract

The development of the economics HOTS test that combines critical thinking skills, problem-solving, and critical thinking are essential to meet the challenges of the 21st century life skills. The combination of these thinking skills in the HOT test will help teachers to diagnose students' strengths and weaknesses. However, it could interfere with the accuracy of the measurement results if analyzed using IRT in a single analysis. The Multidimensional Item Response Theory (MIRT) resolved the accuracy of the measurement issues. This research aimed to develop the economics HOTS test to estimate the student’s abilities in higher order thinking skills using the MIRT. The samples were 750 high school students selected from fourteen high schools in West Sumatera, Indonesia. The data were collected using tests which calibrated through the simple-structure MIRT model using R studio. The test reliability was calculated based on coefficients Alpha and test information function. The results show MIRT offers accurate measurement in estimating multidimensional test parameters. The item had a moderate average multidimensional discriminant and difficulty, while the students had a moderate HOTS ability. Their ability to think creatively was lower than critical thinking and problem-solving abilities. The test was proven to be reliable with coefficients Alpha of 0.81, it yielded a high-test information function of the 4.0124, and a low measurement error of 0.4992. It is suitable to be tested on students who have moderate abilities in problem-solving and critical thinking, but with high creative thinking ability. Keywords: HOTS, critical thinking, creative thinking, problem solving, MIRT.