Machine Learning Techniques with Item Response Data

Abstract

This is a two-pronged study. Two aims of this study are both interested in comparison of 1) different R IRT (Item Response Theory) packages and 2) different machine learning techniques. The first part of the present study addresses the performance of a couple of psychometric software available in R. Then, using item parameters from real data sets from the Program of International Student Assessment 2015, three machine learning techniques (classification tree, random forest, naïve Bayes) are assessed in the second part. Between these two parts, connections of IRT and ML are briefly described and reviewed with reference. Findings are 1) two comparisons of two sets of packages for each IRT model are not informative (from the part I) and 2) the classifiers of random forest and naïve Bayes algorithms performed better than the classification tree, which confirms that more information leads to more accurate prediction and ensemble approaches are better than single tree approaches. (from the part II). Finally, implications and caveats of this study are provided.