Predicting Second Language Learners’ Actual Knowledge Using Self-perceived Knowledge

Abstract

AbstractSelf-perceived knowledge refers to the knowledge that learners believe they possess. To measure actual knowledge, developing a valid test for subjects is typically necessary. Self-perceived knowledge is considerably easier to measure than actual knowledge because it can be obtained by simply asking the participants if they possess the knowledge. Hence, if the actual knowledge of a subject can be predicted with high accuracy from their self-perceived knowledge, the burden of formulating test questions and building a dataset can be reduced. In this study, we created a reliable dataset for predicting actual knowledge from self-perceived knowledge in the field of second-language vocabulary learning; this dataset, to the best of our knowledge, is the first of its type. Herein, we provide detailed item response theory (IRT)–based analyses for our datasets as well as simple IRT-based methods for predicting the responses of learners to actual knowledge. We also demonstrate a deep transfer learning–based approach that slightly outperforms the IRT-based approach in terms of predictive accuracy.KeywordsSelf-perceived knowledgeItem response theoryTransformer modelsVocabulary testing