Predicting Second Language Proficiency Level Using Linguistic Cognitive Task and Machine Learning Techniques

Abstract

This paper proposes a novel method for predicting second language proficiency based on linguistic cognitive ability measured in linguistic cognitive response test. Our method is based on an assumption that there is a correlation between language aptitude test scores and linguistic cognitive ability. Our proposed method for predicting L2 language proficiency uses as input learner's linguistic cognition aptitude data. In our experiment, the method produced promising results with the predictive power as high as 70 %. Linguistic cognitive ability is measured through linguistic cognition tasks, which are: reading lexical decision tasks (LDT), listening LDT, translation recognition tasks, and semantic recognition tasks. Each type of the tasks is related to a different linguistic function in the brain. After measuring the learner's linguistic cognitive aptitude, the result is fed as input for a machine learning model, which makes predictions for the corresponding language proficiency level. In training the linguistic proficiency classifier, we used multi-layer perceptron, Naive Bayes, logistic regression, and random forest model. For input data set in our experiment, we had 42 participants take our cognitive aptitude tests and used the result. Our classifier showed an accuracy >70 % in predicting proficiency level. Among the models, random forest model produced the best predictive power.