Evaluating the Effectiveness of Accommodations Given to Students With Learning Impairments: Ordinal and Interpretable Machine-Learning-Based Methodology

Abstract

In most academic institutions, students with learning impairments (LIs) are entitled to various accommodations as a means of compensating for their impairment. Ensuring that the appropriate accommodations were selected requires an intelligent support tool to track their effectiveness. In this article, we regard the effectiveness of such accommodations in terms of their quality and reliability. High-quality accommodations allow students with LIs equal access and equal opportunity to demonstrate their knowledge compared to their peers who do not have LIs. Highly reliable accommodations mean a significant performance difference between students with LIs who actually use the accommodations compared with students with LIs who do not use the accommodations, given similar student characteristics. Previous literature is inconclusive regarding the evidence of the effectiveness of such accommodations since different accommodations may have a different effect on different subgroups of exams and students. This article proposes a methodology, based on ordinal interpretable models, that produce practical insights to professionals who are responsible for students with LIs, to address the problem of exploring the effectiveness of learning accommodations. The suggested models use the ordinal information of the target variables for evaluating student performance and yield practical insights for designing the most suitable accommodation for a student with LIs based on their characteristics. The ordinal interpretable models are evaluated using a database of tens of thousands of engineering students. The results demonstrate that the suggested interpretable models perform significantly better than the compared algorithms on all measures.