Applying Kano’s two-factor theory to prioritize learning analytics dashboard features for learning technology designers

Abstract

Existing methods for software requirements elicitation, five-point Likert scales and voting methods for requirements prioritization, and usability and user experience evaluation methods do not enable prioritizing the learning analytics dashboard requirements. Inspired by management and product design field, this research applies Kano’s two-factor theory to prioritize the features of learning analytics dashboards (LADs) of adaptive learning platform (ALP) called Rhapsode<sup>TM</sup> learner, based on students’ perceived usefulness to support designers’ decision-making. Comparing usability and user experience methods for evaluating LAD features, this paper contributes with the protocol and a case applying Kano method for evaluating the perceived importance of the dashboards in ALP. The paper applies Kano’s two-factor questionnaire on the 13 LADs features of Rhapsode<sup>TM</sup> learner. Responses from 17 students are collected using a questionnaire, which is used to showcase the strength of the two-factor theory through five tabular and graphical techniques. Through these five tabular and graphical techniques, we demonstrate the application and usefulness of the method as designers and management are often carried away by the possibilities of insights instead of actual usefulness. The results revealed a variation in the categorization of LADs depending on the technique employed. As the complexity of the techniques increases, additional factors that indicate data uncertainty are gradually incorporated, clearly highlighting the growing requirement for data. In the case of RhapsodeTM learner platform, results based on the students responses show that 11 of 13 LADs being excluded due to low significance level in categorization (technique 1) and low response rate.