NASA-TLX–based workload assessment for academic resource recommender system

Abstract

Recommender systems are expected to promote a student-centered teaching and learning environment. The age of information abundance has proven to need such systems. Recommender systems have been used to recommend learning items related to students’ research interests. Serendipity has also made its way into the academic environment, as systems recommend items that are useful and surprising to learners. Understanding user workload is important for students who use serendipitous recommender systems. In this research, we investigate various user interfaces for academic recommender systems by looking at students who are attempting to obtain serendipitous recommendations for their academic tasks. The study was evaluated on the NASA task load index (NASA-TLX). Our priority was to understand the mental, physical, and other workload attributes that can change when students seek serendipitous recommendations. We studied Mendeley, Google Scholar, Academia.edu, and ResearchGate. Our study found no substantial serendipitous recommendations observed by the users, but a few traces of serendipitous experiences were observed. Further, no substantial workload was detected in using the systems. However, the recommender system did create different user experiences across repeated sessions. Further, a diverse range of task loads is associated with the recommenders used in academia, from mixed designs with rich user controls to very few controls. This research provided us with insights that can be used to help designers incorporate and accommodate new features and take calculated risks when designing serendipitous education technology.