Staying on target: A systematic literature review on learner‐facing learning analytics dashboards

Abstract

AbstractThe advances in technology to capture and process unprecedented amounts of educational data has boosted the interest in Learning Analytics Dashboard (LAD) applications as a way to provide meaningful visual information to administrators, parents, teachers and learners. Despite the frequent argument that LADs are useful to support target users and their goals to monitor and act upon the information provided, little is known about LADs’ theoretical underpinnings and the alignment (or lack thereof) between LADs intended outcomes and the measures used to evaluate their implementation. However, this knowledge is necessary to illuminate more efficient approaches in the development and implementation of LAD tools. Guided by the self‐regulated learning perspective and using the Preferred Reporting Items for Systematic Reviews and Meta‐Analyses (PRISMA) framework, this systematic literature review addressed this gap by examining whether and how learner‐facing LAD’s target outcomes align with the domain measures used to evaluate their implementations. Out of the 1297 papers retrieved from 15 databases, 28 were included in the final quantitative and qualitative analysis. Results suggested an intriguing lack of alignment between LADs’ intended outcomes (mostly cognitive domain) and their evaluation (mostly affective measures). Based on these results and on the premise that LADs are designed to support learners, a critical recommendation from this study is that LADs’ target outcomes should guide the selection of measures used to evaluate the efficacy of these tools. This alignment is critical to enable the construction of more robust guidelines to inform future endeavours in the field. Practitioner notesWhat is already known about this topic There has been an increased interest and investment in learning analytics dashboards to support learners as end‐users. Learner‐facing learning analytics dashboards are designed with different purposes, functionalities and types of data in an attempt to influence learners’ behaviour, achievement and skills. What this paper adds This paper reports trends and opportunities regarding the design of learner‐facing learning analytics dashboards, contexts of implementation, as well as types and features of learner‐facing learning analytics dashboard studies. The paper discusses how affect and motivation have been largely overlooked as target outcomes in learner‐facing learning analytics dashboards. Implications for practice and/or policy Based on the evidence gathered through the review, this paper makes recommendations for theory (eg, inclusion of motivation as an important target outcome). The paper makes recommendations related to the design, implementation and evaluation of learning analytics dashboards. The paper also highlights the need for further integration between learner‐facing learning analytics dashboards and open learner models.