Learning Analytics Dashboard Research Articles

A learning analytics dashboard to support students' reflection on collaboration

AbstractBackgroundDashboards play a prominent role in learning analytics (LA) research. In collaboration activities, dashboards can show traces of team participation. They are often evaluated based on students' perceived satisfaction and engagement with the dashboard. However, there is a notable methodological gap in understanding how these dashboards support the nuanced process of student reflection.ObjectiveThis paper presents empirical evidence on how students from high and low‐performing groups reflect individually on their performance while using a Learning Analytics Dashboard (LAD).MethodsWe address this in the context of education in healthcare, wherein we captured actions and positioning data from a simulation‐based collaborative activity and generated a collaborative LAD. A total of 41 nursing students were invited to participate in a post‐hoc semi‐structured individual interview to use a collaborative LAD while answering a set of prompts to reflect on their individual and group performance. Students' reflections were coded and analysed using Bain's 5R reflection framework. We used epistemic network analysis to capture the dynamic reflection process and to understand the connections between the reflection stages (from low to high). We compared how different these connections were for students in high and low‐performing groups.Results and ConclusionsOur results revealed that most students were only able to achieve low and middle stages of reflection. Yet, students in low‐performing groups predominantly followed low‐to‐middle stages of reflection. In contrast, students from high‐performing groups demonstrated the ability to transition between low‐to‐middle and low‐to‐high stages of reflection. Based on these findings, we discuss implications for both research and practice, particularly emphasising the necessity to scaffold reflection when using LADs.

Designing a Human-centred Learning Analytics Dashboard In-use

Despite growing interest in applying human-centred design methods to create learning analytics (LA) systems, most efforts have concentrated on initial design phases, with limited exploration of how LA tools and practices can co-evolve during the actual learning and teaching activities. This paper examines how a human-centred LA dashboard can be further refined and adapted by teachers while actively using it in a real-world scenario (i.e., design-in-use), beyond its intended design (i.e., design-for-use). We use instrumental genesis as a theoretical lens to analyze the temporary and permanent instrumentalization of design features and individual and collective instrumentation of the LA dashboard. The analysis of semi-structured individual interviews with five nursing teachers who used an LA dashboard to guide team reflections with 224 students (56 teams) revealed technical and pedagogical changes that occurred in both the system’s features (instrumentalization) and teaching practices (instrumentation). We found that teachers adopted the LA dashboard beyond initially intended ways by (i) providing emotional support with the analytics, (ii) reducing details in AI-powered data visualizations for easier comprehension, (iii) creating data narratives to address data limitations, and (iv) collectively developing new practices to use the LA dashboard for co-teaching. Therefore, teachers’ design-in-use of the LA dashboard highlights the ongoing need for design improvements to address challenges posed by dynamic data and complex algorithms underlying AI and analytics interfaces.

Learning Analytics Dashboard Design and Evaluation to Support Student Self-Regulation of Study Behaviour

For university students, self-regulation of study behaviour is important. However, students are not always capable of effective self-regulation. Providing study behaviour information via a learning analytics dashboard (LAD) may support phases within self-regulated learning (SRL). However, it is unclear what information a LAD should provide, how to present information in a usable manner, and what the information’s perceived usefulness is in supporting self-regulation of study behaviour. This study entails a sequential mixed design: assessing information needs in focus groups (n=7), exploring usability via think-aloud interviews (n=8), assessing usability with the System Usability Scale (n=42), and assessing perceived usefulness via interviews (n=16). Results showed that students and tutors agreed on the relevance of the constructs chosen from literature but differed in ranking the importance of new constructs. Usability exploration led to several design improvements. Perceived usefulness assessment showed the LAD supported the appraisal of study behaviour. A need for reference frames to facilitate data interpretation was vocalized. Impacts on study behaviour varied, possibly because preparatory activities were not used. Impact could be improved by further integrating the LAD into existing learning processes.

Promoting higher education students’ self-regulated learning through learning analytics: A qualitative study

AbstractLearning analytics provides a novel means to support the development and growth of students into self-regulated learners, but little is known about student perspectives on its utilization. To address this gap, the present study proposed the following research question: what are the perceptions of higher education students on the utilization of a learning analytics dashboard to promote self-regulated learning? More specifically, this can be expressed via the following threefold sub-question: how do higher education students perceive the use of a learning analytics dashboard and its development as promoting the (1) forethought, (2) performance, and (3) reflection phase processes of self-regulated learning? Data for the study were collected from students (N = 16) through semi-structured interviews and analyzed using a qualitative content analysis. Results indicated that the students perceived the use of the learning analytics dashboard as an opportunity for versatile learning support, providing them with a means to control and observe their studies and learning, while facilitating various performance phase processes. Insights from the analytics data could also be used in targeting the students’ development areas as well as in reflecting on their studies and learning, both individually and jointly with their educators, thus contributing to the activities of forethought and reflection phases. However, in order for the learning analytics dashboard to serve students more profoundly across myriad studies, its further development was deemed necessary. The findings of this investigation emphasize the need to integrate the use and development of learning analytics into versatile learning processes and mechanisms of comprehensive support and guidance.

More, better feedback please: are learning analytics dashboards (LAD) the solution to a wicked problem?

There is a long-standing lack of learner satisfaction with quality and quantity of feedback in health professions education (HPE) and training. To address this, university and training programmes are increasingly using technological advancements and data analytic tools to provide feedback. One such educational technology is the Learning Analytic Dashboard (LAD), which holds the promise of a comprehensive view of student performance via partial or fully automated feedback delivered to learners in real time. The possibility of displaying performance data visually, on a single platform, so users can access and process feedback efficiently and constantly, and use this to improve their performance, is very attractive to users, educators and institutions. However, the mainstream literature tends to take an atheoretical and instrumentalist view of LADs, a view that uncritically celebrates the promise of LAD's capacity to provide a 'technical fix' to the 'wicked problem' of feedback in health professions education. This paper seeks to recast the discussion of LADs as something other than a benign material technology using the lenses of Miller and Rose's technologies of government and Barry's theory of Technological Societies, where such technical devices are also inherently agentic and political. An examination of the purpose, design and deployment of LADs from these theoretical perspectives can reveal how these educational devices shape and govern the HPE learner body in different ways, which in turn, may produce a myriad of unintended- and ironic- effects on the feedback process. In this Reflections article we wish to encourage health professions education scholars to examine the practices and consequences thereof of the ever-expanding use of LADs more deeply and with a sense of urgency.

Co‐creating an equality diversity and inclusion learning analytics dashboard for addressing awarding gaps in higher education

AbstractEducational outcomes from traditionally underrepresented groups are generally worse than for their more advantaged peers. This problem is typically known as the awarding gap (we use the term awarding gap over ‘attainment gap’ as attainment places the responsibility on students to attain at equal levels) and continues to pose a challenge for educational systems across the world. While Learning Analytics (LA) dashboards help identify patterns contributing to the awarding gap, they often lack stakeholder involvement, offering very little support to institutional Equality, Diversity and Inclusion (EDI) leads or educators to pinpoint and address these gaps. This paper introduces an innovative EDI LA dashboard, co‐created with diverse stakeholders. Rigorously evaluated, the dashboard provides fine‐grained insights and course‐level analysis, empowering institutions to effectively address awarding gaps and contribute to a diverse and inclusive higher education landscape. Practitioners notesWhat is already known about this topic Traditionally underrepresented groups face educational disparities, commonly known as the awarding gap. Underachievement is a complex multi‐dimensional problem and cannot be solely attributable to individual student deficiencies. LA dashboards targeting this specific problem are often not public, there is little research about them, and are frequently designed with little involvement of educational stakeholders. What this paper adds Pioneers the introduction of a dashboard specifically designed to address the awarding gap problem. Emphasises the significant data needs of educational stakeholders in tackling awarding gaps. Expands the design dimensions of Learning Analytics (LA) by introducing a specific design approach rooted in established user experience (UX) design methods. Implications for practice and/or policy Insights from this study will guide practitioners, designers, and developers in creating AI‐based educational systems to effectively target the awarding gap problem.

Ready or not? Investigating in-service teachers’ integration of learning analytics dashboard for assessing students’ collaborative problem solving in K–12 classrooms

AbstractCollaborative problem solving (CPS) has emerged as a crucial 21st century competence that benefits students’ studies, future careers, and general well-being, prevailing across disciplines and learning approaches. Given the complex and dynamic nature of CPS, teacher-facing learning analytics dashboards (LADs) have increasingly been adopted to support teachers’ CPS assessments by analysing and visualising various dimensions of students’ CPS. However, there is limited research investigating K-12 teachers’ integration of LADs for CPS assessments in authentic classrooms. In this study, a LAD was implemented to assist K-12 teachers in assessing students’ CPS skills in an educational game. Based on the person-environment fit theory, this study aimed to (1) examine the extent to which teachers’ environmental and personal factors influence LAD usage intention and behaviour and (2) identify personal factors mediating the relationships between environmental factors and LAD usage intention and behaviour. Survey data of 300 in-service teachers from ten Chinese K-12 schools were collected and analysed using partial least squares structural equation modelling (PLS-SEM). Results indicated that our proposed model showed strong in-sample explanatory power and out-of-sample predictive capability. Additionally, subjective norms affected technological pedagogical content knowledge (TPACK) and self-efficacy, while school support affected technostress and self-efficacy. Moreover, subjective norms, technostress, and self-efficacy predicted behavioural intention, while school support, TPACK, and behavioural intention predicted actual behaviour. As for mediation effects, school support indirectly affected behavioural intention through self-efficacy, while subjective norms indirectly affected behavioural intention through self-efficacy and affected actual behaviour through TPACK. This study makes theoretical, methodological, and practical contributions to technology integration in general and LAD implementation in particular.

Learning Analytics Dashboards for Assessing Remote Labs Users' Work: A Case Study with VISIR-DB

Learning Analytics Dashboards for Assessing Remote Labs Users' Work: A Case Study with VISIR-DB

Designing data-informed support for building learning habits in the Japanese K12 context

While building proper learning habits has been said to enhance academic performance, it is challenging to give long-term support for building habits in educational contexts due to the lack of continuous tracing of one’s habitual behaviors. With the accumulation of learning logs and the advancement of Learning Analytics (LA) techniques, this paper illustrates the data-informed support for building learning habits, which involves persuading one to change behaviors. Specifically, we tackled 2 research objectives following the Persuasive Systems Design (PSD) model. First, we defined indicators of learning habits from log data and analyzed 115,340 learning logs of 96 learners from a Japanese junior high school. As a result, the learners’ types of habits could be detected even though some might not be efficient and the stages of habits fluctuated over time. We also identified differences when comparing the learning habits extracted from the log data with those reported in the questionnaire. Second, based on the understanding of the learner profiles, we designed elements of an LA dashboard to support habit-building by applying the design principles from the PSD model. Overall, the learners recognized the feasibility of integrating data-informed support into their daily learning. Therefore, we look forward to the evidence of its effectiveness on the behavior change that can be depicted by the transition between stages of different types of learning habits.

Applying Kano’s two-factor theory to prioritize learning analytics dashboard features for learning technology designers

Existing methods for software requirements elicitation, five-point Likert scales and voting methods for requirements prioritization, and usability and user experience evaluation methods do not enable prioritizing the learning analytics dashboard requirements. Inspired by management and product design field, this research applies Kano’s two-factor theory to prioritize the features of learning analytics dashboards (LADs) of adaptive learning platform (ALP) called Rhapsode<sup>TM</sup> learner, based on students’ perceived usefulness to support designers’ decision-making. Comparing usability and user experience methods for evaluating LAD features, this paper contributes with the protocol and a case applying Kano method for evaluating the perceived importance of the dashboards in ALP. The paper applies Kano’s two-factor questionnaire on the 13 LADs features of Rhapsode<sup>TM</sup> learner. Responses from 17 students are collected using a questionnaire, which is used to showcase the strength of the two-factor theory through five tabular and graphical techniques. Through these five tabular and graphical techniques, we demonstrate the application and usefulness of the method as designers and management are often carried away by the possibilities of insights instead of actual usefulness. The results revealed a variation in the categorization of LADs depending on the technique employed. As the complexity of the techniques increases, additional factors that indicate data uncertainty are gradually incorporated, clearly highlighting the growing requirement for data. In the case of RhapsodeTM learner platform, results based on the students responses show that 11 of 13 LADs being excluded due to low significance level in categorization (technique 1) and low response rate.

Learning analytics for student homework activities during a long break: Evidence from K-12 education in Japan

Learning Analytics (LA) is an emergent field that aims to better understand students and provide intelligence to learners, teachers, and administrators using learning log data. Although the use of technology in class is increasing in the K-12 sector and tertiary education, cases of effective implementation of LA in secondary schools have rarely been reported. This study offers an example of LA implemented in a junior high Math class during long vacations in Japan. This paper comprises two studies: first, we analyzed 121 students’ answer logs and their exam performance after vacation by the K-means clustering method. We found that students’ progress patterns were categorized into four types of engagement—early, late, high, and low—and the early and high-engagement groups obtained significantly higher scores than the low-engagement group. In the second study, we implemented a real-time dashboard that visualizes students’ progress patterns and gives students insights about their progress during the vacation period. We found that the dashboard significantly increased students’ interactions with the assignment, and the questionnaire survey determined that the LA dashboard motivated students to learn during the long vacation period. Considering the previous studies of LA, we estimate that LA-based interventions enhance students’ self-regulation skills, which is crucial for learning during long vacation periods. Our study offers a novel approach to implementing LA in K-12 education.

The effects of visualisation literacy and data storytelling dashboards on teachers’ cognitive load

Learning analytics (LA) dashboards are becoming increasingly available in various learning settings. However, teachers may face challenges in understanding and interpreting the data visualisations presented on those dashboards. In response to this, some LA researchers are incorporating visual cueing techniques, like data storytelling (DS), into LA dashboard design to reduce the data visualisation skills – often referred to as visualisation literacy (VL) – and cognitive effort required by teachers to effectively use dashboards. However, despite the potential of DS principles in simplifying data visualisations, there is limited evidence supporting their effectiveness in actually reducing teachers’ cognitive load. The study presented in this paper addresses this gap by investigating the potential impact of LA dashboards, with and without DS elements, on teachers with varying VL levels. Through a quasi-experimental study involving 23 teachers, we analysed changes in pupil dilation – a proxy for cognitive load – as they examined LA dashboards featuring student data captured while participating in synchronous, online collaborative learning tasks. Our findings suggest DS can reduce cognitive load, particularly for teachers with lower VL. These results provide insight into the effects of DS and VL on teachers’ cognitive load, thereby informing the design of LA dashboards. Implications for practice or policy:• Developers of LA dashboards need to pay more attention to incorporating visual and narrative elements that are easily comprehensible and target-oriented, based on users’ visualisation literacy levels.• Educational providers and LA designers can recommend dashboards with DS elements to teachers with low VL to enhance their work efficiency.

Investigation and the development of learning analytics dashboard in open and distance learning using big data mining

<p class="Abstract">The main aim of this study is to provide universities with a way of examining and predicting student performance. The fundamental aim and purpose of this study is to help academic institutions to analyse and predict student performance. The credibility and accuracy of the model was examined by comparing the predicted results of the model with the observed values. And educational data mining techniques were used to create student profiles. Weighted gain, classification analysis, decision tree and rule induction were used in this study. The results of the study showed that the level of students' academic performance varied according to criteria such as academic structure, faculty, mode of enrolment and gender. In order to determine the relative importance of variables, the information weight gain technique was used after generating rule induction parameters and hidden rules between data. Using data mining techniques, we can obtain both guidelines to instruct students and information to help us identify them.</p>

Earning Analytics Dashboards for Advisors – A Systematic Literature Review

Learning Analytics Dashboard for Advisors is designed to provide data-driven insights and visualizations to support advisors in their decision-making regarding student academic progress, engagement, targeted support, and overall success. This study explores the current state of the art in learning analytics dashboards, focusing on specific requirements for advisors. By examining existing literature and case studies, this research investigates the key features and functionalities essential for an effective learning analytics dashboard tailored to advisor needs. This study also aims to provide a comprehensive understanding of the landscape of learning analytics dashboards for advisors, offering insights into the advancements, opportunities, and challenges in their development by synthesizing the current trends from a total of 21 research papers used for analysis. The findings will contribute to the design and implementation of new features in learning analytics dashboards that empower advisors to provide proactive and individualized support, ultimately fostering student retention and academic success.

A Current Overview of the Use of Learning Analytics Dashboards

The promise of Learning Analytics Dashboards in education is to collect, analyze, and visualize data with the ultimate ambition of improving students’ learning. Our overview of the latest systematic reviews on the topic shows a number of research trends: learning analytics research is growing rapidly; it brings to the front inequality and inclusiveness measures; it reveals an unclear path to data ownership and privacy; it provides predictions which are not clearly translated into pedagogical actions; and the possibility of self-regulated learning and game-based learning are not capitalized upon. However, as learning analytics research progresses, greater opportunities lie ahead, and a better integration between information science and learning sciences can bring added value of learning analytics dashboards in education.

Learning analytics dashboards are increasingly becoming about learning and not just analytics - A systematic review

This systematic review explores the emerging themes in the design and implementation of student-facing learning analytics dashboards in higher education. Learning Analytics has long been criticised for focusing too much on the analytics, and not enough on the learning. The review is then guided by an interest in whether these dashboards are still primarily analytics-driven or if they have become pedagogically informed over time. By mapping the identified themes of technological maturity, informing frameworks, affordances, data sources, and analytical levels over publications per year, the review identifies an emerging trajectory towards student-focused dashboards. These dashboards are informed by theory-oriented frameworks, designed to incorporate affordances that supporting student learning, and realised through integration of more than just activity data from learning management systems – allowing the dashboards to better support students' learnings processes. Based on this emerging trajectory, the review provides a series of design recommendations for student-focused dashboards that are connected to learning sciences as well as analytics.

Review of Literature on the Use of Learning Analytics and Learning Analytical Dashboard (LAD) in Improving Student Performance in Higher Education Institutions in Kenya

Purpose – The purpose of the paper is mainly to review the various literature done on Learning Analytics in higher education institutions in Kenya. From the review it aims at identifying the current status of Learning Analytics in Kenya and proposing recommendations for improvement of the same. Methodology – The paper uses desktop approach to breakdown the various studies done in Kenya on Learning Analytics in higher education institutions in Kenya and identifies any research gaps/ areas for improvement from the review of literature. Value – The analysis is valuable to Higher Education institutions in Kenya in coming up with a model that will be used to improve student performance in blended learning through use of learning analytics and learning analytics dashboard. Findings – the findings of the study revealed that there was limited research on learning analytics in Kenya. Moreover, the current study had not used raw data such as the behavioral pattern of students. Therefore, future study can be done using the raw data from Moodle to develop a model for improving student performance.

Transforming Education in the World of Artificial Intelligence

Introduction: Artificial Intelligence (AI) and Machine Learning (ML) are key drivers of innovation and growth among all industries, and the education sector is no exemption. AI is considered as a powerful tool to facilitate new examples for technological development, instructional design and educational research that are otherwise not possible to develop in the traditional education techniques. With the development in information processing and computing techniques, artificial intelligence has been widely applied in educational practices (Artificial Intelligence in Education; AIEd), such as teaching robots, human-computer interactions intelligent tutoring systems, learning analytics dashboards and adaptive learning systems. It has the ability to maximize both teaching and learning, helping the education sector to evolve for better thus benefitting students and teachers both

Towards Optimization of Learning Analytics Dashboards That are Customized for the Students’ Requirements

Towards Optimization of Learning Analytics Dashboards That are Customized for the Students’ Requirements

Student-Facing Learning Analytics Dashboard for Remote Lab Practical Work

Student-Facing Learning Analytics Dashboard for Remote Lab Practical Work