Design Of Learning Analytics Research Articles

Dialogic feedback at scale: Recommendations for learning analytics design

AbstractBackgroundFeedback is central to formative assessments but aligns with a one‐way information transmission perspective obstructing students' effective engagement with feedback. Previous research has shown that a responsive, dialogic feedback process that requires educators and students to engage in ongoing conversations can encourage student active engagement in feedback. However, it is challenging with larger student cohorts. Learning Analytics (LA) provides promising ways to facilitate timely feedback at scale by leveraging large datasets generated during students' learning. However, current LA design and implementation tend to treat feedback as a one‐way transmission rather than a two‐way process.ObjectivesThis case study aims to improve LA design and practice to align with dialogic feedback principles by exploring an authentic dialogic feedback practice at scale.MethodsWe explored a dialogic feedback practice of a course having 700 undergraduate students. The case study used quantitative and qualitative analysis methods to investigate what students expect from feedback, how educators respond to students' feedback requests, and how students experience feedback.Results and ConclusionsThe results emphasise the need to focus on cognitive, relational and emotional aspects of the feedback process. In aligning LA with dialogic feedback principles, we propose that LA should promote the following objectives: reflection, adaption, personalisation, emotional management, and scaffolding feedback provision.

The synergistic and dynamic relationship between learning design and learning analytics

The synergistic relationship between learning design and learning analytics has the potential for improving learning and teaching in near real-time. The potential for integrating the newly available and dynamic information from ongoing analysis into learning design requires new perspectives on learning and teaching data processing and analysis as well as advanced theories, methods, and tools for supporting dynamic learning design processes. Three perspectives of learning analytics design provide summative, real-time, and predictive insights. In a case study with 3,550 users, the navigation sequence and network graph analysis demonstrate the potential of learning analytics design. The study aims to demonstrate how the analysis of navigation patterns and network graph analysis could inform the learning design of self-guided digital learning experiences. Even with open-ended freedom, only 608 sequences were evidenced by learners out of a potential number of hundreds of millions of sequences. Advancements of learning analytics design have the potential for mapping the cognitive, social and even physical states of the learner and optimise their learning environment on the fly.

Learning analytics with iLearn Insights

iLearn Insights is an innovative online application to address the challenge of engaging, motivating and supporting students with automated personalised email for those at risk of failure. It allows academics to identify students who are engaging or otherwise with the Learning Management System (LMS) content, thus allowing early intervention, and ultimately improved retention in units by providing targeted support to a student cohort. In contrast to systems that provide only static data reports, this vendor-independent innovation was built in-house to analyse and visualise student learning data in relation to access patterns, forum activity, learning activity submission and grades to identify a student cohort that needs support, as well as enabling teaching staff to send personalised emails to students on the basis of their level of engagement and/or performance. Feedback has a powerful impact on learning, but students frequently highlight it as an area that can be improved in tertiary education (Dawson and Henderson, 2019). iLearn Insights provides graphical representation of LMS data that enables visual personalised feedback to students. Within three mouse clicks, academic staff can trigger a range of automated communications to commend high-achieving students, offer additional assistance to lower performing students or to recapture disengaged students. These emails can be a targeted to a group of students and configured by the unit convenor to contain motivating information including the top five resources accessed by classmates; a student’s mark in comparison to the class average; number or percentage of students that have already submitted an assignment; clickable links; and support resources for students falling behind. ILearn Insights was developed based on four principals of learning analytics design knowledge: integration, agency, reference frame and dialogue (Wise, 2014). It has been observed that targeted visual feedback with clickable links is the most effective way to engage students quickly.?The positive impact of iLearn Insights is demonstrated by its rapid uptake by teaching staff across Macquarie University. When it launched in Session 1, 2020, after 18 months of piloting, there were 478 users across 763 units (subjects) who sent 125334 targeted personalised emails. In Session 1, 2023 iLearn Insights was used by approximately 808 users across 40 departments and learning support areas and 1237 units, sending over 316576 targeted emails to encourage students to engage with learning activities or offer support. That represents an increase of 169% of users, 162% of units and 253% of emails in three years. These personalised email exchanges have led to enhanced student engagement, which is critical for student success (Kahu and Nelson, 2018; De Villiers and Werner, 2018; McClenney et al, 2012; Klem and Connell, 2004).

Leveraging cultural forms in human‐centred learning analytics design

AbstractIn this article, we offer theory‐grounded narratives of a 4‐year participatory design process of a Learning Analytics tool with K‐12 educators. We describe how we design‐in‐partnership by leveraging educators' routines, values and cultural representations into the designs of digital dashboards. We make our long‐term reasoning visible by reflecting upon how design decisions were made, discussing key tensions and analysing to what extent the developed tools were taken up in practice. Through thick design narratives, we reflect upon how cultural forms—recognizable cultural constructs that might cue and facilitate specific activities—were identified among educators and informed the design of a dashboard. We then examined the extent to which the designed tool supported coaches and teachers to engage in Generative Uncertainty, an interpretive stance in which educators manifest productive inquiries towards data. Our analysis highlights that attuning to cultural forms is a valuable first step but not enough towards designing LA tools for systems in ways that fit institutionalized practices, challenge instrumental uses and spur productive inquiry. We conclude by offering two key criteria for making culturally‐grounded design decisions in the context of long‐term partnerships. Practitioner notesWhat is already known about this topic Participatory design can invite stakeholders to directly inform the creation of LA artefacts that fit their needs, context and cultural markers. What this paper adds Cultural forms can be identified and leveraged in the design of LA tools. HCLA scholars ought to design for systems—the complex body of organizational routines, cultural practices and interactions among multiple stakeholders—and not just for users. Implications for practice and/or policy Leveraging cultural forms in LA needs to be accompanied by a critical view of which practices, behaviours, values and structures are suggested by such forms. Designing features that are easy to use, are associated with concrete tasks, and fit into existing cultural practices are three criteria for embedding cultural forms into LA design.

A human‐centred learning analytics approach for developing contextually scalable K‐12 teacher dashboards

AbstractThis paper describes a Human‐Centred Learning Analytics (HCLA) design approach for developing learning analytics (LA) dashboards for K‐12 classrooms that maintain both contextual relevance and scalability—two goals that are often in competition. Using mixed methods, we collected observational and interview data from teacher partners and assessment data from their students' engagement with the lesson materials. This DBR‐based, human‐centred design process resulted in a dashboard that supported teachers in addressing their students' learning needs. To develop the dashboard features that could support teachers, we found that a design refinement process that drew on the insights of teachers with varying teaching experience, philosophies and teaching contexts strengthened the resulting outcome. The versatile nature of the approach, in terms of student learning outcomes, makes it useful for HCLA design efforts across diverse K‐12 educational contexts. Practitioner notesWhat is already known about this topic Learning analytics that are aligned to both a learning theory and learning design support student learning. LA dashboards that support users to understand the associated learning analytics data provide actionable insight. Design‐based research is a promising methodology for Human‐Centred Learning Analytics design, particularly in the K‐12 educational context. What this paper adds Leveraging a longstanding, yet fluid, research‐practice partnership is an effective design‐based research adaptation for addressing the high variation in instructional practices that characterize K‐12 education. Using both quantitative and qualitative data that reflects students' developing knowledge effectively supports teachers' inquiry into student learning. Teachers' use of learning analytics dashboards is heavily influenced by their perspectives on teaching and learning. Implications for practice and/or policy Impact on student learning outcomes, alongside usability and feasibility, should be included as a necessary metric for the effectiveness of LA design. LA dashboard developers should both leverage learning data that reflect students' developing knowledge and position teachers to take responsive pedagogical action to support student learning. LA researchers and developers should utilize a long‐term, yet fluid, research‐practice partnership to form a multi‐stakeholder, multidisciplinary design team for Human‐Centred Learning Analytics design.

Closing the Experiential Learning Loops Using Learning Analytics Cycle

In ubiquitous learning, authentic experiences are captured and later reused as those are rich resources for foreign vocabulary development. This article presents an experiential theory-oriented approach to the design of learning analytics support for sharing and reusing authentic experiences. In this regard, first, a conceptual framework to support vocabulary learning using learners' authentic experiences is proposed. Next, learning experiences are captured using a context-aware ubiquitous learning system. Finally, grounded in the theoretical framework, the development of a web-based tool called learn from others (LFO) panel is presented. The LFO panel analyzes various learning logs (authentic, partially-authentic, and words) using the profiling method while determining the top-five learning partners inside a seamless learning analytics platform. This article contributes to the research in the area of theory-oriented design of learning analytics for vocabulary learning through authentic activities and focuses on closing the loops of experiential learning using learning analytics cycles.

Informing learning design through analytics: Applying network graph analysis

Learning design has traditionally been thought of as an activity occurring prior to the presentation of a learning experience or a description of that activity. With the advent of near real-time data and new opportunities of representing the decisions and actions of learners in digital learning environments, learning designers can now apply dynamic learning analytics information on the fly in order to evaluate learner characteristics, examine learning designs, analyse the effectiveness of learning materials and tasks, adjust difficulty levels, and measure the impact of interventions and feedback. In a case study with 3550 users, the navigation sequence and network graph analysis demonstrate a potential application of learning analytics design. Implications based on the case study show that integration of analytics data into the design of learning environments is a promising approach.

Designing Effective Digital Learning Environments: Toward Learning Analytics Design

Designing Effective Digital Learning Environments: Toward Learning Analytics Design

A Novel Web-Based Approach for Visualization and Inspection of Reading Difficulties on University Students

Existing tools aim to detect university students with early diagnosis of dyslexia or reading difficulties, but there are not developed tools that let those students better understand some aspects of their difficulties. In this paper, a dashboard for visualizing and inspecting early detected reading difficulties and their characteristics, called PADA (acronym for the Spanish name Panel de Analiticas de Aprendizaje de Dislexia en Adultos ), is presented. PADA is a web-based tool designed to facilitate the creation of descriptive visualizations required for a better understanding by students about their learner model. Through information visualization techniques, PADA shows students the knowledge in their learner models in order to help them to increase their awareness and to support reflection and self-regulation about their difficulties in reading. PADA provides different learning analytics on reading performance of students, so that they can self-identify their strengths and weaknesses and self-regulate their learning. This paper describes examples that cover a variety of visualizations (bar-charts, line-charts, and pie-charts) to show user model fragments as personal details, reading profiles, learning styles, and cognitive traits of the students. We tested PADA with 26 students (aged 21–53 years) of different academic programs and levels, dyslexic and non-dyslexic. The results show that PADA can assist students in creating awareness, and help them to understand their difficulties associated with the reading tasks, as well as facilitate reflection and self-regulation in the learning process. Implications for the design of learning analytics are discussed and directions for future work are outlined.

Privacy-driven Design of Learning Analytics Applications – Exploring the Design Space of Solutions for Data Sharing and Interoperability

Studies have shown that issues of privacy, control of data, and trust are essential to implementation of learning analytics systems. If these issues are not addressed appropriately systems will tend to collapse due to legitimacy crisis, or they will not be implemented in the first place due to resistance from learners, their parents, or their teachers. This paper asks what it means to give priority to privacy in terms of data exchange and application design and offers a conceptual tool, a Learning Analytics Design Space model, to ease the requirement solicitation and design for new learning analytics solutions. The paper argues the case for privacy-driven design as an essential part of learning analytics systems development. A simple model defining a solution as the intersection of an approach, a barrier, and a concern is extended with a process focussing on design justifications to allow for an incremental development of solutions. This research is exploratory of nature, and further validation is needed to prove the usefulness of the Learning Analytics Design Space model.

Developing Learning Analytics Design Knowledge in the “Middle Space”: The Student Tuning Model and Align Design Framework for Learning Analytics Use

This paper addresses a relatively unexplored area in the field of learning analytics: how analytics are taken up and used as part of teaching and learning processes. Initial steps are taken towards developing design knowledge for this “middle space,” with a focus on students as analytics users. First, a core set of challenges for analytics use identified in the literature are compiled. Then, a process model is presented for conceptualizing students’ learning analytics use as part of a self-regulatory cycle of grounding, goal-setting, action and reflection–the Student Tuning Model. Finally, the Align Design Framework is presented with initial validation as a tool for pedagogical design that addresses the identified challenges and supports students’ use of analytics as part of the tuning process. Together, the framework’s four interconnected principles of Integration, Agency, Reference Frame and Dialogue / Audience provide a useful starting point for further inquiry into well-designed learning analytics implementations.