Implementation Of Learning Analytics Research Articles

A Systematic Review of Learning Analytics

The learning management system (LMS) is widely used in educational settings to support teaching and learning practices. The usage log data, generated by both learners and instructors, enables the development and implementation of learning analytics (LA) interventions aimed at facilitating teaching and learning activities. To examine the current status of the development and empirical impacts of learning analytics–incorporated interventions within LMSs on improving teaching and learning practices, we conducted a systematic review that examined 27 articles published from 2012 through 2023. The outcomes of this review provided valuable insights into the design and development of learning analytics–incorporated interventions implemented on LMSs and empirical evidence of the impacts of these interventions, along with implications to inform future design and applications.

“Trust us,” they said. Mapping the contours of trustworthiness in learning analytics

Purpose The purpose of this paper is to explore and establish the contours of trust in learning analytics and to establish steps that institutions might take to address the “trust deficit” in learning analytics. Design/methodology/approach “Trust” has always been part and parcel of learning analytics research and practice, but concerns around privacy, bias, the increasing reach of learning analytics, the “black box” of artificial intelligence and the commercialization of teaching and learning suggest that we should not take stakeholder trust for granted. While there have been attempts to explore and map students’ and staff perceptions of trust, there is no agreement on the contours of trust. Thirty-one experts in learning analytics research participated in a qualitative Delphi study. Findings This study achieved agreement on a working definition of trust in learning analytics, and on factors that impact on trusting data, trusting institutional understandings of student success and the design and implementation of learning analytics. In addition, it identifies those factors that might increase levels of trust in learning analytics for students, faculty and broader. Research limitations/implications The study is based on expert opinions as such there is a limitation of how much it is of a true consensus. Originality/value Trust cannot be assumed is taken for granted. This study is original because it establishes a number of concerns around the trustworthiness of learning analytics in respect of how data and student learning journeys are understood, and how institutions can address the “trust deficit” in learning analytics.

Are MOOC learning designs culturally inclusive (enough)?

AbstractBackgroundExtensive research on massive open online courses (MOOCs) has focused on analysing learners' behavioural trace data to understand navigation and activity patterns, which are known to vary systematically across geo‐cultural contexts. However, the perception of learners regarding the role of different learning design elements in sustaining their engagement in the course is still unclear.ObjectivesThis study aimed to examine learners' perception of learning design elements in MOOCs and explore the ways in which these perceptions differ between geo‐cultural contexts.MethodsWe conducted interviews with 22 learners from seven geo‐cultural regions to gather insights into their learning design preferences.ResultsOur findings indicate that learners from regions such as South Asia exhibit a strong inclination towards video‐based content and a lesser preference for reading textual resources. In contrast, learners from regions such as Anglo‐Saxon demonstrate a high preference for reading texts such as articles and video transcripts.ConclusionThe observed variations in self‐reported interests in various learning design elements raise intriguing questions about the nature and extent of participation of various geo‐cultural groups. This study underscores the need to develop inclusive MOOC designs and implement learning analytics approaches that adapt to the cultural preferences of learners.

Single-case learning analytics: Feasibility of a human-centered analytics approach to support doctoral education

Recent advances in machine learning and natural language processing have the potential to transform human activity in many domains. The field of learning analytics has applied these techniques successfully to many areas of education but has not been able to permeate others, such as doctoral education. Indeed, doctoral education remains an under-researched area with widespread problems (high dropout rates, low mental well-being) and lacks technological support beyond very specialized tasks. The inherent uniqueness of the doctoral journey may help explain the lack of generalized solutions (technological or otherwise) to these challenges. We propose a novel approach to apply the aforementioned advances in computation to support doctoral education. Single-case learning analytics defines a process in which doctoral students, researchers, and computational elements collaborate to extract insights about a single (doctoral) learner's experience and learning process. The feasibility and added value of this approach are demonstrated using an authentic dataset collected by nine doctoral students over a period of at least two months. The insights from this exploratory proof-of-concept serve to spark a research agenda for future technological support of doctoral education, which is aligned with recent calls for more human-centred approaches to designing and implementing learning analytics technologies.

Using networked learning to improve learning analytics implementation

Using networked learning to improve learning analytics implementation

Using learning analytics in clinical competency committees: Increasing the impact of competency-based medical education

ABSTRACT Graduate medical education (GME) and Clinical Competency Committees (CCC) have been evolving to monitor trainee progression using competency-based medical education principles and outcomes, though evidence suggests CCCs fall short of this goal. Challenges include that evaluation data are often incomplete, insufficient, poorly aligned with performance, conflicting or of unknown quality, and CCCs struggle to organize, analyze, visualize, and integrate data elements across sources, collection methods, contexts, and time-periods, which makes advancement decisions difficult. Learning analytics have significant potential to improve competence committee decision making, yet their use is not yet commonplace. Learning analytics (LA) is the interpretation of multiple data sources gathered on trainees to assess academic progress, predict future performance, and identify potential issues to be addressed with feedback and individualized learning plans. What distinguishes LA from other educational approaches is systematic data collection and advanced digital interpretation and visualization to inform educational systems. These data are necessary to: 1) fully understand educational contexts and guide improvements; 2) advance proficiency among stakeholders to make ethical and accurate summative decisions; and 3) clearly communicate methods, findings, and actionable recommendations for a range of educational stakeholders. The ACGME released the third edition CCC Guidebook for Programs in 2020 and the 2021 Milestones 2.0 supplement of the Journal of Graduate Medical Education (JGME Supplement) presented important papers that describe evaluation and implementation features of effective CCCs. Principles of LA underpin national GME outcomes data and training across specialties; however, little guidance currently exists on how GME programs can use LA to improve the CCC process. Here we outline recommendations for implementing learning analytics for supporting decision making on trainee progress in two areas: 1) Data Quality and Decision Making, and 2) Educator Development.

Frameworks of Designing and Implementing Learning Analytics in Educational Institutions: A Review

Frameworks of Designing and Implementing Learning Analytics in Educational Institutions: A Review

Higher Education Stakeholders’ Views on Guiding the Implementation of Learning Analytics for Study Success


 
 
 Learning analytics show promise to support study success in higher education. Hence, they are increasingly adopted in higher education institutions. This study examines higher education stakeholders’ views on learning analytics utilisation to support study success. Our main research question was to investigate how ready higher education institutions are to adopt learning analytics. We derived policy guidelines from an international systematic review of the last five years of learning analytics research. Due to the lack of rigorous learning analytics research and adoption, this study examines how ready university stakeholders are to adopt learning analytics. In order to validate the guidelines, we conducted an interview study with 37 higher education stakeholders. The majority of participants stated that their institutions required further resources in order to adopt learning analytics but were able to identify what these resources were in order for successful implementation. Overall, stakeholders agree that learning analytics show much promise to support study success at higher education institutions.
 
 

Adaptability and Procedural Content Generation for Educational Escape Rooms

We present a literature review that aims to understand the role of the Educational Escape Room (EER) in improving the teaching, learning, and assessment processes through an EER design framework. The main subject is to identify the recent interventions in this field in the last five years. Our study focuses on understanding how it is possible to create an EER available to all students, namely visually challenged users. As a result of the implementation of new learning strategies that promote autonomous learning, a concern arose in adapting educational activities to each student’s individual needs. To study the adaptability of each EER, we found the EER design framework essential to increase the student experience by promoting the consolidation of knowledge through narrative and level design. The results of our study show evidence of progress in students’ performance while playing an EER, revealing that students' learning can be effective. Research on Procedural Content Generation (PCG) highlighted how important it is to implement adaptability in future studies of EERs. However, we found some limitations regarding the process of evaluating learning through the EERs, showing how important it is to study and implement learning analytics in future studies in this field.

Implementing learning analytics in wiki-supported collaborative learning in secondary education: A framework-motivated empirical study

Implementing learning analytics in wiki-supported collaborative learning in secondary education: A framework-motivated empirical study

Implementation of Learning Analytics in Primary and Secondary School: A Systematic Literature Review

Increased adoption of educational technologies, the emergence of digital classroom concepts, and the interest in big data innovations have led to a growing awareness of the potential implementation of learning analytics to support learning development in educational institutions. However, most studies have focused on the implementation of learning analytics in higher education. As a result, research evidence and studies on actual and ongoing implementation in pre-higher education are still scarce. Therefore, there is a need to understand better the implementation of learning analytics at primary and secondary from the current learning analytics literature. This systematic literature review (SLR) aimed to identify learning analytics research that focuses on implementing learning analytics at pre-higher education levels, including in pre-school, primary, or secondary school. This SLR was carried out based on the SALSA framework to determine the protocol, search, appraisal, synthesis, analysis, and reporting approaches. The findings of the SLR support the arguments that the implementation of learning analytics in school is still in its infancy, where the implementation has been observed mostly in developed countries. Most of the Implementation is aimed at descriptive analytics, focusing on the purpose of monitoring, analysis, and feedback. The literature review has shown a lack of research on the implementation of learning analytics at the primary and secondary education levels.

Stakeholders’ insights on learning analytics: Perspectives of students and staff

Learning analytics has drawn the attention of academics and administrators in recent years as a tool to better understand students' needs and to tailor appropriate and timely responses. While many value the potential of learning analytics, it is not without critics, especially with regards to ethical concerns surrounding the level and type of data gathered, and scepticism on data's ability to measure something useful and actionable. This paper gathers the thoughts of key stakeholders, including staff and students, and their expectations of learning analytics, their priorities for using student data, and how they should be supported to act on the data, with the aim of aiding institutions with their plans to implement learning analytics. For this analysis we explored stakeholder awareness, concerns, priorities and support needs with respect to effectively accessing, interpreting and utilising learning data through the use of surveys and focus groups. These were adapted from the previously published SHEILA framework protocols with additional topics added relating to awareness, uses of data, and support. The focus groups were used to capture prevalent themes, followed by surveys to gain perspectives on these themes from a wider stakeholder audience. Overall, results suggest there are significant differences in the perspectives of each of the stakeholders. There is also a strong need for both additional training and ongoing support to manage and realise stakeholder understanding and goals around learning analytics. Further research is necessary to explore the needs of a greater diversity of stakeholders.

Learning Analytics implementations in universities: Towards a model of success, using multiple case studies


 
 
 In these pioneering days of Learning Analytics in higher education, universities are pursuing a diverse range of in-house implementation strategies, with varying degrees of success. In this exploratory study we compare and contrast the approaches taken at three demographically different Australian universities. The comparison is made in the context of Delone and McLean’s information system success model (1992). In time, a consensus-driven method for using Learning Analytics to improve student learning outcomes will eventuate, including individualized learning, but we are still some distance from this level of maturity. It seems likely that user-friendly proprietary platforms will prosper in the climate of uncertainty. Participants in the study see potential in Learning Analytics but are not sure about how best to realize that potential as the implementation of Learning Analytics systems at Australian universities are still very much in their infancy. Proprietary approaches offering sophisticated functionality seem likely to emerge and take precedence over the trial and error approach. This study addresses an apparent gap in the research as limited studies exist targeting both learning analytics and information system success. The methodology taken explores the research topic through a qualitative lense utlising thematic analysis. The study concludes that digital interventions such as Learning Analytics has great potential to optimize teaching and learning practices. Information systems success research can provide insights into what works and what does not in terms of Learning Analytics implementations. The discipline needs to be systematized for efficient implementation, and must deliver tangible benefits over time.
 
 

Exploring students’ expectations of learning analytics: A person-centered approach

In order to successfully implement learning analytics (LA), we need a better understanding of student expectations of such services. Yet, there is still a limited body of research about students’ expectations across countries. Student expectations of LA have been predominantly examined from a view that perceives students as a group of individuals representing homogenous views. This study examines students’ ideal (i.e., representing their wanted outcomes) and predicted expectations (i.e., unveiling what they realistically expect the LA service is most likely to be) of LA by employing a person-centered approach that allows exploring the heterogeneity that may be found in student expectations. We collected data from 132 students in the setting of Swedish higher education by means of an online survey. Descriptive statistics and Latent Class Analysis (LCA) were used for the analysis. Our findings show that students’ ideal expectations of LA were considerably higher compared to their predicted expectations. The results of the LCA exhibit that the Swedish students’ expectations of LA were heterogeneous, both regarding their privacy concerns and their expectations of LA services. The findings of this study can be seen as a baseline of students’ expectations or a cross-sectional average, and be used to inform student-centered implementation of LA in higher education.

Conducting a Formative Evaluation on a Course-Level Learning Analytics Implementation Through the Lens of Self-Regulated Learning and Higher-Order Thinking

Self-regulated learning (SRL) and higher-order thinking skills (HOTS) are associated with academic achievement, but fostering these skills is not easy. Scholars have suggested an alternative way to scaffold these important skills through learning analytics (LA). This paper presents a formative evaluation of a course-level LA implementation through the lens of self-regulated learning (SRL) and higher-order thinking skills (HOTS). We explored the changes in students’ SRL, HOTS, and perceptions at the end of the course term. Results indicate an increase in some elements of SRL and HOTS, and positive student perceptions. Discussion on implications and opportunities for informing future teaching strategies and course design reiteration are included.

Conducting a Formative Evaluation on a Course-Level Learning Analytics Implementation Through the Lens of Self-Regulated Learning and Higher-Order Thinking

Self-regulated learning (SRL) and higher-order thinking skills (HOTS) are associated with academic achievement, but fostering these skills is not easy. Scholars have suggested an alternative way to scaffold these important skills through learning analytics (LA). This paper presents a formative evaluation of a course-level LA implementation through the lens of self-regulated learning (SRL) and higher-order thinking skills (HOTS). We explored the changes in students’ SRL, HOTS, and perceptions at the end of the course term. Results indicate an increase in some elements of SRL and HOTS, and positive student perceptions. Discussion on implications and opportunities for informing future teaching strategies and course design reiteration are included.

Are We Living In LA (P)LA Land? Reporting on the Practice of 30 STEM Tutors in their Use of a Learning Analytics Implementation at the Open University

Most higher education institutions view their increasing use of learning analytics as having significant potential to improve student academic achievement, retention outcomes, and learning and teaching practice but the realization of this potential remains stubbornly elusive. While there is an abundance of published research on the creation of visualizations, dashboards, and predictive models, there has been little work done to explore the impact of learning analytics on the actual practice of teachers. Through the lens of social informatics (an approach that views the users of technologies as active social actors whose technological practices constitute a wider socio-technical system) this qualitative study reports on an investigation into the practice of 30 tutors in the STEM faculty at Europe’s largest distance learning organization, The Open University UK (OU). When asked to incorporate learning analytics (including predictive learning analytics) contained in the Early Alert Indicator (EAI) dashboard during the 2017–2018 academic year into their practice, we found that tutors interacted with this dashboard in certain unanticipated ways and developed three identifiable “shadow practices”.

The challenge of learning analytics implementation: lessons learned


 
 
 Despite broad interest in learning analytics across the Australian Higher Education sector, there remains few examples of institution-wide implementations. Learning analytics implementation is currently under-theorised with little knowledge of the complexities that mediate the systemic uptake required for an institution-wide implementation. It has been established that approaches to learning analytics that are exclusively top-down or bottom-up, are insufficient for successful implementation across an enterprise. Drawing upon an award-winning and institution-wide learning analytics intervention that has been used across almost 5,000 unit offerings, this paper formulates an initial set of theory informed design principles that can help learning analytics practitioners mediate the complexities of institution-wide implementation.
 
 

Piloting Learning Analytics in a Multidisciplinary Online Program

Learning analytics is a recent innovation that holds promise for improving retention in fully online programs. However, only a few case studies exist to show models for and outcomes of the implementation of learning analytics systems. This paper reports on a learning analytics implementation in a fully online, multidisciplinary program designed for nontraditional students using a pilot planning group with stakeholders from various roles. The processes for selecting reports, creating communication structures, and evaluating outcomes are outlined. Overall, faculty and advisors were positive about the project and found the reports to be helpful. The results suggest that the actions most often triggered by learning analytics reports were emails to students. Evaluation results suggest that the implementation of the learning analytics program and the interventions enacted had a positive impact on student success, though we acknowledge that it is difficult to isolate the impact of the learning analytics tool itself. We also address several challenges that came along with the implementation of learning analytics including understanding the efficacy of interventions, data security, and ethics.

A study on posture-based teacher-student behavioral engagement pattern

As an important part of sustainable society, smart campuses have attracted more and more attention. By associating artificial intelligence with school infrastructure, it makes teaching services more convenient and personalized. Smart classroom is one of the important solutions of smart campuses because teaching and learning are always the primary tasks of smart campuses. At present, many intelligent technologies have been put into smart classrooms for monitoring and analyzing teaching activities, but the complexity of classroom behavior still impedes the development of smart classrooms. Existing researches on learning behavior analytics only focus on student actions. However, student learning behaviors are often closely related to teacher behaviors, so it is not objective and accurate to measure engagement only by analyzing student actions. In this paper, we propose a teacher-student behavioral engagement pattern (TSBEP) to synthetically measure student engagement by adding teacher behaviors. And the decision tree model based on classification and regression tree (CART) is used to predict engagement levels referring to the TSBEP. Experimental results show that the proposed TSBEP can measure the teaching and learning more accurately. And it is helpful to sustainably deploy and implement learning analytics to improve the quality of learning and teaching in smart classrooms.