Integrating Learning Analytics Research Articles

A learning analytics pilot in Moodle and its impact on developing organisational capacity in a university

Moodle is used as a learning management system around the world. However, integrated learning analytics solutions for Moodle that provide actionable information and allow teachers to efficiently use it to connect with their students are lacking. The enhanced Moodle Engagement Analytics Plugin (MEAP), presented at ASCILITE2015, enabled teachers to identify and contact students at-risk of not completing their units. Here, we discuss a pilot using MEAP in 36 units at Macquarie University, a metropolitan Australian university. We use existing models for developing organisational capacity in learning analytics and to embed learning analytics into the practice of teaching and learning to discuss a range of issues arising from the pilot. We outline the interaction and interdependency of five stages during the pilot: technology infrastructure, analytics tools and applications; policies, processes, practices and workflows; values and skills; culture and behaviour; and leadership. We conclude that one of the most significant stages is to develop a culture and behaviour around learning analytics.

Artificial Intelligence Bringing Improvements to Adaptive Learning in Education: A Case Study

Despite promising outcomes in higher education, the widespread adoption of learning analytics remains elusive in various educational settings, with primary and secondary schools displaying considerable reluctance to embrace these tools. This hesitancy poses a significant obstacle, particularly given the prevalence of educational technology and the abundance of data generated in these environments. In contrast to higher education institutions that readily integrate learning analytics tools into their educational governance, high schools often harbor skepticism regarding the tools’ impact and returns. To overcome these challenges, this work aims to harness learning analytics to address critical areas, such as school dropout rates, the need to foster student collaboration, improving argumentation and writing skills, and the need to enhance computational thinking across all age groups. The goal is to empower teachers and decision makers with learning analytics tools that will equip them to identify learners in vulnerable or exceptional situations, enabling educational authorities to take suitable actions that are aligned with students’ needs; this could potentially involve adapting learning processes and organizational structures to meet the needs of students. This work also seeks to evaluate the impact of such analytics tools on education within a multi-dimensional and scalable domain, ranging from individual learners to teachers and principals, and extending to broader governing bodies. The primary objective is articulated through the development of a user-friendly AI-based dashboard for learning. This prototype aims to provide robust support for teachers and principals who are dedicated to enhancing the education they provide within the intricate and multifaceted social domain of the school.

Assessment of Teachers’ Needs to Integrate Learning Analytics in Higher Education: The Morocco Case

Learning Analytics (LA) is a better solution for harnessing the enormous data that learners leave behind on learning platforms. To take full advantage of it, it needs to be well integrated by teachers. The aim of this study was therefore to explore the needs and willingness of teachers to integrate learning analytics into their courses. This study used a quantitative method to collect data based on a questionnaire distributed to 140 higher education teachers affiliated with Ibn Tofail University. The results of this study show that most teachers are willing and available to commit to the possibility of adopting learning analytics if training and preparation programs on the subject are implemented. They also have other needs for guidance, assistance, easy-to-use tools, and need access to all student data on the appropriate platforms. The study concludes by discussing and proposing a series of recommendations for successfully integrating learning analytics into higher education.

Preconditions for the development of the integrated learning analytics and action research model for Lithuanian general education schools: theoretical insights and research design

Preconditions for the development of the integrated learning analytics and action research model for Lithuanian general education schools: theoretical insights and research design

TEACHERS' DATA LITERACY SKILLS FOR PEDAGOGICAL DECISION MAKING: NEEDS ANALYSIS IN LITHUANIA AND GERMANY

The purpose of the article is to analyse the needs of general education schoolteachers’ data literacy skills that are important for the effective use of learning analytics in the teaching-learning process. The theoretical part of the article presents the idea of big data in education, highlights the aspects of pedagogical value of learning analytics technologies, provides the overview of learning analytic tools. Some overview and comparison of spread of learning analytics tools in general education schools in Lithuania and Germany is presented in the context of data-driven education. The empirical part of the article presents some results from a big qualitative study of teachers’ experiences applying learning analytics tools in teaching - learning process. The main question of the current research is what data literacy skills teachers need in order to use learning analytics tools and make data based pedagogical decisions. Semi-structured interviews were conducted with 10 Lithuanian and 9 German teachers from general education schools, who already have had experience in working with learning experience platforms (digital learning platforms that integrate learning analytics tools). Interview data were analysed by means of content analysis. The results of the qualitative study showed that in order to use learning analytics tools it is important for teachers to have such skills as: digital literacy, data collection, data analysis and interpretation, etc. Comparative analysis of informants’ answers showed that teachers in Lithuania and Germany expressed similar needs for data literacy skills.

Latent factors on the design and adoption of gamified apps in primary education

The main objective of this research is to determine the perception of teachers about the elements that increases the educational effectiveness of gamified apps in primary education. A methodology based on an importance-performance analysis was daeveloped, using a structural equations model to calcuate the degree of importance of each variable. The sample was formed of 212 Spanish teachers with experience using educational apps in the teaching–learning process. Six categories were identified as precursors of educational effectiveness: (1) curriculum connection, (2) feedback and operational experience, (3) assessment and learning analytics, (4) sustainability (Protection Personal data), (5) equal access and (6) flow. These six categories enhance the three traditional areas of gamification intervention: cognitive, emotional and social. In this sense, the design and adoption of an educational gamified app should: (1) establish a clear link between the game and curricular content and competence development; (2) promote self-regulated learning through individual and collaborative activities; (3) offer adapted learning by integrating differentiated personalized learning pathways; (4) integrate learning analytics that can be consulted by teacher, student and family; (5) comply with data protection regulation and promote a safe, sustainable and ethical use of the information generated; (6) take into account different levels of functional diversity. When the gamified app design incorporates these attributes, primary education teachers perceive that such resources can be integrated effectively into the teaching–learning processes.

Is Learning Analytics the Future of Online Education?

Educational structures have been evolving, that even so rapidly with the revolution of information technology and internet. Recent pandemic and its after effects are still looming over the globe, posing as challenge and an opportunity for educators. Online education was one such innovation, which has changed the dynamics of education around the world. The purpose of the paper is three-fold, first, to assess the levels of student engagement in the online learning environment, second, to examine how student engagement is related to their academic performance using learning analytic tools and third, to propose an integrated learning analytics framework. The study used, an exploratory research method and the data was collected from multiple sources; LMS Logs, self-administered questionnaires from students, and interviews with the instructor. The study was conducted at a course level in a private university. The finding suggests a positive relationship between student engagement and their academic performance and advocate the application of an analytics plug-in on LMS which supported instructors in identifying students at risk and providing them with real-time feedback aiming to improve their performance. The study provides insights into the field of online learning and offers evidence-based recommendations to educators. The utilization of Learning Analytics to examine student engagement and understanding of how students learn would contribute to the development of learning theories and in designing an appropriate digital learning environment that supports and improves their learning.

Integrating Learning Analytics into the Flipped Classroom: A Study on Data-Enabled Flipped Learning

Integrating Learning Analytics into the Flipped Classroom: A Study on Data-Enabled Flipped Learning

Strengthening Student Engagement in Learning Through Use of Digital Tools

This paper analyzes teaching and learning in the context of the implementation of digital technology in educational practices. On the basis of the data obtained from focus group discussions and creative workshops with primary school teachers, ways are identified to recognize and strengthen student engagement in learning through the use of digital tools, based on artificial intelligence and integrating learning analytics. As evidenced by the analysis of the research data, students’ cognitive, emotional, and behavioral engagement in learning increases due to the use of digital learning tools. To support and strengthen student engagement, teachers need to rethink their habitual educational practices, anticipate the barriers to engagement caused by digital technologies and the ways of coping with them, make use of digital tools for personalized and in-depth learning, and apply effective means to manage classrooms and interactions between students.

Developing a Stealth Assessment System Using a Continuous Conjunctive Model

Integrating learning analytics in digital game-based learning has gained popularity in recent decades. The interactive nature of educational games creates an ideal environment for learning analytics data collection. However, past research has limited success in producing accessible and effective assessments using game learning analytics. In this study, a mathematics educational game called The Nomads was designed and developed to train learners’ adaptive expertise in rational number arithmetic. Players’ game log data were captured and fitted to a cognitive diagnostic model (CDM) — CCM (continuous conjunctive model). CCM lends itself well to the complex and dynamic nature of game learning analytics. Unlike traditional CDMs, CCM generates parameters at an attribute level and offers more parsimonious diagnoses using continuous variables. The findings suggest that learners’ attribute mastery improved during the gameplay and that learners benefit from using the scaffolds for three of the attributes instructed by the game. This study presents the application of a powerful new tool for game learning analytics. Future studies can benefit from more generalized analytics models and more specified learning attributes and game tasks.

Aligning objectives with assessment in online courses: Integrating learning analytics and measurement theory

Well-designed online courses enhance learning experiences and allow effective development of learners' skills and knowledge. A critical factor contributing to the design of online courses in the higher education settings are well-defined learning objectives that align with course assessments and learning activities. While there are several introspective instruments to evaluate course designs, with the broader adoption of educational technologies and digital tools, there is a wealth of data that offers insights on the alignment of learning objectives to assessments. Such data has paved the way for evidence-based methods of investigating course effectiveness within higher education. This study outlines a methodology for designing and evaluating the alignment between course learning objectives and assessment activities at scale, utilising a combination of learning analytics and measurement theory approaches, more specificially exploratory multi-dimensional item response theory (MIRT) models. We demonstrate the proposed methodology within a professional development MOOC on leadership skills development, where we evaluate the alignemnt between course objectives and reflective writing assessments activities. Our results suggested that the alignment of the existing course objectives to assessment activities can be improved, showing the practical value of the proposed approach. The theoretical and practical implications of this research are further illustrated.

Frameworks for Integrating Learning Analytics With the Electronic Health Record.

The information systems designed to support clinical care have evolved separately from those that support health professions education. This has resulted in a considerable digital divide between patient care and education, one that poorly serves practitioners and organizations, even as learning becomes ever more important to both. In this perspective, we advocate for the enhancement of existing health information systems so that they intentionally facilitate learning. We describe three well-regarded frameworks for learning that can point toward how health care information systems can best evolve to support learning. The Master Adaptive Learner model suggests ways that the individual practitioner can best organize their activities to ensure continual self-improvement. The PDSA cycle similarly proposes actions for improvement but at a health care organization's workflow level. Senge's Five Disciplines of the Learning Organization, a more general framework from the business literature, serves to further inform how disparate information and knowledge flows can be managed for continual improvement. Our main thesis holds that these types of learning frameworks should inform the design and integration of information systems serving the health professions. An underutilized mediator of educational improvement is the ubiquitous electronic health record. The authors list learning analytic opportunities, including potential modifications of learning management systems and the electronic health record, that would enhance health professions education and support the shared goal of delivering high-quality evidence-based health care.

Implementing next generation privacy and ethics research in education technology

AbstractFor the developers of next‐generation education technology (EdTech), the use of Learning Analytics (LA) is a key competitive advantage as the use of some form of LA in EdTech is fast becoming ubiquitous. At its core LA involves the use of Artificial Intelligence and Analytics on the data generated by technology‐mediated learning to gain insights into how students learn, especially for large cohorts, which was unthinkable only a few decades ago. This LA growth‐spurt coincides with a growing global “Ethical AI” movement focussed on resolving questions of personal agency, freedoms, and privacy in relation to AI and Analytics. At this time, there is a significant lack of actionable information and supporting technologies, which would enable the goals of these two communities to be aligned. This paper describes a collaborative research project that seeks to overcome the technical and procedural challenges of running a data‐driven collaborative research project within an agreed set of privacy and ethics boundaries. The result is a reference architecture for ethical research collaboration and a framework, or roadmap, for privacy‐preserving analytics which will contribute to the goals of an ethical application of learning analytics methods. Practitioner notesWhat is already known about this topic Privacy Enhancing Technologies, including a range of provable privacy risk reduction techniques (differential privacy) are effective tools for managing data privacy, though currently only pragmatically available to well‐funded early adopters. Learning Analytics is a relatively young but evolving field of research, which is beginning to deliver tangible insights and value to the Education and EdTech industries. A small number of procedural frameworks have been developed in the past two decades to consider data privacy and other ethical aspects of Learning Analytics. What this paper adds This paper describes the mechanisms for integrating Learning Analytics, Data Privacy Technologies and Ethical practices into a unified operational framework for Ethical and Privacy‐Preserving Learning Analytics. It introduces a new standardised measurement of privacy risk as a key mechanism for operationalising and automating data privacy controls within the traditional data pipeline; It describes a repeatable framework for conducting ethical Learning Analytics. Implications for practice and/or policy For the Learning Analytics (LA) and Education Technology communities the approach described here exemplifies a standard of ethical LA practice and data privacy protection which can and should become the norm. The privacy risk measurement and risk reduction tools are a blueprint for how data privacy and ethics can be operationalised and automated. The incorporation of a standardised privacy risk evaluation metric can help to define clear and measurable terms for inter‐ and intra‐organisational data sharing and usage policies and agreements (Author, Ruth Marshall, is an Expert Contributor on ISO/IEC JTC 1/SC 32/WG 6 "Data usage", due for publication in early 2022).

MOKYMOSI ANALITIKOS NAUDOJIMAS MATEMATIKOS IR GAMTAMOKSLINIO UGDYMO PAMOKOSE: PAGRINDINIO UGDYMO MOKYTOJŲ PATIRTYS

Learning analytics is identified as one of the essential preconditions for ensuring the quality of learning for each student and is associated with the wider possibilities of organizing individualized learning. One of the priorities of Lithuanian education is the individualization and personalization of science and mathematics teaching, which is related to one of the priorities of Lithuanian education, that is recognizing the need to develop students' mathematics, science, and technology competencies as well as to foster a culture of innovation in schools. The importance of integrated teaching (learning) for the sustainable development of a student's science and mathematics competence is recognized. However, problems arise in addressing the issues of integrated science and mathematics organization in the classroom, in finding the most appropriate didactic solutions at the level of a student and a classroom. The benefits of learning analytics in modern education are not in doubt, but in educational practice the approach to it is ambiguous: the search for learning analytics tools, the system of its use, the definitions of benefits for the learner. It is acknowledged that in the discourse of the use of learning analytics in science education, there is little research, examples of pedagogical practice that contain analysis of the possibilities of digital platforms with artificial intelligence and learning analytics tools, and the analysis of teachers' experiences. In the conducted qualitative study (focus group discussion) with mathematics and science teachers, who have accumulated experience in working with digital platforms and applying artificial intelligence-based learning analytics, the possibilities of using learning analytics in the lesson have been disclosed. Focus groups participants are teachers who in 2021. September - December participated in a project with the aim to test learning analytics tools in science education and math lessons. The results of the study revealed that teachers do not question the benefits of integrating digital platforms with artificial intelligence-based learning analytics in identifying student (classroom) learning gaps, learning characteristics, and making evidence-based decisions about learning differentiation and individualization. The results of the focus group discussion with science education and mathematics teachers regarding the use of digital teaching and learning platforms integrating learning analytics in lessons revealed that the priority of learning analytics in lessons is to identify and capture gaps in students' learning achievements and knowledge in a timely manner. The analysis of a student (students) learning data that is provided by digital platforms, which integrate artificial intelligence and learning analytics, allows teachers to make the most appropriate decisions about the organization of teaching: to differentiate and individualize teaching, to consistently develop pupils' general competencies. The results of the discussion highlighted the benefits of learning analytics tools for the learner (students): learning analytics tools allow students to see personal progress; receive the tasks assigned to them individually; implement collaborative learning; engage (intellectually and emotionally) in learning activities; learn not only during lessons. An important criterion for the integration of mathematics and science lessons is the use of the learning analytics tools, the joint work of teachers in analyzing students' learning strengths and weaknesses, finding the best learning opportunities, and making similar or different lesson organization decisions. Participants of the study emphasized the importance of learning analytics data in planning and organizing integrated mathematics and science lessons, i.e. synergistic opportunities for learning analytics in the organization of integrated mathematics and science education. The results of the research do not allow making generalized conclusions that would be suitable for the whole Lithuania, however the results of the research revealed that the development of models for the application of learning analytics and the analysis of their effectiveness are important directions for further research. Keywords: focus group interviews, learning analytics, science education, math lessons

Evaluating student engagement and deep learning in interactive online psychology learning activities

There is growing demand for online learning activities that offer flexibility for students to study anywhere, anytime, as online students fit study around work and family commitments. We designed a series of online activities and evaluated how, where, and with what devices students used the activities, as well as their levels of engagement and deep learning with the activities. A mixed-methods design was used to explore students’ interactions with the online activities. This method integrated learning analytics data with responses from 63 survey, nine interview, and 16 focus group participants. We found that students used a combination of mobile devices to access the online learning activities across a variety of locations during opportunistic study sessions in order to fit study into their daily routines. The online activities were perceived positively, facilitating affective, cognitive, and behavioural engagement as well as stimulating deep learning. Activities that were authentic, promoted problem-solving, applied theory to real-life scenarios, and increased students’ feelings of being supported were perceived as most beneficial to learning. These findings have implications for the future design of online activities, where activities need to accommodate students’ need for flexibility as students’ study habits become more mobile.

Integrating Learning Analytics and Collaborative Learning for Improving Student’s Academic Performance

Integrating Learning Analytics and Collaborative Learning for Improving Student’s Academic Performance

Modeling the relationship between students' prior knowledge, causal reasoning processes, and quality of causal maps

Despite the 30 years of research on the effects of using computer-based mapping tools to construct concept maps, the research has provided little insight into the cognitive processes and strategies students use to create more accurate maps and the types of behaviors that can be mined in computer-based mapping tools to reveal and support processes that improve student performance. This study used video analysis to identify, operationally define, and code processes of reasoning students used while using a computer-based tool to construct causal maps. The process data was then used to conduct a path analysis to determine which and how specific processes impact the quality of students' causal maps and to determine how students’ prior knowledge on the given problems influence which processes students use. The findings reveal that students exhibiting the tendency to use more backward than forward processing and use more breadth than depth-first processing produced higher quality causal maps, and that the influence of prior content knowledge on what processes students use are not conclusive. The findings and the methods developed in this study present new ideas on ways to integrate learning analytics into computer-based mapping tools to model and manage the cognitive processes students use to construct better cognitive maps.

Developing computer-aided diagramming tools to mine, model and support students’ reasoning processes

Despite the last 40 years of research showing that computer-aided diagramming tools improve student learning, very little research reveals the cognitive processes that explain why and how diagramming tools and specific features of the tools affect learning. This study developed a tool that graduate students used to diagram and analyze arguments as the tool mined each students’ actions. The mined data was used to develop algorithms to operationalize and measure the use of backward, forward, breadth, depth-first reasoning. Regression models were then compared to identify which algorithm produced measures that best predicted diagram scores, and to determine the combined and relative impact of each reasoning process on diagram scores. The findings show that observing the placement and location of the first five nodes moved and positioned on screen in relation to the location of the previously moved node provides sufficient data to generate backward/forward and breadth/depth-first ratio scores that predict diagram scores, while individual frequency counts of each of the four processes do not predict scores. The best-fit regression model using the ratio scores show that students using more backward and depth-first processing construct diagrams with higher scores – scores based on criteria that gauge the depth of analysis and not just the number of correct diagram links. This study presents new tools, methods, and new lines of inquiry to advance research on ways to integrate learning analytics into computer-aided diagramming tools.

Assessment Innovation in Higher Education by Integrating Learning Analytics

Assessment Innovation in Higher Education by Integrating Learning Analytics

Specification and evaluation of an assessment engine for educational games: Integrating learning analytics and providing an assessment authoring tool

Educational games are highly engaging, motivating and they offer many advantages as a supplementary tool for education. However, the development of educational games is very complex, essentially because of its multidisciplinary aspect. Fully integrating assessment is challenging and the games created are too often distributed as blackboxes; unmodifiable by the teachers and not providing much insight about the gameplays. We propose an assessment engine, EngAGe, to overcome these issues. EngAGe is used by both developers and educators. It is designed to separate game and assessment. Developers use it to easily integrate assessment into educational games and teachers can then modify the assessment and visualise learning analytics via an online interface. This paper focuses on EngAGe’s benefits for games developers. It presents a quantitative evaluation carried out with 36 developers (7 experienced and 29 students). Findings were very positive: every feature of the engine was rated useful and EngAGe received a usability score of 64 using the System Usability Scale. A Mann-Whitney U test showed a significant difference in usability (Z = −3.34, p < 0.002) between novice developers (mean = 56) and experienced developers (mean = 71) but none in terms of usefulness. This paper concludes that developers can use EngAGe effectively to integrate assessment and learning analytics in educational games.