Learning Analytics Initiatives Research Articles

Co‐designing enduring learning analytics prediction and support tools in undergraduate biology courses

AbstractEven highly motivated undergraduates drift off their STEM career pathways. In large introductory STEM classes, instructors struggle to identify and support these students. To address these issues, we developed co‐redesign methods in partnership with disciplinary experts to create high‐structure STEM courses that better support students and produce informative digital event data. To those data, we applied theory‐ and context‐relevant labels to reflect active and self‐regulated learning processes involving LMS‐hosted course materials, formative assessments, and help‐seeking tools. We illustrate the predictive benefits of this process across two cycles of model creation and reapplication. In cycle 1, we used theory‐relevant features from 3 weeks of data to inform a prediction model that accurately identified struggling students and sustained its accuracy when reapplied in future semesters. In cycle 2, we refit a model with temporally contextualized features that achieved superior accuracy using data from just two class meetings. This modelling approach can produce durable learning analytics solutions that afford scaled and sustained prediction and intervention opportunities that involve explainable artificial intelligence products. Those same products that inform prediction can also guide intervention approaches and inform future instructional design and delivery.Practitioner notesWhat is already known about this topic Learning analytics includes an evolving collection of methods for tracing and understanding student learning through their engagements with learning technologies. Prediction models based on demographic data can perpetuate systemic biases. Prediction models based on behavioural event data can produce accurate predictions of academic success, and validation efforts can enrich those data to reflect students' self‐regulated learning processes within learning tasks. What this paper adds Learning analytics can be successfully applied to predict performance in an authentic postsecondary STEM context, and the use of context and theory as guides for feature engineering can ensure sustained predictive accuracy upon reapplication. The consistent types of learning resources and cyclical nature of their provisioning from lesson to lesson are hallmarks of high‐structure active learning designs that are known to benefit learners. These designs also provide opportunities for observing and modelling contextually grounded, theory‐aligned and temporally positioned learning events that informed prediction models that accurately classified students upon initial and later reapplications in subsequent semesters. Co‐design relationships where researchers and instructors work together toward pedagogical implementation and course instrumentation are essential to developing unique insights for feature engineering and producing explainable artificial intelligence approaches to predictive modelling. Implications for practice and/or policy High‐structure course designs can scaffold student engagement with course materials to make learning more effective and products of feature engineering more explainable. Learning analytics initiatives can avoid perpetuation of systemic biases when methods prioritize theory‐informed behavioural data that reflect learning processes, sensitivity to instructional context and development of explainable predictors of success rather than relying on students' demographic characteristics as predictors. Prioritizing behaviours as predictors improves explainability in ways that can inform the redesign of courses and design of learning supports, which further informs the refinement of learning theories and their applications.

Learning Analytics in Introductory Programming Courses: A Showcase from the Federal University of Amazonas

Introductory computing courses have a high failure rate worldwide. At the Federal University of Amazonas, this also happens and, since 2016 a group of professors decided to reformulate the course at the institution and some learning analytics initiatives have been adopted. The reformulation included a review of the course program and the use of an online judge. After all these years of research, the group has enough material and data and it is a good moment to summarize what has been done and the achievements so far. In this article, the focus will be the learning analytics in three main areas: student performance prediction, classification of difficulty of programming exercises, and gamification. Also, as a contribution, for the first time in a journal, the whole dataset is available to the community.

Wild brooms and learning analytics.

In this commentary we present an analogy between Johann Wolfgang Von Goethe's classic poem, The Sorcerer's Apprentice, and institutional learning analytics. In doing so, we hope to provoke institutions with a simple heuristic when considering their learning analytics initiatives. They might ask themselves, "Are we behaving like the sorcerer's apprentice?" This would be characterized by initiatives lacking faculty involvement, and we argue that when initiatives fit this pattern, they also lack consideration of their potential hazards, and are likely to fail. We join others in advocating for institutions to, instead, create ecosystems that enable faculty leadership in institutional learning analytics efforts.

Early Prediction of At-Risk Students in Secondary Education: A Countrywide K-12 Learning Analytics Initiative in Uruguay

This paper describes a nationwide learning analytics initiative in Uruguay focused on the future implementation of governmental policies to mitigate student retention and dropouts in secondary education. For this, data from a total of 258,440 students were used to generate automated models to predict students at risk of failure or dropping out. Data were collected from primary and secondary education from different sources and for the period between 2015 and 2020. Such data contains demographic information about the students and their trajectories from the first grade of primary school to the second grade of secondary school (e.g., student assessments in different subjects over the years, the amount of absences, participation in social welfare programs, and the zone of the school, among other factors). Predictive models using the random forest algorithm were trained, and their performances were evaluated with F1-Macro and AUROC measures. The models were planned to be applied in different periods of the school year for the regular secondary school and for the technical secondary school ((before the beginning of the school year and after the first evaluation meeting for each grade). A total of eight predictive models were developed considering this temporal approach, and after an analysis of bias considering three protected attributes (gender, school zone, and social welfare program participation), seven of them were approved to be used for prediction. The models achieved outstanding performances according to the literature, with an AUROC higher than 0.90 and F1-Macro higher than 0.88. This paper describes in depth the characteristics of the data gathered, the specifics of data preprocessing, and the methodology followed for model generation and bias analysis, together with the architecture developed for the deployment of the predictive models. Among other findings, the results of the paper corroborate the importance given in the literature of using the previous performances of the students in order to predict their future performances.

Learning analytics initiatives in Latin America: Implications for educational researchers, practitioners and decision makers

AbstractThis study presents several Latin American research initiatives in the field of learning analytics (LA). The study’s purpose is to enhance awareness and understanding of LA among researchers, practitioners and decision makers, and to highlight the importance of supporting research on LA. We analyzed case studies of LA research conducted at four levels of the educational system (the national, institutional, classroom and student levels), which were implemented in four countries (Brazil, Ecuador, Mexico and Uruguay). Diversified cases were selected to demonstrate the use of LA in primary, secondary and higher education, and to allow the inclusion of different types of datasets. These cases also showed the development of legal frameworks for handling ethical issues, and they met the requirements for data privacy protection in Latin America. The study concludes with a discussion of the findings and their implications for further research and practice in the field of LA for teaching and learning.

Mapping Learning Analytics initiatives in Latin America

AbstractThe growth of Learning Analytics (LA) as a research field has been extensively documented since its beginnings. This paper provides a broad overview of the publications that Latin American authors have published in the last years by performing a quantitative review of the literature (from 2011 to 2019). A total of 282 papers were collected and categorized according to a multidimensional model composed of seven dimensions (data collected, stakeholders targeted, goals of the paper, techniques employed, maturity of the tool, research approaches and ethical issues). The paper describes how the field has evolved over the years in Latin America, presenting the main characteristics of the papers from different perspectives (year, type of venue, language) together with the descriptives of the categorized dimensions and the relations between them. At last, papers are presented in number by countries and institutions, and we perform an authorship and coauthorship analysis with information extracted from the selected papers.

Towards the implementation of a countrywide K-12 learning analytics initiative in Uruguay

ABSTRACTThe present work describes the challenges faced during the development of a countrywide Learning Analytics study and tool focused on tracking and understanding the trajectories of Uruguayan students during their first three years of secondary education. Due to the large scale of the project, which covers an entire national educational system, several challenges and constraints were faced during its conception and development. Examples of key challenges were: to understand the different nuances of the educational system, to satisfy ethical and legal requirements without narrowing down the scope and potential novelty of the initiative and to deal with integration and inconsistencies in the databases. This paper presents the design decisions and solutions found to address or mitigate the problems found as a contribution to facilitate similar large-scale projects. Three main experiments using data mining were conducted and their results are also described, pointing out the feasibility of finding meaningful patterns that can be used by educational authorities to improve their decision making and foster public educational policies. Finally, the paper describes the use of the data and the findings of the study to create an interactive visual tool to explore the relationship between student variables, performance and persistence in the system.

Learning Analytics and the Academic Library: Professional Ethics Commitments at a Crossroads

In this paper, the authors address learning analytics and the ways academic libraries are beginning to participate in wider institutional learning analytics initiatives. Since there are moral issues associated with learning analytics, the authors consider how data mining practices run counter to ethical principles in the American Library Association’s “Code of Ethics.” Specifically, the authors address how learning analytics implicates professional commitments to promote intellectual freedom; protect patron privacy and confidentiality; and balance intellectual property interests between library users, their institution, and content creators and vendors. The authors recommend that librarians should embed their ethical positions in technological designs, practices, and governance mechanisms.

The problems and promise of learning analytics for increasing and demonstrating library value and impact

PurposeThe purpose of this paper is to describe the need for academic libraries to demonstrate and increase their impact of student learning and success. It highlights the data problems present in existing library value correlation research and suggests a pathway to surmounting existing data obstacles. The paper advocates the integration of libraries into institutional learning analytics systems to gain access to more granular student learning and success data. It also suggests using library-infused learning analytics data to discover and act upon new linkages that may reveal library value in an institutional context.Design/methodology/approachThe paper describes a pattern pervasive in existing academic library value correlation research and identifies major data obstacles to future research in this vein. The paper advocates learning analytics as one route to access more usable and revealing data. It also acknowledges several challenges to the suggested approach.FindingsThis paper describes learning analytics as it may apply to and support correlation research on academic library value. While this paper advocates exploring the integration of library data and institutional data via learning analytics initiatives, it also describes four challenges to this approach including librarian concerns related to the use of individual level data, the tension between claims of correlation and causation in library value research, the need to develop interoperability standards for library data and organizational readiness and learning analytics maturity issues.Originality/valueThis paper outlines a path forward for academic library value research that may otherwise be stymied by existing data difficulties.

Contexts, critiques, and consequences: A discussion about educational data mining and learning analytics

ABSTRACTThe capture, aggregation, and analysis of student data is becoming ubiquitous at all levels of education—from primary to post‐secondary—as institutions increase their adoption of information technologies to serve administrative and educational ends. In part, this has led to a growing interest among education scientists and administrators in strategically mining and analyzing troves of data in order to uncover student behaviors and intervene in student life. As is the case in other contexts where actors are pursuing Big Data's supposed benefits, educational data mining is fraught with moral, ethical, and political conflict. The panel is composed of four researchers who analyze and critique educational data mining practices and learning analytics initiatives in particular micro‐and macro‐contexts, including academic libraries and professional advising, urban schools, graduate‐level online learning, and higher education generally. Each panelist also represents a unique conceptual background, pulling from work in information ethics and policy, critical data studies, documentation studies, and higher education policy to empirically analyze and critically evaluate tools, systems, practices, policies, and values. Through their individual but thematically intertwined perspectives, the panelists will present their research and lead audience members in discussion.

Analytics for Knowledge Creation: Towards Epistemic Agency and Design-Mode Thinking

Innovation and knowledge creation call for high-level epistemic agency and design-mode thinking, two competencies beyond the traditional scopes of schooling. In this paper, we discuss the need for learning analytics to support these two competencies, and more broadly, the demand for education for innovation. We ground these arguments on a distinctive Knowledge Building pedagogy that treats education as a knowledge-creation enterprise. By critiquing current learning analytics for their focus on static-state knowledge and skills, we argue for agency-driven, choice-based analytics more attuned to higher order competencies in innovation. We further describe ongoing learning analytics initiatives that attend to these elements of design. Prospects and challenges are discussed, as well as broader issues regarding analytics for higher order competencies.

Getting Ready & Getting Started: Academic Librarian Involvement in Institutional Learning Analytics Initiatives

Getting Ready & Getting Started: Academic Librarian Involvement in Institutional Learning Analytics Initiatives

LEA in Private: A Privacy and Data Protection Framework for a Learning Analytics Toolbox

To find a balance between learning analytics research and individual privacy learning analytics initiatives need to appropriately address ethical, privacy and data protection issues and comply with relevant legal regulations. A range of general guidelines, model codes, and principles for handling ethical issues and for appropriate data and privacy protection exist, which may serve the consideration of these topics in a learning analytics context. The importance and significance of data security and protection are also reflected in national and international laws and directives, where data protection is usually considered as a fundamental right. Existing guidelines, approaches and relevant regulations served as a basis for elaborating a comprehensive privacy and data protection framework for the LEA’s BOX project. It comprises a set of eight principles to derive implications for ensuring an ethical treatment of personal data in a learning analytics platform and its services. The privacy and data protection policy set out in the framework is suitable to be used as best practice for other learning analytics projects.

Widening the Field and Sparks of the Future

This issue of the Journal of Learning Analytics comprises two special issue sections. The first of which presents five papers from the 4th International Learning Analytics and Knowledge conference held in Indianapolis. The second showcases the current or recent work of doctoral students who attended the 2nd Learning Analytics Summer Institute at Harvard University, Boston. The issue also includes two articles in the Hot Spots section, discussing the application of learning analytics initiatives in higher education institutions from different perspectives – broad-scale initiatives to individual course design. The breadth and diversity of the articles covered in this issue demonstrate how the discipline has matured and moved towards understanding student learning to inform pedagogical practice and curricular redesign coupled with strategies for the application and adoption of LA strategies across institutions

A Resource-Constrained Approach to Implementing Analytics in an Institution of Higher Education: An Experience Report

Academic analytics and learning analytics have been increasingly adopted by academic institutions of higher learning for improving student performance and retention. While several studies have reported the implementation details and the successes of specific analytics initiatives, relatively fewer studies exist in literature that describe the possible constraints that can preclude an academic or learning analytics initiative from succeeding fully, meeting the criteria of success as defined by the stakeholders affected by such initiatives. Our aim in this article is to describe the constraints that precluded a successful completion of our analytics initiative and how we re-envisioned our approach and scope to achieve our primary goals while operating within the constraints and tools associated with our academic environment.

Introduction to the Special Issue on Learning Analytics

This introduction to the special issue on learning analytics provides an overview of the area, acknowledging the research traditions it emerges from, such as computer-supported collaborative learning, academic analytics, and educational data mining, and the way the field aims to bridge from technological innovation to learning purposes. The introduction provides examples of areas and educational stakeholders who are served by and can benefit from learning analytics initiatives, referring throughout to the articles in this special issue.