UK Open University Research Articles

From hype to evidence: exploring large language models for inter-group bias classification in higher education

ABSTRACT Inter-group biases can diminish student achievements in several ways. Yet, manually identifying those biases in vast learning texts is challenging because of their subtle nature. In the light of processing nuanced language, approaches based on large language models (LLMs) have emerged as promising mechanisms (e.g. ChatGPT). However, their potential for classifying bias in learning text seems under-explored. This study examines the ability of three LLMs (BERT, GPT, and PaLM) to classify inter-group bias within learning texts. Accordingly, 2024 sentences from 91 Higher Education (HE) courses at The Open University UK were assessed for potential biases by LLM-based approaches. Then, we used a sub-sample of sentences (n = 30) for comparison with student classifications. The models suggested varying degrees of biases within the larger sample (BERT = 49.5%; GPT = 10.2%; PaLM = 11.0%). However, varied agreement levels were found within the sub-samples, where Kappa values for model-to-model and model-to-human comparisons ranged from .0 to .77. This underscores the complexity of inter-group bias in learning texts where context and culture can be crucial. We discuss the relevance of those potential biases for learning settings and advocate for tools that are both context- and culture-aware, ensuring more inclusive learning experiences.

The motivation to earn digital badges: a large-scale study of online courses

Digital badges as a recognition for learning, can motivate, provide a means to demonstrate credibility and professional development, and allow learners to showcase their skills and interests. They are an important means to recognise non-accredited learning and can be delivered in agile, educational or professional settings. The Open University UK (OU), has issued over a quarter of a million digital badges via their free online learning platform OpenLearn as recognition for completing and passing free, assessed online courses. This study examines data from over 25,000 survey responses completed by learners who had studied a course that offers an OU OpenLearn digital badge. It builds on previous studies and explores badge earning motivation, related demographic profiles and desire for demonstrating achievements. Data supports the key message that digital badges have a role in supporting employment goals and personal fulfilment, and are an important tool in motivating engagement in lifelong learning.

Using Short Questionnaires to Support Students' Emotion Awareness and Regulation During an Online Group Project

Over the past two decades, research has highlighted the profound influence emotions can have on students’ learning, achievement, and wellbeing. Being able to successfully regulate one’s emotions is now viewed as crucial when learning in educational settings. To support students with this process, a growing line of research has explored various ways of helping students understand and manage their emotions (e.g., via the use of digital tools aimed at helping students become more aware of their emotions and prompting ways to regulate these emotions). Despite this, limited research has been undertaken in online group learning settings, where students work remotely together as part of their course using collaborative communication tools. These learning contexts present unique challenges which can make emotion regulation more difficult than in face-to-face learning environments (e.g., delayed response times when communicating with peers and teachers and a reduced sense of how others are feeling, due to limited emotional cues when learning online). In this paper, we describe an approach used at the UK Open University (UKOU) to assist students’ emotion awareness and regulation during a 9-week group project. This involves the use of short questionnaires which have been embedded into the Virtual Learning Environment (VLE) of a module. Since their inclusion, these questionnaires have been found to enhance emotional awareness in many students, as well as support the regulation of emotions in the group project. To conclude this paper, reflections on using the short questionnaires are discussed and implications for educators and institutions are identified.

Impact of Online Exams on the Quality of Distance Learners' Exam and Exam Revision Experience: Perspectives from The Open University UK

This paper reports findings from a research study at The Open University, UK into the quality of distance learners’ online exam experience and the differences in experience between online (pandemic) and in-person (pre-pandemic) modes of examination. Our research responds to the ongoing need for greater insight into the exam experience and is uniquely positioned in two ways. First, we made use of a robust reference dataset collected before the COVID-19 pandemic and compared this with a second survey administered a year after the pandemic started; second, we asked students about their experience preparing and revising for the exam as well as the exam itself. Exam revision represents an important transitional period for learners. Our results show that, overall, the shift to online remote exams did not impact the quality of distance learners’ experience of revising for exams or taking the exam itself. We found no significant change in the revision experience across six of eight measures, including the learning benefits of learning while revising, enjoyment, and support. However, students reported feeling less anxious when revising for online exams. The quality of the exam experience itself was largely unaffected by the move from in-person to online remote exams. No significant differences were found for seven of the nine measures of exam experience. However, we found satisfaction with the exam environment was significantly higher for online exams and that learners felt the online exam was harder than they expected. Age and gender differences are also explored.

Multidisciplinary team working in science distance learning

ABSTRACT We describe findings from a study based upon a collaborative project carried out online as the end of module assessment activity of a multidisciplinary science module at the UK Open University. Our aim was to determine how well students dealt with working in a team of unfamiliar multidisciplinary scientists and to identify tools and processes that improved the team working experience. The principal source of data was in the form of reflective materials embedded in the module’s final assessment. A two-stage thematic analysis of student reflective narrative texts was performed to draw conclusions about student perceptions of the online team project. In the context of activity levels among the student teams, consistent differences in the way in which team members reflected upon their team working experience were apparent in three of the themes considered: participation, relationships and organisation. We provide a core checklist of actions that can be taken, by students and tutors, to enhance and improve the online collaborative experience.

EXAMINING THE USE OF DIGITAL LEARNING PLATFORMS BY EDUCATORS FOR KNOWLEDGE ACQUISITION AND IMPROVEMENTS IN TEACHING PRACTICES: FUTURE-LEARN BY “THE OPEN UNIVERSITY”

The embracement of the digital opportunities has become revolutionary for the sector of education where learners and educators can benefit from the infinite possibilities for knowledge sharing. The educators can utilize the digital learning platforms for enhancement of their knowledge and professional capacity building. One such digital platform is “Futurelearn” managed by “The Open University UK”. The study is aimed to examine the usefulness of digital learning platforms specifically Futurelearn for the diversified learning experiences by the educators which can be utilized during their teaching practices. The identified objectives of the study were educator’s knowledge, utilization and gained benefits about the digital learning platforms. The post graduate level educators of Open University having M.Phil with 10 years or above teaching experience were the population of the study. Semi structured interviews were conducted from the 20 sample participants and thematic content analysis techniques were used. Knowledge of digital learning platform, Utility of digital learning platforms, learning opportunities and professional capacity building and Benefits of digital learning platforms for educators were identified themes. The findings of the study revealed that the benefits of digital learning platforms include diversified knowledge, professional capacity building, interactive and better communication with trainers of diversified backgrounds and inclusive and blended learning opportunities.

The UK Open University COVID Response: A Sector Case Study

During the coronavirus pandemic, nearly all forms of education underwent an online pivot, to some form of internet-based instruction. The UK Open University (UKOU), like many other distance education and online universities, found its expertise in designing and delivering effective online teaching in demand. This paper reports how the UKOU responded to this demand through a range of mechanisms in three main areas: curriculum, research, and outreach. The different responses in these areas are categorised to highlight six main requirements from the sector: Support, Understanding, Knowledge Sharing, Replacement, Resources and Capacity Building. Using these as a model, the discussion argues that they represent responses that could be undertaken at global, national, and regional levels to develop a more resilient and robust higher education sector that would be better equipped to cope with future disruptions.

Bibliometric Analysis of Research on Learning Analytics Based on Web of Science Database

The purpose of this study is to reveal the status of scientific publications on learning analytics from the past to the present in terms of bibliometric indicators. A total of 659 publications on the subject between the years 2011-2021 were found in the search using keywords after various screening processes. Publications were revealed through descriptive and bibliometric analyses. In the study, the distribution of publications by years and citation numbers, the most published journals on the subject, the most frequently cited publications, the most prolific countries, institutions and authors were examined. In addition, the cooperation between the countries, authors and institutions that publish on the subject was mentioned and a network structure was created for the relations between the keywords. It has been determined that research in this field has progressed and the number of publications and citations has increased over the years. As a result of the bibliometric analysis, it was concluded that the most influential countries in the field of learning analytics are the USA, Australia and Spain. The University of Edinburgh and Open University UK ranked first in terms of the number of citations and Monash University as the most prolific institutions in terms of the number of publications. According to the keyword co-occurrence analysis, educational data mining, MOOCS, learning analytics, blended learning, social network analysis keywords stand out in the field of learning analytics.

How do we talk about race…and mental health?

This paper reports research undertaken with Access students at the UK Open University which sought to address the Black awarding gap through an intersectional approach. Noting Black students were less likely than White students to declare a mental health disability (for which institutional support would be triggered), the authors explored Black students' reasons for non-disclosure using an ethically sensitive methodology. A self-selecting sample of Black students were interviewed by Black tutors, resulting in deep insights into the lived experiences of Black students. Due to the challenging personal stories elicited, these were presented as five composite personas. Institutional recommendations around the need for enhanced cultural competence amongst staff, and more inclusive language in policy implementation may also address issues across the sector.

Are We Living In LA (P)LA Land? Reporting on the Practice of 30 STEM Tutors in their Use of a Learning Analytics Implementation at the Open University

Most higher education institutions view their increasing use of learning analytics as having significant potential to improve student academic achievement, retention outcomes, and learning and teaching practice but the realization of this potential remains stubbornly elusive. While there is an abundance of published research on the creation of visualizations, dashboards, and predictive models, there has been little work done to explore the impact of learning analytics on the actual practice of teachers. Through the lens of social informatics (an approach that views the users of technologies as active social actors whose technological practices constitute a wider socio-technical system) this qualitative study reports on an investigation into the practice of 30 tutors in the STEM faculty at Europe’s largest distance learning organization, The Open University UK (OU). When asked to incorporate learning analytics (including predictive learning analytics) contained in the Early Alert Indicator (EAI) dashboard during the 2017–2018 academic year into their practice, we found that tutors interacted with this dashboard in certain unanticipated ways and developed three identifiable “shadow practices”.

Professional Development for Sustaining the ‘Pivot’: The impact of the Learning Design and Course Creation Workshop on Six Belarusian HEIs

Throughout 2020, and into 2021, national governments were forced at different times to impose ‘lockdown’ on traditional approaches to education to cope with the impact of COVID-19. Higher education institutions (HEI) with face-to-face models scrambled to ‘pivot’ to distance and online learning. Whilst originally conceived as a temporary measure, the longevity of the virus has raised significant questions about the sustainability of these approaches and revealed a need for teacher professional development (TPD) activity to support teachers in designing learning based on robust distance and online education frameworks. One example of such teacher professional development is the Learning Design and Course Creation (LDCC) Workshop from The Open University UK. Prior to the pandemic, in September 2018, this workshop was attended by staff from six Belarusian HEIs involved in a project titled Enhancement of Lifelong Learning in Belarus (BELL). The 18-month longitudinal study presented here used the Academic Professional Development Effectiveness Framework as a structure to evaluate the impact of this activity on the participants’ practice, and the five courses they were tasked with creating. The findings suggest that on the whole the team-based, reflective and experiential TPD pedagogy employed by the LDCC Workshop was effective in preparing the BELL Project participants for designing and creating their online and blended learning courses and this approach should be utilised effectively to support other HEIs that plan to enhance their distance capacity, either due to COVID-19, or for other reasons.

Widening HE participation in the arts: Impacts of an access module on learner preparedness

Despite the plethora of research on widening participation in the last 20 years, access to the arts and humanities has remained relatively under-explored, especially in relation to the preparedness of adult learners. This article reports a case study investigating the impact of an arts and languages Access module at the UK Open University. Findings from interviews with 37 Access students were analysed in relation to four themes: the need for Access preparation; generic studentship skills; discipline-specific skills; intrinsic enjoyment and interdisciplinary study. We conclude embedded generic skills enhance learner confidence and time management, while academic literacy skills relevant to the arts and humanities enhance cultural capital and enable disadvantaged learners to access challenging disciplines. The impact of a preparatory arts and humanities module extends into the lives of individual students, suggesting a counter-narrative to the prevailing, ‘economic value’ paradigms of higher education policymakers.

Enhancing the quality of open and distance learning in China through the identification and development of learning design skills and competencies

PurposeThe purpose of this paper is to identify staff skills and competencies on which Chinese open universities should focus their professional development activities in order to enhance the quality of open and distance learning (ODL) in China.Design/methodology/approachData were collected from surveys of 220 academic and academic-related Chinese staff, each of whom participated in one of seven 3-day Learning Design and Course Creation (LDCC) workshops at three large regional open universities in China between 2017 and 2019. The workshops were based on the UK Open University (UKOU) approach to learning design (LD) and course creation. Using content analysis methodology, textual responses were analysed and compared against the Instructional Design Competencies framework provided by the International Board of Standards for Training, Performance and Instruction (IBSTPI).Findings(1) Designing instructional interventions, (2) keeping up to date with design theories, (3) communicating in order to manage stakeholders, teams and projects were the main competencies identified from the participants responses. However, these three identified competencies differed in emphasis between the institutions that took part.Originality/valueIn Western higher education institutions (HEIs), LD has developed as an important approach to improving quality. A need has been identified for robust approaches to quality and professional development opportunities to enhance teaching and learning standards in open and distance learning (ODL) in China. This paper identifies and discusses specific LD skills and competencies that could be targeted to improve the quality of ODL in China.

How Can Predictive Learning Analytics and Motivational Interventions Increase Student Retention and Enhance Administrative Support in Distance Education?

Despite the potential of Predictive Learning Analytics (PLAs) to identify students at risk of failing their studies, research demonstrating effective application of PLAs to higher education is relatively limited. The aims of this study are 1) to identify whether and how PLAs can inform the design of motivational interventions and 2) to capture the impact of those interventions on student retention at the Open University UK. A predictive model — the Student Probabilities Model (SPM) — was used to predict the likelihood of a student remaining in a course at the next milestone and eventually completing it. Undergraduate students (N=630) with a low probability of completing their studies were randomly allocated into the control (n=312) and intervention groups (n=318), and contacted by the university Student Support Teams (SSTs) using a set of motivational interventions such as text, phone, and email. The results of the randomized control trial showed statistically significant better student retention outcomes for the intervention group, with the proposed intervention deemed effective in facilitating course completion. The intervention also improved the administration of student support at scale and low cost.

Scientific Development of Educational Artificial Intelligence in Web of Science

The social and technological changes that society is undergoing in this century are having a global influence on important aspects such as the economy, health and education. An example of this is the inclusion of artificial intelligence in the teaching–learning processes. The objective of this study was to analyze the importance and the projection that artificial intelligence has acquired in the scientific literature in the Web of Science categories related to the field of education. For this, scientific mapping of the reported documents was carried out. Different bibliometric indicators were analyzed and a word analysis was carried out. We worked with an analysis unit of 379 publications. The results show that scientific production is irregular from its beginnings in 1956 to the present. The language of greatest development is English. The most significant publication area is Education Educational Research, with conference papers as document types. The underlying organization is the Open University UK. It can be concluded that there is an evolution in artificial intelligence (AI) research in the educational field, focusing in the last years on the performance and influence of AI in the educational processes.

Analytics for Action: Assessing effectiveness and impact of data informed interventions on online modules

La efectividad del uso de las analíticas de aprendizaje es un tópico de gran relevancia en la literatura sobre el tema. Una revisión sistemática reciente encontró 689 artículos en este campo (Larrabee Sonderlund et al., 2019). Sin embargo, solamente 11 de los 689 artículos destacan el potencial de las intervenciones basadas directamente en el análisis de los datos disponibles. La Open University UK (OU) es una de las pocas instituciones que desarrolla e implementa sistemáticamente un marco de uso de las análiticas a gran escala. Este documento revisa el impacto de una parte de este marco: el proceso de Analytics for Action (A4A). Utilizando datos del curso académico 2017-18, revisamos los comentarios de los participantes y las intervenciones acordadas como parte del proceso.El proceso A4A implica la capacitación práctica del personal, seguida de reuniones sucesivas en las que se discuten los datos cuando el curso ya está disponible en línea. El objetivo del proceso es ayudar a los educadores a planificar y realizar intervenciones basadas en la evidencia, con el fin de mejorar la retención y satisfacción de los estudiantes. Los resultados de este estudio indican que los participantes están satisfechos con la capacitación y que las reuniones de apoyo están ayudando a proporcionar nuevas perspectivas sobre los datos. El alcance y la naturaleza de las intervenciones varían ampliamente, desde la no intervención hasta intervenciones que abarcan múltiples presentaciones (cohortes) del curso. En algunos casos, medir el impacto real de las acciones tomadas requerirá el análisis de los datos de otras presentaciones. El trabajo también presenta hallazgos que indican que todavía hay margen para mejorar el seguimiento de las acciones acordadas, el apoyo brindado a los equipos académicos para implementar tales acciones y la evaluación final del impacto en los resultados y satisfacción de los estudiantes.

Design education in the Open

From its inception in the 1970s the UK Open University faced the challenge of teaching design to students at a distance and with open entry qualifications. Teaching design ‘in the open’ has required creative approaches to aid students in the acquisition of requisite skills, knowledge and values. OU design courses pioneered the teaching of design for a broad, non- specialist audience and in identifying the particular characteristics of design thinking, influencing not only OU students but wider teaching in the higher education sector. These principles have been applied during the development of design education at the OU from printed text and broadcast TV into the use of digital media and the Internet. Over time, technological changes, together with concomitant changes in HE generally, have brought different modes of design education closer together, but the OU continues to pioneer in design pedagogy.

Success of distance learning graduates and the role of intrinsic motivation

ABSTRACT This study explored the perceptions of 51 graduates of a distance learning health and social care programme, identifying what factors contributed to students continuing for the duration of their distance learning studies and completing their degree. An appreciative inquiry approach was used to identify the positive aspects of the students’ experience. Support from family, tutors and employers as well as flexibility of studying at The Open University UK, enabled graduates to continue their studies. A key aspect of the graduates’ success was the supportive feedback received from tutors, which they reported enabled them to build their knowledge, skills and confidence as they progressed to completion of their degree. Flexibility came in different forms, from being able to access their learning materials at any time around their busy lives, to tutors being very responsive to students’ needs. The findings are discussed in relation to the students’ development of competence and autonomy, factors that contribute to strengthen students’ intrinsic motivation.

Effective usage of learning analytics: what do practitioners want and where should distance learning institutions be going?

ABSTRACT The implementation of learning analytics may empower distance learning institutions to provide real-time feedback to students and teachers. Given the leading role of the Open University UK (OU) in research and application of learning analytics, this study aims to share the lessons learned from the experiences of 42 participants from a range of faculty, academic and professional positions, and expertise with learning analytics. Furthermore, we explored where distance learning institutions should be going next in terms of learning analytics adoption. The findings from the Learning Analytics User Stories (LAUS) workshop indicated four key areas where more work is needed: communication, personalisation, integrated design, and development of an evidence-base. The workshop outputs signalled the aspiration for an integrated analytics system transcending the entire student experience, from initial student inquiry right through to qualification completion and into life-long learning. We hope that our study will spark discussion on whether (or not) distance learning institutions should pursue the dream of an integrated, personalised, and evidence-based learning analytics system that clearly communicates useful feedback to staff and students, or whether this will become an Orwellian nightmare.

Using Technology-Enabled Learning Networks to Drive Module Improvements in the UK Open University

This paper describes a work-in-progress action research project to investigate how technology-enabled learning networks may achieve practical organisational improvement outcomes in the author’s own institution, a UK distance learning higher education (HE) context. Collaborative learning technology, typically used for academic learning or professional development, affords the geographically scattered and disparate practitioner stakeholders involved in distance learning module design and delivery the ability to grapple together with a problem area requiring improvement. In this particular context, there is a perennial organisational need to close a feedback loop between remote module tutors and campus based teams, to develop a joint understanding of teaching and learning design challenges, and to put tutors as close as possible to the development of solutions. However, there is insufficient conceptual and practical understanding about the mechanisms by which this unfolding process of technology-enabled organisational learning might happen. Conceptual frameworks in the field of learning networks and networked learning are still developing (Sloep, 2016). The aim of this research is to investigate the active use of learning networks to achieve practical improvement outcomes, and to explore a new conceptual framework covering all stages of the required learning process. The research aims to make an original contribution to the call for actionable knowledge in organisational research, using a collaborative, equitable and rigorous action-oriented and theory-building approach, which also aims to achieve measurable impact.