Abstract
In this study, we work towards a strategy to measure and enhance the quality of interactions in discussion forums at scale. We present a machine learning (ML) model which identifies the phase of cognitive presence exhibited by a student’s post and suggest future applications of such a model to help online students develop higher-order thinking. We collect discussion forum transcript data from two online courses: CS1301 (an introductory computer programming MOOC) offered by edX and CS6601 (a graduate course on artificial intelligence) which uses the Piazza online discussion tool. We manually code a random sample of students’ posts based on the Community of Inquiry coding scheme and explore trends in cognitive presence within and across the courses. We further use this coded data to analyze the relationship between students’ observed cognitive presence and course grades. In terms of testing and building an ML model, we use a Bidirectional Encoder Representations from Transformers model that uses a deep learning technique to train large text corpus and fine-tune the language model. Our results suggest that deeper cognitive engagement with course concepts, as expressed by higher cognitive presence, are associated with better learning outcomes for students in both course settings. Our ML approach achieves 92.5% accuracy on the classification task, motivating the use of ML for instructional interventions in online courses. We expect that our research study will not only contribute to extending the literature on cognitive presence but also have a beneficial impact on online instructors or curriculum developers in higher education.
Highlights
In this study, we work towards a strategy to measure and enhance the quality of interactions in discussion forums at scale
From the perspective of the practical inquiry model, which serves as the theoretical basis of community of inquiry (CoI), our study focuses on measuring online students’ levels of cognitive presence which can be manifested in four phases, including: triggering event, exploration of ideas, integration of the ideas generated in the exploratory phase, and resolution of the problem or issue (Garrison et al, 2001; Sadaf & Olesova, 2017)
By using the set of manually coded text data that consisted of forum posts and their corresponding cognitive presence scores, we explored an machine learning (ML) approach as a primary technique to automatically identify cognitive presence levels of individual posts generated by participants in discussion forums for online courses
Summary
We work towards a strategy to measure and enhance the quality of interactions in discussion forums at scale. Despite its beneficial influence on students’ learning, online discussion forums pose some challenges in terms of promoting active participation among students, effectively facilitating conversation, organizing an optimal structure for co-constructing knowledge, and dealing with time constraints commonly confronted by instructors (Mazzolini & Maddison, 2007; Zhu, Bonk, & Sari, 2018) To overcome these challenges, deNoyelles, Zydney and Chen (2014) proposed a list of strategies for instructors based on the CoI framework. An instructor can use social modeling cues (e.g., calling a student by name), graded discussion assignments, discussion prompts, facilitation techniques (e.g., questioning), modest feedback (e.g., posting less often but in a meaningful way) and protocol prompts with structured goals and roles in a specific deadline Beyond these strategies, the purposeful design of online platform interfaces (Quintana, Pinto, & Tan, 2021; Zhu et al, 2018) and implementation of instructional strategies to improve students’ cognitive engagement (Garrison & Akyol, 2015; Kilis & Yildirim, 2019) have been shown to enhance successful and engaging online learning. Among these four phases of cognitive presence, the phase of integration has been found as the most difficult to detect because it is often difficult to catalyze the advancement from the exploration phase without appropriate support from instructors or advanced peers (Garrison et al, 2001)
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
