Online Course Outcomes Research Articles

Predictive Modeling for Enhancing MOOC Completion Rates: A Case Study

In the realm of online asynchronous learning platforms, accurately tracking student performance to predictcourse completion times poses a significant challenge. Completion rates for MOOCs are typically low, with abias towards participants with higher education levels. Understanding factors such as student motivation,engagement, participation, and learning pathway design is crucial for improving student outcomes in onlinecourses. This research developed a predictive framework utilizing advanced deep learning techniques toaccurately forecast course completion times for participants enrolled in an introductory programming course("Python for Beginners" course on the Open Learning Platform of University of Moratuwa Sri Lanka). Byaccurately tracking student performance and leveraging a diverse dataset encompassing demographic andeducational variables, the research seeks to identify factors influencing course completion and predictindividual student outcomes. By utilising deep learning techniques, the prediction performance of the modelwill be improved, ultimately contributing to a more precise forecast of course completion times forparticipants. Evaluation of the model resulted in low Mean Absolute Error (MAE) of 0.0080 and low MeanSquared Error (MSE) of 0.0033 which promises the effectiveness of the developed method in accuratelypredicting course completion times for students. The findings of this study may help increase the successfulcompletion rate of such courses which are delivered in the online asynchronous mode. The study employedadvanced deep learning models optimized through Bayesian methods, highlighting the potential of thesetechniques to enhance MOOC completion rates by offering precise forecasts and actionable insights intostudent engagement. The comprehensive analysis revealed that variables such as 'Current_Lesson', 'SessionTime Category', and 'District_Score' significantly influence completion times. The robust methodologicalframework, including feature engineering, model training, and hyperparameter optimization, sets a precedentfor future research in the field. This research contributes to educational data mining and predictive analytics,offering a scalable approach to improving completion rates and educational outcomes across various onlinelearning platforms. Future research should explore incorporating real-time data and longitudinal studies toenhance model accuracy and generalizability. Additionally, addressing potential biases in the dataset, such asdemographic, prior knowledge, and resource access disparities, is essential to ensure the fair and equitableapplication of the model across diverse student populations. Expanding the research to include a wider rangeof courses and institutions will further validate the model's robustness and applicability in differenteducational contexts.

Towards automated assessment of participation and value creation in networked learning

This paper investigates the integration of artificial intelligence (AI) in Massive Open Online Courses (MOOCs) to improve student engagement and learning outcomes. MOOCs have transformed the landscape of online education, offering wide accessibility but at the price of high dropout rates and limited ability to measure student engagement. With MOOCs' expansive user base, it has been difficult until now to move beyond conventional metrics of course completion and quiz scores, which are insufficient for understanding the multifaceted nature of student engagement. The study addresses these challenges through a novel method of applying a large language model (LLM) to develop a more complex and meaningful understanding of student engagement and experiences in MOOCs.The research is grounded in the networked learning (NL) research tradition by using the Community of Inquiry framework to explore social, cognitive, and teaching presence in online educational environments and automate the assessment of student learning participation. The study focuses on social engagement and interaction, crucial for perceived value creation. Theoretical soundness, data targeting and quality, interpretation benchmarks, and robust data sampling and analysis strategies constitute the foundational pillars of the proposed predictive system. This framework guides the study's approach to address common MOOC issues, such as learner isolation and limited interaction, which are critical factors in students' perceived value and learning.The methodology involves a dual approach of human and LLM coding, testing the ability of LLMs to replicate human coding accuracy. The study was conducted in a non-massive open online course to test system effectiveness before applying it to larger MOOCs. Analysis indicates that an initial noncomplex LLM model can match human coding performance to 60% reliability, representing a significant step forward in automating qualitative analysis in online education, with potential for enhanced results using more complex LLM modelling. Key findings show that AI can effectively measure and enhance student engagement in MOOCs. The use of LLMs in coding qualitative data offers a reliable method for understanding student participation, which is vital for improving educational outcomes in online courses. The research highlights the potential of AI in transforming online education by providing deeper insights into student engagement and learning processes.The study also provides insights into the interplay of AI-measured engagement and learners' value perception, advancing the understanding of networked learning participation and its impact on educational outcomes. Implications extend to curriculum development, pedagogical strategies, and the broader field of online education. Future work includes expanding this methodology to more extensive MOOC environments and incorporating qualitative interviews to enrich the analysis.

Examining students’ behavioural engagement in lecture videos with and without embedded quizzes in an online course.

This study reports part of a larger study that explores three technologies – Padlet, Panopto videos and Google Docs – and their effectiveness in enhancing university students’ engagement in online learning. The current paper explores the role of interactive quizzes embedded in recorded lectures in enhancing student engagement in higher education focused on behaviour engagement. Panopto video analytics and a survey were used as a source of data. Descriptive statistics were used to analyse the data, including percentage and frequency. The analyses focused on 59 university students’ behavioural engagement in the recorded videos with and without embedded quizzes in relation to their video dropout, average viewing time, completion rate and frequency of viewing. The paper highlights the benefits of incorporating interactive quizzes within lecture videos, including increased motivation, engagement, and academic performance. However, this study also shows some interesting results on students’ behavioural engagement when it comes to learning online. Overall, findings suggest that the incorporation of quizzes in lecture videos can be an effective tool for increasing student engagement and improving learning outcomes in online courses. Implications for online course design and future research are discussed.

A Crowd–AI Collaborative Approach to Address Demographic Bias for Student Performance Prediction in Online Education

Recent advances in artificial intelligence (AI) and crowdsourcing have shown success in enhancing learning experiences and outcomes in online education. This paper studies a student performance prediction problem where the objective is to predict students' outcomes in online courses based on their behavioral data. In particular, we focus on addressing the limitation of current student performance prediction solutions that often make inaccurate predictions for students from underrepresented demographic groups due to the lack of training data and differences in behavioral patterns across groups. We develop DebiasEdu, a crowd–AI collaborative debias framework that melds the AI and crowd intelligence through 1) a novel gradient-based bias identification mechanism and 2) a bias-aware crowdsourcing interface and bias calibration design to achieve an accurate and fair student performance prediction. Evaluation results on two online courses demonstrate that DebiasEdu consistently outperforms state-of-the-art AI, fair AI, and crowd–AI baselines by achieving an optimized student performance prediction in terms of both accuracy and fairness.

How Hands-On Assessments Can Boost Retention, Satisfaction, Skill Development, and Career Outcomes in Online Courses

Hands-on assessments provide active opportunities for students to practice new skills they have just learned. Massive open online course (MOOC) platforms offer a uniquely large dataset to track the impact of hands-on assessments on learners’ skill development, satisfaction, and career trajectory. While existing MOOC literature explore enrollment and demographic data, few have investigated the outcomes for learners who engage with different types of assessments within these online courses. This article is important because it quantifies the learner impact of hands-on experiences in MOOCs. With innovative analytics and hundreds of millions of course enrollments, online course platforms can shed light on the influence of alternative teaching decisions and assessment types. MOOCs offer data to quantify individual learners’ skill development in different topics before averaging across all course completers. Metrics, such as satisfaction, utilize learners’ self-reported star ratings of course material. Finally, for career outcomes, MOOC platforms can interact with learners after completing an online course to ask them how the content impacted their job-related outcomes, such as confidence in their role, receiving a promotion, or starting a new position. Control variables such as course domain, instructor characteristics, and learner demographics provide researchers with a robust dataset and thorough methodology to systematically track the benefits of hands-on opportunities in online content. This article examines the content structure and learning behavior data on a MOOC platform. The goal of this empirical study was to examine the impact of hands-on assessments on learner outcomes, including retention, satisfaction, skill development, and career outcomes.

Online higher education during and beyond COVID-19 pandemic: Insights from professors and students at a Taiwanese university

Due to the COVID-19 pandemic, higher education institutions worldwide were compelled to close their campuses and relocate their educational operations to online platforms. This study investigates the aspects that contribute to the efficacy of online education and examines the perceptions of professors and students at a Taiwanese university about the benefits, limits, and suggestions of online learning in COVID-19. A three-stage research framework comprising literature review, interviews and thematic analysis was applied to obtain a thorough understanding of the phenomenon. Remote learning, convenience, and easy accessibility were among the benefits, whereas inefficiency and difficulties in preserving academic integrity were among the drawbacks. Students suggested that blended lectures, dividing lectures into chapters, weekly assignments, regular online meetings and open-book exams might enhance learning outcomes in online courses. According to the suggestions, professors should be trained on using online technologies and building courses with more interactivity. Despite the importance of digital technologies in online learning, professors need to consider minimizing technological needs based on students’ situations. Key lessons learnt from this study are discussed in the hopes of providing beneficial recommendations for educational policymakers, educators, students, and researchers to promote effective education in both online and classroom modes during and after the COVID-19 pandemic.

Time Poverty: A Hidden Factor Connecting Online Enrollment and College Outcomes?

ABSTRACT This study explored the relationships between voluntary online course enrollment (pre-pandemic), time poverty, and college outcomes. Results indicate that students who enrolled in at least one fully online course were significantly more time poor than other students; these differences were largely explained by age, parental status, and paid work. Yet, despite being more time poor, students who enrolled in online courses were more likely to successfully complete their courses, especially after controlling for time poverty. While students who took at least one online course were less likely to be retained in college and accumulated on average fewer credits, outcomes in online courses did not explain these differences; rather, other factors that make students both more likely to enroll online and to drop out or take fewer credits likely play a role. In particular, time poverty fully mediated the relationship between online enrollment and credit accumulation.

Predicting Student Outcomes in Online Courses Using Machine Learning Techniques: A Review

Recent years have witnessed an increased interest in online education, both massive open online courses (MOOCs) and small private online courses (SPOCs). This significant interest in online education has raised many challenges related to student engagement, performance, and retention assessments. With the increased demands and challenges in online education, several researchers have investigated ways to predict student outcomes, such as performance and dropout in online courses. This paper presents a comprehensive review of state-of-the-art studies that examine online learners’ data to predict their outcomes using machine and deep learning techniques. The contribution of this study is to identify and categorize the features of online courses used for learners’ outcome prediction, determine the prediction outputs, determine the strategies and feature extraction methodologies used to predict the outcomes, describe the metrics used for evaluation, provide a taxonomy to analyze related studies, and provide a summary of the challenges and limitations in the field.

Evolution and growth of the ECHO (Enriching Communication skills for Health professionals in Oncofertility) program: a 5-year study in the training of oncofertility professionals.

PurposeAYAs with cancer have unique psychosocial needs, with reproductive health being a primary concern. The ECHO training program provides reproductive health communication training to individuals providing care for AYAs with cancer. The purpose of this project is to describe the growth of ECHO and evaluate changes in learner engagement over a 5-year period.MethodsECHO is an 8-week online training program offered annually, with the program including learning modules, discussion topics and reflections, and synchronous discussions. Reflection quality scores and number of words were compared between the 5 cohorts using ANOVA with a p < .05 level of significance. Descriptive statistics summarized module topics, reflections, and synchronous discussions.ResultsThe average number of reflections per unique learner increased each year (1.4 in cohort 1 vs 4.1 in cohort 5), as did average length and quality of reflections (72.1 words in cohort 1 vs 203.4 words in cohort 5, p < .0001; score of 1.21 in cohort 1 vs 4.46 in cohort 5, p < .0001). The percentage of learners in attendance at synchronous discussions increased between cohorts 4 and 5 (4.8% of learners in cohort 4 vs 18.8% of learners cohort 5).ConclusionsThe ECHO program has seen significant growth and improvement in learner engagement over a 5-year period. This is particularly important given that student learning outcomes in online courses can be predicted by the level of engagement with online content.Implications for Cancer SurvivorsAs fertility and reproductive health remain a top life goal and discussion priority for AYAs surviving cancer, increasing clinical competencies of AHPs in oncofertility is essential.

Student Outcomes in Online Courses: When Does Class Size Matter?

This quantitative study investigated the relationship between class size and student outcomes (final grades and DFW rates) in online higher education courses offered by a large, 4 year public institution in the United States. The following class size cut-off points were used: 8-15 vs. 16 or more students, 8-30 vs. 31 or more students, 8-40 vs. 41 or more students, and 8-50 vs. 51 or more students. Course level data included average final grades and DFW rates for 391 online undergraduate courses taught during the years 2017 and 2018. Significant results suggest that students earned higher grades in STEM (Science Technology Engineering Mathematics) and upper-division courses when online courses included 30 or fewer students. This suggests that it may be beneficial to limit certain kinds of courses to 30 students or fewer, as 30 students may be a tipping point where the benefits of smaller online classes wear off.

Impact of Simulated Electronic Health Records on Informatics Competency of Students in Informatics Course.

ObjectivesNursing has embraced online education to increase its workforce while providing flexible advanced education to nurse professionals. Faculty use virtual simulation and other adaptive learning technologies to enhance learning efficiency and student outcomes in online courses. The purpose of this study was to assess the impact of simulated Electronic Health Records (EHRs) on informatics competency in a graduate online informatics course.MethodsA two-group independent measures study design was adopted to assess students’ perception of a simulated EHR while comparing differences in informatics competencies between an intervention group and a control group. A simulated EHR assignment was provided to students in the intervention group, and a paper assignment was provided to those in the control group. The informatics competency of the students was measured using the Self-Assessment of Informatics Competency Scale for Health Professionals (SICS).ResultsStudents who were enrolled in a family nurse practitioner program in fall of 2019 participated in this study (n = 39). The students expressed positive perceptions of a simulated EHR experience. The SICS results indicated that students in the intervention (simulated EHR) group showed higher informatics competency than those in the control group.ConclusionsThe positive results of this study support incorporating simulated EHR exercises in online courses. Higher informatics competency in the intervention group implies that the use of simulated EHR facilitated learning of complicated informatics concepts.

Online Learning Readiness

ABSTRACT This paper examines the relationship between student characteristics of online learning readiness and student outcomes in online courses at two higher educational institution. Data were collected from student surveys (student characteristics of readiness and outcomes) and merged with institutional student information systems data (e.g., demographics and course grades). Multiple regression analyses revealed that several student characteristics of online learning readiness significantly influenced student outcomes. MANOVA analyses were conducted to examine between group differences of each student characteristics among underrepresented student groups. Significant findings are reported for minorities and for students with disabilities.

The Impact of Computer Self Efficacy on Student Engagement and Group Satisfaction in Online Business Courses

As countless regional, national, and international accrediting bodies continue to employ student engagement measures as mechanisms for quality assurance, universities become more intent on achieving this important gauge of student success. Specifically, the growth in enrollment in distance learning programs adds a unique level of complexity leading researchers to search for ways to increase engagement in the online course environment. Organizations continue to value teamwork and many instructors have incorporated group work into their online courses to teach students this important skill. The present study examines the impact of student engagement on group satisfaction. Furthermore, this research places student engagement at the center of a structural equation model to determine both predictors and outcomes of this important element of student learning. Specifically, this analysis examines whether students’ perceptions of computer self‑efficacy impact student engagement and group satisfaction in online business courses. Our findings indicate that computer self‑efficacy leads to student engagement and, further, that student engagement influences group satisfaction. Importantly, the relationship between student engagement and group satisfaction is mediated by group expectations. Discussions of findings can be utilized to understand the factors that lead to student engagement and its outcomes in online courses.

A Cross-institutional Study of Instructional Characteristics and Student Outcomes: Are Quality Indicators of Online Courses Able to Predict Student Success?

A study was conducted to examine instructional characteristics and their relationship to student outcomes in online courses at a 2-year and 4-year higher educational institution. Instructional characteristics included learner support, course design and organization, content design and delivery, interactivity (student-instructor and student-student), and assessment and evaluation. A student survey instrument was created that captures student perceptions of the instructional characteristics of their course, their learning, and their satisfaction with the course. The data collected from the student survey was merged with data from institutional student information systems (e.g., demographics and course grade).This article examines the relationship between these instructional characteristics, sometimes referred to as indicators of online course quality, and their relationship to student outcomes for all students and for underrepresented students. Significant findings from multiple regression analyses are reported. Additional analyses were conducted to examine differences among underrepresented students (minorities, first-generation, low-income, students with impairments/disabilities) using MANOVA. No significant differences are reported.

Making the Connection: Broadband Access and Online Course Enrollment at Public Open Admissions Institutions

Postsecondary students increasingly enroll in online courses, which have the potential to further democratize higher education by expanding access for historically underserved populations. While a number of studies have investigated student outcomes in online courses, past data limitations have hindered robust examination of a potential mechanism underlying the decision to enroll in an online course: access to high speed broadband. With data from the National Broadband Map and IPEDS, I fit a number of Bayesian regression models to investigate the relationship between various measures of broadband access—download speed, upload speed, and the number of providers—and the number of students who take online courses at public colleges and universities with open admissions policies. Results show that increases in broadband speed at the lower end of the speed spectrum are positively associated with the number of students who take some of their courses online, but that the marginal gain diminishes as speeds increase. This finding suggests that there may be a minimum threshold of necessary broadband access, beyond which increases in speed become a less important factor in the take up of online coursework. Open admissions colleges seeking to improve access for local students through increased online course offerings should consider broadband access in the area, particularly if the targeted populations live in communities with low average broadband speeds.

Online student perceptions of engagement, transactional distance, and outcomes

ABSTRACTThis study explores student perceptions of engagement, transactional distance, and outcomes in online courses. Researchers also investigated linkages between these elements. Data were collected from students enrolled in online courses at three private universities located at different geographical regions in the United States. Six hundred sixty-seven students completed the Revised Scale of Transactional Distance developed by Paul, Swart, Zhang, and MacLeod and a modified version of Dixson’s Online Student Engagement scale during spring 2016 semester. Results indicate students experienced a relatively high level of engagement and a moderate level of transactional distance. Respondents perceived outcomes such as satisfaction, progression, and learning very positively. There were significant differences in responses based on gender and college standing. Student engagement, transactional distance, and outcomes were moderately correlated, and researchers confirmed transactional distance is a valid predictor of student engagement.

Student Predisposition to Instructor Feedback and Perceptions of Teaching Presence Predict Motivation Toward Online Courses

As debates over the value and effectiveness of online courses continue, more research is needed to assist in identifying predictors of positive student outcomes in online courses. Building from previous research in Feedback Intervention Theory (Kluger &amp; DeNisi, 1996) and the Community of Inquiry framework (Anderson, Rourke, Garrison, &amp; Archer, 2001; Garrison, Anderson, &amp; Archer, 1999), this study sought to identify predictors of student motivation toward online courses. Results from a hierarchical multiple regression, using data from 170 online undergraduate students, suggest that student predisposition to receiving instructor feedback and student perceptions of teaching presence provide strong prediction of student motivation toward online courses.

Predicting Student Satisfaction and Outcomes in Online Courses Using Learning Activity Indicators

The premise for this study was that learner interaction in an online web-based course could be assessed in relation to academic performance, or in other words, e-learning. Although some studies reveal that learner interaction with online content is related to student academic performance, it remains unproven whether this is casual, or even if there may be a significant correlation. Thus, this study seeks to measure if there is a directional and then a casual relationship between student online academic performance, engagement analytics and other online activity factors. A unique aspect of this study is that data is collected from Moodle engagement analytics as well as from the activity logs. Student academic performance is measured based on the grade achieved from an assessment designed to map to the course learning objectives.

Online Course-taking and Student Outcomes in California Community Colleges

This paper uses fixed effects analyses to estimate differences in student performance under online versus face-to-face course delivery formats in the California Community College system. On average, students have poorer outcomes in online courses in terms of the likelihood of course completion, course completion with a passing grade, and receiving an A or B. These estimates are robust across estimation techniques, different groups of students, and different types of classes. Accounting for differences in instructor characteristics (including through the use of instructor fixed effects) dampens but does not fully explain the estimated relationships. Online course-taking also has implications for downstream outcomes, although these effects are smaller. Students are more likely to repeat courses taken online, but are less likely to take new courses in the same subject following courses taken online.

Online Learning Based on Essential Concepts and Formative Assessment

BackgroundEngineers are increasingly learning from online courses, in school and the workplace, using online social and informational networks for instruction and to access expertise. Small effective changes in online learning can have large cumulative impact through‐out an organization.Purpose (Hypothesis)This study is a part of a research effort between industry and learning scientists to develop an evidence‐based methodology for online learning. We investigate how learning outcomes are impacted by online courses designed to support feedback and formative assessment during learning.Design/MethodWe compared two versions of an online course derived from identical resources. The new course (treatment) used integrated formative assessments to provide feedback to students during learning; focused video resources to anchor the formative assessments; and an instructional design process based on essential concepts to organize the overall design. The course was implemented with a freely available Learning Management System (LMS).ResultsTreatment course participants learned more overall than did control course participants, with more positive attitudes towards the course content and their future learning. Learning had less dependence on initial knowledge. The collaboration capabilities of the LMS supported course development, but had limited spontaneous use by students during the experiment.ConclusionsThe paper demonstrates that formative feedback and design methods which stress essential concepts can make significant differences in learning outcomes in online courses, a straightforward format for implementation, and ways of measuring effectiveness. Areas for further work are integrating feedback with authentic tasks and mining the data produced by student's online activities.