Exploring the Influence of Students’ Modes of Behavioral Engagement in an Online Programming Course Using the Partial Least Squares Structural Equation Modeling Approach

Abstract

Aim/Purpose: The goal of this study was twofold: first, to examine how learners’ behavioral engagement types affect their final grades in an online programming course; and second, to explore which factors most strongly affect student performance in an online programming course and their connection to the types of cognitive engagement. Background: During the COVID-19 pandemic situation, information technology educational methods and teaching have been transforming rapidly into online or blended. In this situation, students learn course content through digital learning management systems (LMSs), and the behavioral data derived from students’ interactions with these digital systems is important for instructors and researchers. However, LMSs have some limitations. For computer science students, the traditional learning management system is not enough because the coding behavior cannot be analyzed. Through the OpenEdu platform, we collected log data from 217 undergraduates enrolled in a Python programming course offered by Feng Chia University in Taiwan in the spring semester of 2021. Methodology: We applied the evaluation framework of learning behavioral engagement conducted on a massive open online course (MOOC) platform and integrated it with the partial least squares structural equation modeling (PLS-SEM) approach. PLS-SEM is widely used in academic research and is appropriate for causal models and small sample sizes. Therefore, this kind of analysis is consistent with the purpose of our study. Contribution: In today’s fast-paced world of information technology, online learning is becoming an important form of learning around the world. Especially in computer science, programming courses teach many skills, such as problem-solving, teamwork, and creative thinking. Our study contributes to the understanding of how behavioral engagements in distance programming learning affect student achievement directly and through cognitive engagement. The results can serve as a reference for practitioners of distance programming education. Findings: Our results demonstrate that: (1) online time and video-watching constructs had significant effects on the self-assessment construct, self-assessment and video-watching constructs had significant effects on the final grade construct, and online document reading was not a significant factor in both self-assessments and final grades; (2) video watching had a most significant effect than other behavioral constructs in an online programming course; (3) cognitive engagement types are inextricably linked to the development of a behavioral engagement framework for online programming learning. The mediation analysis and the importance-performance map analysis supported the importance of cognitive engagement. Recommendations for Practitioners: (1) Online education platform developers and university policymakers should pay close attention to the development of self-assessment systems and design such systems based on students’ cognitive skills. (2) Instructors are advised to put substantial effort into the creation of videos for each course session and to actively promote students’ interest in the course material. Recommendation for Researchers: The empirical results reported in this study allow a better understanding of the connection between behavioral engagement and final achievement. However, there are still great challenges in trying to explore more kinds of engagement, like emotional or social engagement. It would be interesting to deepen the results obtained by integrating programming behavior like debugging and testing. Impact on Society: Online programming courses allow students to improve their coding skills and computer science background. Students’ behavioral engagement strongly affects their academic achievement, their ability to complete a course successfully, and the quality of the learning process. Our work can encourage more people who are different majors in society to learn coding in an online environment even not only computer science students. Moreover, the findings of this study can be recommendations for understanding students’ learning behavior and the development of distance programming learning. Future Research: We suggest for future studies: (1) include a wider range of participants, such as students enrolled in MOOCs environments; (2) include more log data items that can express various students’ behavior, depending on the reliability and validity of the research model; and (3) conduct more detailed studies of the effects of emotional engagement as well as additional aspects of students’ social engagement to elucidate the factors affecting students’ behavioral participation and performance more thoroughly.