CodEval

Programming courses can be challenging for first year university students, especially for those without prior coding experience. Students initially struggle with code syntax, but as more advanced topics are introduced across a semester, the difficulty in learning to program shifts to learning computational thinking (e.g., debugging strategies). This study examined the relationships between students' rate of programming errors and their grades on two exams. Using an online integrated development environment, data were collected from 280 students in a Java programming course. The course had two parts. The first focused on introductory procedural programming and culminated with exam 1, while the second part covered more complex topics and object-oriented programming and ended with exam 2. To measure students' programming abilities, 51095 code snapshots were collected from students while they completed assignments that were autograded based on unit tests. Compiler and runtime errors were extracted from the snapshots, and three measures -- Error Count, Error Quotient and Repeated Error Density -- were explored to identify the best measure explaining variability in exam grades. Models utilizing Error Quotient outperformed the models using the other two measures, in terms of the explained variability in grades and Bayesian Information Criterion. Compiler errors were significant predictors of exam 1 grades but not exam 2 grades; only runtime errors significantly predicted exam 2 grades. The findings indicate that leveraging Error Quotient with multiple error types (compiler and runtime) may be a better measure of students' introductory programming abilities, though still not explaining most of the observed variability.

The assessment of programming assignments is a costly task. Several tools have been proposed and developed in order to automate the repetitive tasks performed by instructors in the assessment of programming assignments and provide a faster and more adequate feedback to students. However, adding a new tool increases the overload of new information and environments that students have to deal with. Similarly, the assessment tool is one more resource that the instructor has to configure, maintain and teach students to use, spending time and effort that could be used in other pedagogical activities. For this reason, several works have been conducted to integrate assessment tools for programming assignments in learning management systems (LMSs). The integration of assessment tools into LMSs promotes their adoption in computing courses, since they will be in agreement with the LMS already familiar to students and instructors, without the need to adopt, learn and manage the submission and correction of assignments in another environment. However, these works consist only of abstract conceptual models, do not propose integration with several LMSs, or do not address certain specific difficulties faced when integrating assessment tools. In this perspective, this work aims at the proposition of the IMPACTLE architecture, a solution that allows the integration of different assessment tools for programming assignments in LMSs. The idea is that students and teachers can access the features of the tools through the LMSs they are already used to, without the need to learn how to use and adopt a new tool. Architecture prototypes were instantiated and experiments were carried out involving the use of different assessment tools through LMSs. In general, we noticed that the use of IMPACTLE enables instructors and students perform tasks related to programming activities in a more efficient, efficacious and effective way through the LMSs.

Purpose – Learning management systems (LMSs) became more prominent during the Coronavirus Disease 2019 (COVID-19) pandemic. Universities around the globe find the best solutions and which LMS is the best to deliver their education. This study determined the technology acceptance of using Canvas LMS Free for Teachers in delivering a programming course. Method – A descriptive survey method was implemented in this study on the students of Bulacan State University, a state university in the Philippines, who are taking Bachelor of Science in Information Technology (BSIT). Web Systems and Technologies 2 course was subjected to using Canvas LMS and was evaluated by the students. Results – Within the course, lessons in the form of modules were provided. Additionally, summative assessments were provided to the students at the end of each lesson. Third-year BSIT students were asked to evaluate the LMS used. Out of 169 students enrolled in the course, 132 students answered the survey questionnaire provided. Upon tabulating students’ responses, it was deemed found that they “Strongly Agree” with using Canvas LMS. In terms of its technicalities, the overall mean is M=4.52, which shows that Canvas LMS offers quality features fit for use in delivering programming courses. On the other hand, in terms of its presence, the overall mean is M=4.50, which shows that by using the LMS, students were able to learn the programming language being delivered asynchronously. Conclusion – The results show that Canvas LMS is one of the better options for delivering programming courses through asynchronous modalities. Recommendations – Other programming courses being offered in the university may utilize the same LMS. Additionally, with high results of acceptance, the university may consider a subscription to Canvas LMS to deliver its courses online. Research Implications – As the study’s findings highlighted the high acceptance of students on using Canvas LMS, this presents that utilizing a learning management system in programming is an effective approach as it offers features for programming courses.

Learning how to program is an essential skill, but many students struggle with debugging errors in their code due to inadequate compiler error messages. To better understand students' errors while working on programming assignments, we automatically collect snapshots of students' code after a period of inactivity. We ran the snapshot codes to extract all potential errors. We developed a feedback tool to provide customized responses for runtime errors extracted from the snapshots. We used a survey to evaluate the effectiveness of the feedback tool and found that as the semester progressed, more students used the feedback tool. The survey also revealed that around 40% of the respondents found the tool useful.

The use of a learning management system (LMS) is one of the media that can be used to disseminate lecturer materials to students. Materials that can be uploaded on the LMS can be in the form of lecture materials in the form of files, videos, or questions. The effectiveness of LMS can be evaluated by looking at activities in using LMS. The effectiveness of using LMS can be seen from the log. Log results from LMS can be evaluated in various ways and one way is to use data mining clustering models. The clustering model can be used to create student groupings and the clustering results can be labeled in the form of categories, such as very good, good, and bad categories. This labeling depends on the clustering results that will be processed in the modeling. The research method uses CRISP DM which consists of business understanding, data understanding, data preparation, modeling, evaluation, and deployment. The beginning of the research process is carried out by taking log data in the Moodle LMS. The clustering model in this research will use the K-Means algorithm and the evaluation of clustering results will be evaluated for performance using the Davies-Bouldin method. Implementation of data mining processing using Rapid Miner application. The datasheet used is a datasheet taken from the LMS log of the Computer Programming course in the Mechanical Engineering study program - AKPRIND Institute of Science & Technology Yogyakarta odd semester of the 2021/2022 and 2022/2023 academic years. The results of the study resulted in the best clustering based on the Davies Bouldin method of 2. The clustering results, cluster 0 consists of 28 data named the category of frequent access to LMS and cluster 1 consists of 54 with the category of not frequent access to LMS.

Technology enhanced learning (TEL) is increasingly influencing university education, mainly in overcoming disadvantages of direct instruction teaching approaches, and encouraging creativity, problem solving and critical thinking in student-centered, interactive learning environments. In this paper, experiences from object-oriented programming (OOP) courses that are taught in three institutions from three different European countries are presented and compared. The courses are based on Java and are delivered in the second year of studies, after students have attended an introductory programming course. The emphasis is given on TEL approaches and accompanying tools and services, focusing mainly on Learning Management Systems (LMS). Our students completed an appropriate questionnaire to evaluate the importance and utilization of TEL services that are used or planned to be used at the programming courses. The results of statistical analysis of collected data show that students from all three institutional groups consider organizational services provided by TEL tools as much or very much important in their education, while communicational services are rarely used. Using non-parametric statistical tests we studied the similarities and differences in perceived importance of TEL services among students from different institutional groups.

Aim/Purpose: The goal of this study was twofold: first, to examine how learners’ behavioral engagement types affect their final grades in an online programming course; and second, to explore which factors most strongly affect student performance in an online programming course and their connection to the types of cognitive engagement. Background: During the COVID-19 pandemic situation, information technology educational methods and teaching have been transforming rapidly into online or blended. In this situation, students learn course content through digital learning management systems (LMSs), and the behavioral data derived from students’ interactions with these digital systems is important for instructors and researchers. However, LMSs have some limitations. For computer science students, the traditional learning management system is not enough because the coding behavior cannot be analyzed. Through the OpenEdu platform, we collected log data from 217 undergraduates enrolled in a Python programming course offered by Feng Chia University in Taiwan in the spring semester of 2021. Methodology: We applied the evaluation framework of learning behavioral engagement conducted on a massive open online course (MOOC) platform and integrated it with the partial least squares structural equation modeling (PLS-SEM) approach. PLS-SEM is widely used in academic research and is appropriate for causal models and small sample sizes. Therefore, this kind of analysis is consistent with the purpose of our study. Contribution: In today’s fast-paced world of information technology, online learning is becoming an important form of learning around the world. Especially in computer science, programming courses teach many skills, such as problem-solving, teamwork, and creative thinking. Our study contributes to the understanding of how behavioral engagements in distance programming learning affect student achievement directly and through cognitive engagement. The results can serve as a reference for practitioners of distance programming education. Findings: Our results demonstrate that: (1) online time and video-watching constructs had significant effects on the self-assessment construct, self-assessment and video-watching constructs had significant effects on the final grade construct, and online document reading was not a significant factor in both self-assessments and final grades; (2) video watching had a most significant effect than other behavioral constructs in an online programming course; (3) cognitive engagement types are inextricably linked to the development of a behavioral engagement framework for online programming learning. The mediation analysis and the importance-performance map analysis supported the importance of cognitive engagement. Recommendations for Practitioners: (1) Online education platform developers and university policymakers should pay close attention to the development of self-assessment systems and design such systems based on students’ cognitive skills. (2) Instructors are advised to put substantial effort into the creation of videos for each course session and to actively promote students’ interest in the course material. Recommendation for Researchers: The empirical results reported in this study allow a better understanding of the connection between behavioral engagement and final achievement. However, there are still great challenges in trying to explore more kinds of engagement, like emotional or social engagement. It would be interesting to deepen the results obtained by integrating programming behavior like debugging and testing. Impact on Society: Online programming courses allow students to improve their coding skills and computer science background. Students’ behavioral engagement strongly affects their academic achievement, their ability to complete a course successfully, and the quality of the learning process. Our work can encourage more people who are different majors in society to learn coding in an online environment even not only computer science students. Moreover, the findings of this study can be recommendations for understanding students’ learning behavior and the development of distance programming learning. Future Research: We suggest for future studies: (1) include a wider range of participants, such as students enrolled in MOOCs environments; (2) include more log data items that can express various students’ behavior, depending on the reliability and validity of the research model; and (3) conduct more detailed studies of the effects of emotional engagement as well as additional aspects of students’ social engagement to elucidate the factors affecting students’ behavioral participation and performance more thoroughly.

Web development is a learning context with the potential to support rich computational thinking. Large-scale analysis of compilation and runtime errors have been used in introductory programming courses and similar approaches can be used to understand learning in web development environments. We investigated activity logs of a novel web coding game to uncover learning trajectories and what people struggle with when learning flexible box (flexbox), a collection of new CSS layout features. We designed a game called Flexbox Froggy, in which learners solve challenges by writing a few lines of CSS code, moving from simple levels that require knowledge of one flexbox property, to complex levels combining multiple properties. We investigate learning curves based on the changes in syntactic and semantic errors learners make as they complete the game. Our findings show that people performed better encountering a single new property than combined with properties they had already practiced. Clusters of learners at different levels did not demonstrate expected error rates based on learning curve theory. Also unexpectedly, advanced groups that mastered syntax had higher semantic error rates than the beginner group, especially when attempting new properties or complex use cases. We conclude with implications for designing and developing introductory web programming games and other instructional materials.

The teaching of programming and problem solving has been a challenge and has excited much debate in the literature, so too has been the use of learning management systems (LMSs) to facilitate teaching and enhance learning. In this case study we explore students' perceptions and expectations of the effectiveness of using a constructivist approach for a LMS in a programming course. The constructivist theory was used as a framework for the implementation of a LMS. Interviews and a survey method were used to gather information after participants completed the course. Qualitative and quantitative findings have been presented. Findings showed that the respondents rated the communication and discussion aspects among the highest and guidance to solving problems among the lowest. While discussion and the chat forums are important and support collaboration and social learning, learning problem solving in programming is considered not fully supported. This paper constitutes an important contribution to research in teaching and learning programming using LMSs. While the constructivists approach to using the LMS in programming courses has proved useful, it alerts instructors to the importance of face-to-face learning in the development of problem solving skills (using higher levels of thinking) as well. Some implications for instructors are described.

The purpose of this research is to increase learning achievement by applying the Blended Learning Approach. The Algorithm and Programming course is a subject that requires more problem-solving practice. Learning is done by applying the Flipped Classroom. In face-to-face activities, students do cooperative learning. They discuss the solution of a given computational problem. Student motivation and engagement were measured using the MUSIC® Model of Academic Motivation. MUSIC® Model measures five components, namely Empowerment, Usefulness, Success, Interest, and Caring. Log activities are used to see the interaction between students and Moodle’s Learning Management System (LMS). Log activities show student contributions to discussion forums, number of hits on LMS, number of learning object accesses, and days to access learning objects. Face-to-face learning with cooperative learning trains the ability to work together in groups. The study showed that the Usefulness component had the highest value, with an average of 80.11, while success had the lowest score with a value of 70.16. The learning experience of each student using LMS was different. Students who actively use LMS have better abilities when learning face-to-face. Research shows there is a link between online learning activities and face-to-face learning on learning achievement.

The COVID-19 pandemic has had a significant impact on educational organizations, one of which is the Telkom University educational institute. Telkom University using Learning Management System (LMS) during pandemic covid-19. The use of an LMS will have a huge number of data records stored in the form of event logs. Opportunities to process data recorded in event logs must be maximized to improve Telkom University's LMS continuously. In this study, the process mining method conducted using event logs from Website Application Development (WAD) and Enterprise System (ES) courses to compare student learning patterns between programming courses and non-programming courses. Process mining conducted using PROM 5.2 and the Heuristic Miner plugins. The modeling process uses a heuristic miner algorithm because of its ability to present the main behavior recorded in the event log well by analyzing conformance checking. The result of conformance checking shows a fitness value of 0.887 for WAD and 0.847 for ES. It means the process model can represent the event log well. WAD's value of advanced behavioral appropriateness and degree of model flexibility are 0.594 and 0.411, whereas ES 0.181 and 0.155. The value of the structure shows 1.0 for the two subjects. After obtaining a process model, the next step is to analyze student learning patterns based on the frequency of access to the LMS. With this research, it is hoped that this research can contribute to adding new insights regarding the use of event logs in the field of education and knowing the activities carried out by students through LMS learning media, as well as providing visualization of student behavior patterns in the LMS (Learning Management System) of Telkom University.

A Learning Content Management System (LCMS) is an integrated multi-user administrative, authoring, and delivery platform. It allows administrators to host, schedule, manage registrations, assess, test, and track online training activities. These systems also allow Instructional Designers to create and house course materials, and learners to access course schedules, register for training, take assessments, and manage transcripts. The tools may base on content management or learning content management. Presently a composition of Learning Management System (LMS) and Content Management System (CMS) is used in eLearning. This paper helps you to understand the basic functionality of LMS, CMS and LCMS and how these are helpful in eLearning.

In recent years, social networking websites play a very important role as the interactive platform between users as well as lecturers and learners. Thus, it would be a good idea if a learning management system can be built based on those social networking websites, such as Facebook. In this paper, we therefore proposed to construct an online learning and course management system, which can transform traditional learning management system functions to enable better interactions between lecturers and learners. Furthermore, the system has been implemented and the functions will be shown in the paper. We also concluded this paper with some future directions and suggestions about e-learning, learning management systems and social networking website.

The number of students who decided to study information technology related study programs is continually increasing. Introductory programming courses represent the most crucial milestone in information technology education and often reflect students’ ability to think abstractly and systematically, solve problems, and design their solutions. Even though many students who attend universities have already completed some introductory courses of programming, there is still a large group of students with limited programming skills. This drawback often increases during the first term, and it is often the main reason why students leave study too early. There is a myriad of technologies and tools which can be involved in the programming course to increase students’ chances of mastering programming. The introductory programming courses used in this study has been gradually extended over the four academic years with the automated source code assessment of students’ programming assignments followed by the implementation of a set of suitably designed microlearning units. The final four datasets were analysed to confirm the suitability of automated assessment and microlearning units as predictors of at-risk students and students’ outcomes in the introductory programming courses. The research results proved the significant contribution of automated code assessment in students’ learning outcomes in the elementary topics of learning programming. Simultaneously, it proved a moderate to strong dependence between the students’ activity and achievement in the activities and final students’ outcomes.

In the second-semester programming course at the University of Utah, we have observed that our students suffer unnecessarily from a mismatch between the course content and the programming environment. The course is typical, in that it exposes students to Java a little at a time. The programming environments are also typical, in that they report compilation and run-time errors in the jargon of professional programmers who use the full Java language. As a result, students rely heavily on teaching assistants to interpret error messages, and valuable classroom time is wasted on syntactic diversions.ProfessorJ is our new programming environment that remedies this problem. Like other pedagogical environments, such as BlueJ and DrJava, ProfessorJ presents the student with a simplified interface to the Java compiler and virtual machine. Unlike existing environments, ProfessorJ tailors the Java language and error messages to the students' needs. Since their needs evolve through the course, ProfessorJ offers several language levels, from Beginner Java to Full Java.