CodeBuddy: A Programming Assignment Management System for Short-Form Exercises

A system for automatically testing, evaluating, grading, and providing critical feedback for submitted ‘C’ programming assignments has been implemented. The interface and key features of the system are described in detail along with some examples. The system gives proper attention towards the monitoring of a student’s progress and provides complete automation of the evaluation process, with a fine-grained analysis. It also provides online support to both the instructors and students and is designed for serviceoriented integration with a course management system using Web services.

<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Contribution:</i> Research has shown that computer science (CS) students who start programming assignments (PAs) early generally receive higher grades. This article presents and evaluates a gamification approach that utilizes software engineering tools to motivate CS students to start and finish PAs earlier. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Background:</i> CS can be difficult to learn because students often struggle with errors and how to properly test their code. For these reasons, it is essential that students start their PAs early. Furthermore, software engineering tools, such as version control and unit testing, are increasingly important for students to learn early in their career. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Intended Outcomes:</i> This gamification approach aims to motivate CS students to start and complete PAs earlier, as well as instill software engineering best practices. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Application Design:</i> To motivate students to start and finish assignments early, an open-source gamification system called the Leaderboard was developed. Using gamified points, the Leaderboard rewards students who pass PA unit tests well before the assignment is due. The system is fully automated using Github Classroom, a build server, and the Moodle learning management system. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Findings:</i> Results indicate students who used the Leaderboard did not start assignments significantly earlier; however, they finished assignments earlier, committed code more frequently, and passed more unit tests. The students found the Leaderboard to be motivating and passing unit tests was exciting for them.

The aim of the experiments described in this paper is to evaluate the use of keyboard and mouse dynamics as an appropriate non-obtrusive sensory input for an system that is sensitive to the affective state of its user. Our motivation for starting this research line has been the lack of tools and methodologies for taking into account this affective state in learning environments. In an ideal situation, when instructors have to choose from a collection of programming assignments, they should consider the studentś affective state and skills in order to select a learning resource with the appropriate level of difficulty. However, neither the data or the ability to process it are present in current learning management systems. This work tries to address this problem, by focused on the capture and pre-processing of data that is going to be fed to several machine learning techniques with the objective of classifying the affective states of users with different levels of expertise when learning a new programming language. We capture student data from a web-based platform, a learning management system where students interact with programming exercises. We introduce in this paper a series of pre-processing techniques that are able to convert data from keyboard and mouse dynamics captured from students as they were coding basic Python programming assignments into feature vector, that have been later used for the classification into five affective states: boredom, frustration, distraction, relaxation and engagement. The following classification algorithms have been evaluated: k-nearest neighbors,feed forward neural networks, naive Bayes classifier, J-48 tree induction algorithm, deep learning, random forest, gradient boosted trees and naive Bayes Kernel). The best accuracy was around 78% and was achieved by the tree induction algorithms. Results show that data gathered from ready-available, non-obtrusive sensors can be used successfully as another input to hybrid classification models in order to predict an individualś affective states, and these in turn can be used to make students more engaged and thus learn more efficiently.

Programming assignments are usually considered as a major assessment component of a programming course. As the number of students enrolling in programming courses has been always high, it becomes a difficult task to mark a large number of programming assignments effectively in a short period of time. Moreover, plagiarism on program codes has become a serious problem recently. Markers may not be able to locate similar scripts that they have marked before. This paper introduces an online assignment management system which allows programming assignments to be submitted online and marked effectively. The marking of programming assignments involves two processes: plagiarism detection among different submitted source codes and automatic marking of individual assignment which includes program testing on different test cases and checking across the model answer. In this paper, we propose the use of parse tree for checking the similarity between program codes. The method can be employed in plagiarism detection and automatic marking of programming assignments.

Social learning theory emphasizes the importance of providing learners with opportunities to observe their peers, and to participate actively in a community. Unfortunately, early computing courses tend to emphasize individual programming assignments, which discourage learners from observing and working with their peers. In order to explore the possibility that increased opportunities for social awareness and interaction while working on programming assignments might influence learning outcomes in early computing courses, we are studying the design and use of social networking-style activity streams in such courses. In an empirical study of the use of two types of activity streams in a CS 2 course - one that was part of a learning management system, and one integrated directly into students' programming environment - we found that students who used the integrated stream were twice as socially active; however, social participation in both environments was positively correlated with students' grades. Our results suggest that the use of activity streams as an adjunct to individual programming assignments can positively influence learning; computing instructors would do well to find ways to get their students to participate actively in activity streams during the programming process.

Rapidly growing computing-class sizes across college campuses are challenging instructional resources and staffing. We have developed an automatic testing and grading software system called Code Assessment Extension (CODE). It is integrated with our existing Canvas Learning Management System (LMS). This paper presents experiences from both student and instructor perspectives with using auto-grading in a junior-level, systems course with a heavy emphasis on programming in C++, with several challenges, including - 1 this course was the first experience for both the students and the instructor in using any form of automatic grading, 2 the students have limited experience with C++ programming, particularly in Linux, and 3 the course includes complex concepts on operating systems, multithreading, and networking. The paper presents quantitative results from 3,300 submissions (from 1 course, 1 semester, 6 programming assignments, 54 students, multiple submissions per-student per-assignment) and analysis of end-of-course surveys from 54 students. The inferences from the statistics strongly support the use of automatic grading systems such as CODE to enhance learning in programming-centric courses.

The multimedia-based e-Learning methodology provides virtual classrooms to students. The teacher uploads learning materials, programming assignments and quizzes on university’ Learning Management System (LMS). The students learn lessons from uploaded videos and then solve the given programming tasks and quizzes. The source code plagiarism is a serious threat to academia. However, identifying similar source code fragments between different programming languages is a challenging task. To solve the problem, this paper proposed a new plagiarism detection technique between C++ and Java source codes based on semantics in multimedia-based e-Learning and smart assessment methodology. First, it transforms source codes into tokens to calculate semantic similarity in token by token comparison. After that, it finds semantic similarity in scalar value for the complete source codes written in C++ and Java. To analyse the experiment, we have taken the dataset consists of four (4) case studies of Factorial, Bubble Sort, Binary Search and Stack data structure in both C++ and Java. The entire experiment is done in R Studio with R version 3.4.2. The experimental results show better semantic similarity results for plagiarism detection based on comparison.

Grading coding assignments call for a lot of work. There are numerous aspects of the code that need to be checked, such as compilation errors, runtime errors, the number of test cases passed or failed, and plagiarism. Automated grading tools for programming assignments can be used to help instructors and graders in evaluating the programming assignments quickly and easily. Creating the assignment on Canvas is again a time taking process and can be automated. We developed CodEval, which instantly grades the student assignment submitted on Canvas and provides feedback to the students. It also uploads, creates, and edits assignments, thereby making the whole experience streamlined and quick for instructors and students. It is simple to use, easily integrated with the learning management system, and has a low learning curve. This report shows the background, implementation, and results of using CodEval for programming courses.

With the rapid development of computer science and technology, Computer Aided Instruction(CAI) has being playing a more crucial role in modern teaching management and education itself. While in the teaching modules related to the art of programming, the position of human graders can be well taken by automated programming assignments graders such as Online Judge(OJ). At present, a large proportion of universities and institutions have adopted OJ program which were developed by themselves in programming modules, which leads to huge cost of human resources in its development and maintenance. In this paper, the author proposed an online teaching management system, also called Tsinghua University Online Judger(THUOJ), which is public and universities oriented, to address the above issue. Besides some basic management functions, the system mainly focuses on the programming assignment grading and program assessment customization for varies programming courses, which makes the system be suitable for diversities between different modules and can provide personalized programming grading services. In this article, the author made his design according to the potential users of the system, and proposed a design of the system's structure on the basis of the frame using Linux+Apache+MySQL+PHP(LAMP). Especially, the author adopted C and PHP to program the online judge module. Finally, there is a simple website demo has been implemented on the basis of OJ module, in order to demonstrate the usage of proposed system and online judge module.

In the area of programming assignment assessment the majority of works are focused on control and evaluation of the final submissions. We present an evaluation process and a system, which we are currently working on, for tracking students' behavior during the whole semester. The system stores increments in a source code and presents characteristics from gathered data in a graphical form. Spotted characteristics are: commit and code activity, workload, aggregated frequencies of code additions for the whole group of students, etc. The tool implementation is based on a free repository management system Gitlab.

Learning programming is a complex and challenging task for many students. It involves both understanding theoretical concepts and acquiring practical skills. Hence, analyzing learners’ data from online learning environments alone fails to capture the full breadth of students’ actions if part of their learning process takes place elsewhere. Moreover, existing studies on learning analytics applied to programming education have mainly relied on frequency analysis to classify students according to their approach to programming or to predict academic achievement. However, frequency analysis provides limited insights into the individual time-related characteristics of the learning process. The current study examines students’ strategies when learning programming, combining data from the learning management system and from an automated assessment tool used to support students while solving the programming assignments. The study included the data of 292 engineering students (228 men and 64 women, aged 20–26) from the two aforementioned sources. To gain an in-depth understanding of students’ learning process as well as of the types of learners, we used learning analytics methods that account for the temporal order of learning actions. Our results show that students have special preferences for specific learning resources when learning programming, namely, slides that support search, and copy and paste. We also found that videos are relatively less consumed by students, especially while working on programming assignments. Lastly, students resort to course forums to seek help only when they struggle.

CodEval is a code evaluation tool that integrates with the Canvas Learning Management System to automatically evaluates students' work within a few minutes of the submission. This early feedback allows students to catch and correct problems in their submissions before their submission is graded and gives them a clear idea of the quality of their submission. CodEval handles the tedious aspects of grading, such as compiling and running tests, leaving graders more time to spend on the qualitative aspect of grading. Before using CodEval, instructors would not have a clear view of the student's comprehension of the concept evaluated by the assignment until after the due date. CodeEval helps instructors identify and address the gaps in students' understanding and thus helps more students successfully complete the assignment. We implemented CodEval using Python using the public Canvas API. Any instructor or grader for a Canvas course can use CodEval to automatically evaluate submissions for programming assignments. We developed a syntax to express requirements of submissions such as compilation parameters, inputs, outputs, command-line arguments, timeouts, exit codes, functions used, files generated, output validators, and more. We have made CodEval open source. CodEval is an easy tool for students, graders, and instructors and seamlessly integrates with Canvas. We share our experience with using CodEval in two classes with a total of 90 students and multiple coding assignments.

This article describes a model and implementation of a system for automatically testing, evaluating, grading and providing critical feedback for submitted programming assignments. Complete automation of the evaluation process, with proper attention towards monitoring student’s progress and performing a finegrained analysis is addressed. The tool provides online support to both the instructors and students and is designed for service-oriented integration with a course management system using Web services.

Continuous evaluation of computer programs and providing informative assessments are crucial for computer programming students. However, swift and formative feedback can be challenging to achieve as it is usually a stressful and tedious task for professors merely through manual grading. There is an urgent need for a Learning management system (LMS) that offers instant and detailed feedback in a competitive environment for a better education experience. In this study, we introduce the EvalSeer learning management system. EvalSeer is an LMS equipped with an intelligent auto-grading engine to keep learners motivated and help them move forward. The code evaluation process covers various criteria that strengthen coding abilities and provides learners with the directions they need to improve. These criteria include coding style, code features, dynamic test cases, and successful compilation. EvalSeer uses Long short-term memory (LSTM) networks for code analysis to detect syntax errors and predict potential fixes. Also, the system shall explain suggested fixes backed up with related references. EvalSeer is an easy-to-use cloud-based system with a learner-first approach that can be applied both on-campus and in elearning systems. This work is timely with the dramatic education change, with a notable rise of e-learning due to the COVID-19 pandemic.

Learning programming language of Java is a basic part of the Computer Science and Engineering curriculum. Specific Java compiler is a requirement for writing and convert the writing code to executable format. However, some local installed Java compiler is suffering from compatibility, portability and storage space issues. These issues sometimes affect student-learning interest and slow down the learning process. This paper is directed toward the solution for such problems, which offers a new programming assignment submission system with cloud-based Java compiler and is known as CheckMyCode. Leveraging cloud-computing technology in terms of its availability, prevalence and affordability, CheckMyCode implements Java cloud-based programming compiler as a part of the assignment management system. CheckMyCode system is a cloud-based system that allows both main users, which are a lecturer and student to access the system via a browser on PC or smart devices. Modules of submission assignment system with cloud compiler allow lecturer and student to manage Java programming task in one platform. A framework, system module, main user and feature of CheckMyCode are presented. Also, taking into account are the future study/direction and new enhancement of CheckMyCode.