Automatic Programming Assessment Research Articles

Integrating k-means clustering into automatic programming assessment tool for student performance analysis

Computer programming is a subject involving a large number of logic programming activities. A programmer is compulsory to master skills of algorithms, logic, and programming language to conduct programming. An automatic programming assessment tool is an automated tool used to assist instructors in assessing programming tasks. The technology used in this application is open-source based with an evaluation module that will evaluate the sent program code, assessment, and classification. The evaluation results were then processed in the assessment module, where a comparison process with the test case was performed along with the point calculation. The classification module was used to divide students into five groups based on the point of each practicum. This study used k-means clustering classification method. The entities included were lecturers, assistants, students, and compilers. This application had 2 levels of users namely admin and students. Scoring results were then used in the process of determining the classification of student’s performance based on the k-means clustering method. In connection with the classification test results with three iterations, three practicum scores resulted that the classification process was successfully carried out with student’s performance divided into five groups covering very good, good, sufficient, less, and very less. The data used in the clustering process consisted of 41 students with 10 attributes which were then grouped into 3 groups (clusters).

Automated Test Cases and Test Data Generation for Dynamic Structural Testing in Automatic Programming Assessment Using MC/DC

Automatic Programming Assessment (or APA) is known as a method to assist educators in executing automated assessment and grading on students’ programming exercises and assignments. Having to execute dynamic testing in APA, providing an adequate set of test data via a systematic process of test data generation is necessarily essential. Though researches respecting to software testing have proposed various significant methods to realize automated test data generation, it occurs that recent studies of APA rarely utilized these methods. Merely some of the limited studies appeared to resolve this circumstance, yet the focus on realizing test set and test data covering more thorough dynamic-structural testing are still deficient. Thus, we propose a method that utilizes MC/DC coverage criteria to support more thorough automated test data generation for dynamic-structural testing in APA (or is called DyStruc-TDG). In this paper, we reveal the means of deriving and generating test cases and test data for the DyStruc-TDG method and its verification concerning the reliability criteria (or called positive testing) of test data adequacy in programming assessments. This method offers a significant impact on assisting educators dealing with introductory programming courses to derive and generate test cases and test data via APA regardless of having knowledge of designing test cases mainly to execute structural testing. As regards to this, it can effectively reduce the educators’ workload as the process of manual assessments is typically prone to errors and promoting inconsistency in marking and grading.

An Analysis on the Applicability of Meta-Heuristic Searching Techniques for Automated Test Data Generation in Automatic Programming Assessment

Automatic Programming Assessment (APA) has been gaining lots of attention among researchers mainly to support automated grading and marking of students’ programming assignments or exercises systematically. APA is commonly identified as a method that can enhance accuracy, efficiency and consistency as well as providing instant feedback on students’ programming solutions. In achieving APA, test data generation process is very important so as to perform a dynamic testing on students’ assignment. In software testing field, many researches that focus on test data generation have demonstrated the successful of adoption of Meta-Heuristic Search Techniques (MHST) so as to enhance the procedure of deriving adequate test data for efficient testing. Nonetheless, thus far the researches on APA have not yet usefully exploited the techniques accordingly to include a better quality program testing coverage. Therefore, this study has conducted a comparative evaluation to identify any applicable MHST to support efficient Automated Test Data Generation (ATDG) in executing a dynamic-functional testing in APA. Several recent MHST are included in the comparative evaluation combining both the local and global search algorithms ranging from the year of 2000 until 2018. Result of this study suggests that the hybridization of Cuckoo Search with Tabu Search and lévy flight as one of promising MHST to be applied, as it’s outperforms other MHST with regards to number of iterations and range of inputs.

Test Data Generation Approaches for Structural Testing and Automatic Programming Assessment: A Systematic Literature Review

Test Data Generation Approaches for Structural Testing and Automatic Programming Assessment: A Systematic Literature Review

IMPROVING THE RELIABILITY AND VALIDITY OF TEST DATA ADEQUACY IN PROGRAMMING ASSESSMENTS

Automatic Programming Assessment (or APA) has recently become a notable method in assisting educators of programming courses to automatically assess and grade students’ programming exercises as its counterpart; the typical manual tasks are prone to errors and lead to inconsistency. Practically, this method also provides an alternative means of reducing the educators’ workload effectively. By default, test data generation process plays an important role to perform a dynamic testing on students’ programs. Dynamic testing involves the execution of a program against different inputs or test data and the comparison of the results with the expected output, which must conform to the program specifications. In the software testing field, there have been diverse automated methods for test data generation. Unfortunately, APA rarely adopts these methods. Limited studies have attempted to integrate APA and test data generation to include more useful features and to provide a precise and thorough quality program testing. Thus, we propose a framework of test data generation known as FaSt-Gen covering both the functional and structural testing of a program for APA. Functional testing is a testing that relies on specified functional requirements and focuses the output generated in response to the selected test data and execution, Meanwhile, structural testing looks at the specific program logic to verify how it works. Overall, FaSt-Gen contributes as a means to educators of programming courses to furnish an adequate set of test data to assess students’ programming solutions regardless of having the optimal expertise in the particular knowledge of test cases design. FaSt-Gen integrates the positive and negative testing criteria or so-called reliable and valid test adequacy criteria to derive desired test data and test set schema. As for the functional testing, the integration of specification-derived test and simplified boundary value analysis techniques covering both the criteria. Path coverage criterion guides the test data selection for structural testing. The findings from the conducted controlled experiment and comparative study evaluation show that FaSt-Gen improves the reliability and validity of test data adequacy in programming assessments.

Improving Automated Programming Assessments: User Experience Evaluation Using FaSt-generator

Automatic Programming Assessment (APA) has been known as a method used to automatically mark and grade students’ programming solutions. In order to realise APA as a tangible deliverable, a number of automated tools which are called Automated Programming Assessment Systems (APAS) have been developed and tested for decades. Basically, the need for decreasing the load of work among lecturers, timely feedback to students and accuracy on the grading results are the common reasons that motivate the need for APAS. In order to carry out a dynamic testing in APA, it is necessary to prepare an appropriate and adequate set of test data to judge the correctness quality of students’ programming solutions in terms of the aspects of functional and/or structural testing. Manual preparation of quality test data becomes a hard, time consuming, and feasible task in the practice of both software testing and APA. Thus, the generation of automated test data is highly desirable to alleviate the humans’ burden from performing repetitive tasks. This paper aims to describe the design, implementation and user experience when evaluating a tool developed to support APA as a test data generator that is called FaSt-generator. The tool plays an important role to furnish a test set that includes an adequate set of test data to execute both the functional and structural testing in APA. Results collected from the conducted user experience evaluation using FaSt-generator reveal that all the subjects had relatively positive opinions and greatly favour the criteria of User Perception and End-User Computing Satisfaction (EUCS).

Automatic programming assessment

Software metrics have been used extensively to provide quantitative measures of software characteristics. This paper aims at evaluating the relevance of using software metrics as means of assessing students' performance in programming. The study focusses on the use of four basic software metrics which are combined to form a single assessment score. The four metrics are respectively those which measure programming skill, complexity, programming style and programming efficiency. Measurements suggested that the lines of code metric is a good candidate for measuring programming skill. McCabe's cyclomatic complexity metrics have been adopted for measuring program complexity. Program execution times are used as the measuring yardsticks for programming efficiency. To facilitate automatic assessment, a program analyzer has been constructed which can provide measures of all the relevant software metrics together with the appropriate assessment scores. The tool was tested with sample assignments of Pascal programs and good distribution of marks has been obtained.