Compiler Error Messages Research Articles

A Token‐based Compilation Error Categorization and Its Applications

AbstractCompilation errors are unavoidable during the debugging process of novice students. Compiler error messages can help novices to localize and remove errors, but these messages are difficult to understand for students. Previous studies have investigated the compilation error categorization by analyzing compiler error messages, but the categorization cannot cover all kinds of errors, which limits the evaluation of compilation error studies. Therefore, a comprehensive categorization for compilation errors is needed for evaluating the performance of models or tools related to compilation errors. In this study, we first propose a new compilation error categorization, which is based on the smallest unit of the program, tokens. The experiments on 29,573 programs from three datasets show that our proposed compilation error categorization can cover more types of errors and the distribution of the error categorization are significantly different between the datasets. Then, based on our proposed categorization, we develop a neural network model CLACER (CLAssification of Compilation ERrors) for predicting the compilation errors. The results indicate that CLACER can improve the compiler's error localization accuracy and predicts the compilation error effectively. Moreover, based on the proposed categorization, we conduct empirical studies to evaluate the performance of three repairing tools (i.e., DeepFix, RLAssist, and MACER). The comparison results illustrate that DeepFix and RLAssist can fix more errors in the category of delimiter than errors in other categories. Furthermore, MACER performs better than DeepFix and RLAssist because it has a sufficient repairing pattern set for the errors. We also provide some suggestions for improving the repairing tools in the future.

Error messages in relational database management systems: A comparison of effectiveness, usefulness, and user confidence

The database and the database management system (DBMS) are two of the main components of any information system. Structured Query Language (SQL) is the most popular query language for retrieving data from the database, as well as for many other data management tasks. During system development and maintenance, software developers use a considerable amount of time to interpret compiler error messages. The quality of these error messages has been demonstrated to affect software development effectiveness, and correctly formulating queries and fixing them when needed is an important task for many software developers. In this study, we set out to investigate how participants (N=152) experienced the qualities of error messages of four popular DBMSs in terms of error message effectiveness, perceived usefulness for finding and fixing errors, and error recovery confidence. Our results show differences between the DBMSs by three of the four metrics, and indicate a discrepancy between objective effectiveness and subjective usefulness. The results suggest that although error messages have perceived differences in terms of usefulness for finding and fixing errors, these differences may not necessarily result in differences in query fixing success rates.

Analyzing the Effects of Error Messages Presentation on Debugging and Programming

Programming is a fundamental skill of computer science students. However, it can be troublesome to learn. It is notable that a programming error messages can be hard for beginners to comprehend, hampering progress and prompting disappointment. Effectively, translating compiler error messages is significant to rectify errors and advance toward victory in programming. However, these messages are often hard to understand and pose an obstruction to progress for many beginners. Descriptive messages are helpful for students in the beginning phase of learning a programming language. In this article the effect of error messages presentation on debugging and programming score is analyzed. The controlled experiment suggests that presentation of error messages can have a strong effect on debugging score and also helpful to increase the programming score. However, no correlation between debugging score and programming score is identified. On the whole a positive impact of the descriptive error messages is observed during the study.

Effective compiler error message enhancement for novice programming students

Programming is an essential skill that many computing students are expected to master. However, programming can be difficult to learn. Successfully interpreting compiler error messages (CEMs) is crucial for correcting errors and progressing toward success in programming. Yet these messages are often difficult to understand and pose a barrier to progress for many novices, with struggling students often exhibiting high frequencies of errors, particularly repeated errors. This paper presents a control/intervention study on the effectiveness of enhancing Java CEMs. Results show that the intervention group experienced reductions in the number of overall errors, errors per student, and several repeated error metrics. These results are important as the effectiveness of CEM enhancement has been recently debated. Further, generalizing these results should be possible at least in part, as the control group is shown to be comparable to those in several studies using Java and other languages.

CALEE: A computer-assisted learning system for embedded OS laboratory exercises

With the popularity of embedded systems and the consequently rising demand in the job market for professionals well-versed in them, embedded operating systems (EOSs) have become one of the core courses in computer science studies across the world. The objective of EOS courses is to develop students' ability to port, modify, and customize an embedded operating system through a series of laboratory exercises. However, our teaching experience has revealed that beginners require a considerable amount of time to familiarize themselves with the development environment and the relevant processes, such as operating in a command line interface, setting environment variables, and kernel configurations. Furthermore, students need to constantly handle compiler error messages, malfunctions of the target EOS, and incompatibility issues related to development tools. These problems may frustrate and discourage students. A common strategy to address this problem is to dedicate more hours to teaching or to hire more teaching assistants to help students progress. However, none of these methods is suitable for institutions with limited resources. Therefore, in this paper, we develop a computer-assisted learning system called the Computer-assisted Learning Environment (CALEE) to assist students with their assignments and thus motivate them. CALEE consists of two parts: a self-learning assistant (SLAT) and a collaborative learning website (CLW). SLAT is a software application that provides a set of useful functions to help students perform EOS laboratory exercises, whereas the collaborative learning website seeks to encourage greater interaction among students. Our experiments show that CALEE expedites learning, improves students' motivation, and reduces the teaching load.

Understanding beginners' mistakes with Haskell

AbstractThis article presents an overview of student difficulties in an introductory functional programming (FP) course taught in Haskell. The motivation for this study stems from our belief that many student difficulties can be alleviated by understanding the underlying causes of errors and by modifying the educational approach and, possibly, the teaching language accordingly. We analyze students' exercise submissions and categorize student errors according to compiler error messages and then manually according to the observed underlying cause. Our study complements earlier studies on the topic by applying computer and manual analysis while focusing on providing descriptive statistics of difficulties specific to FP languages. We conclude that the majority of student errors, regardless of cause, are reported by three different compiler error messages that are not well understood by students. In addition, syntactic features, such as precedence, the syntax of function application, and deeply nested statements, cause difficulties throughout the course.

On Compiler Error Messages: What TheySayand What TheyMean

Programmers often encounter cryptic compiler error messages that are difficult to understand and thus difficult to resolve. Unfortunately, most related disciplines, including compiler technology, have not paid much attention to this important aspect that affects programmers significantly, apparently because it is felt that programmers should adapt to compilers. In this article, however, this problem is studied from the perspective of the discipline of human-computer interaction to gain insight into why compiler errors messages make the work of programmers more difficult, and how this situation can be alleviated. Additionally, because poorly designed error messages affect novice programmers more adversely, the problems faced by computer science students while learning to program are analyzed, and the obstacles originated by compilers are identified. Examples of actual compiler error messages are provided and carefully commented. Finally, some possible measures that can be taken are outlined, and some principles for compiler error message design are included.

Compiler error messages

Novices find it difficult to understand and use compiler error messages. It is useful to refine this observation and study the effect of different message styles on how well and quickly students identify errors in programs. For example, does an increased level of detail simplify the understanding of errors and their correction? We analyzed messages produced by a number of compilers for five programming languages, and grouped them into three style categories from their level of detail and presentation format, and correlated the level of experience and error type with performance and speed of response. The study involved two groups of students taking an introductory programming course at two different institutions; they used messages in these three styles to debug erroneous code. The results indicate that more detailed messages do not necessarily simplify the understanding of errors but that it matters more where information is placed and how it is structured.

Identifying and correcting Java programming errors for introductory computer science students

Programming in Java can be a daunting task for introductory students, one that is only compounded by the cryptic compiler error messages they see when they first start to write actual code. This article details a project conducted by faculty and advanced students in the creation of an educational tool for Java programming, called Expresso. This paper discusses some existing programming tools, explains their drawbacks, and describes why Expresso is different. We also include a detailed list of typical errors made by novice programmers, used in the construction of the Expresso tool.