Visual Attention Patterns in Finding Source Code Defects

Abstract

Existing research used visual effort metrics to determine the visual attention patterns of participants with varying skill levels while finding source code defects. While most of the findings of these studies agree on the results for fixation count metrics, there are differences in the results for fixation duration metrics. Therefore, there is a need for further investigations on the use of visual effort metrics to determine the difference in the visual effort of experts and novices between multiple programs. Thus, we aimed to identify the factors affecting the varying results on fixation duration metrics and validate the results on fixation count metrics. We used visual effort metrics to identify the visual attention patterns of high and low-performing students engaged in defect-finding tasks on multiple programs. We performed statistical tests on the total fixation count, fixation counts on the error lines, total fixation duration, and fixation duration on the error lines to determine the difference in the visual attention patterns between the groups. Among the fixation metrics used, only the total fixation duration metric revealed a significant difference between the high and low-performing students across all programs. High-performing students spent less time on simple programs with simple error types but spent more time on complex programs with logical and semantic error types. In contrast, low-performing students focused more attention on easy programs with one or more syntax errors compared to high-performing students. The results of this study could shed some light on the contrasting findings of previous studies regarding fixation duration. These findings suggest that visual attention patterns of high and low-performing students may vary on multiple programs. The amount of visual effort exerted by the group depends upon the program’s complexity, location of errors in the source code, type of errors injected, and the number of lines of code. This implies that the time spent finding the errors may be associated with the characteristics of the programs and the location and type of injected errors. Therefore, researchers must provide detailed information on these characteristics when describing differences in visual effort metrics between subjects engaged in bug-finding tasks.