Comparison of visual and textual languages via task modeling

Abstract

In order for comparative studies of programming languages to be meaningful, differences between the languages need to be carefully studied and well understood. Languages that appear to differ only in syntax (for example, visual vs. textual syntax) may in fact differ greatly in usability. Such differences can confound comparative studies unless they are controlled for. In this paper, we examine the usefulness of fine-grained task modeling for studying the usability of programming languages. We focus on program entry, and demonstrate how to create models of program entry tasks for both visual and textual languages. We also demonstrate how to derive performance time estimates from the models using keystroke-level analysis. A by-product of the model building is a collection of functional-level models that can serve as building blocks for modeling higher-level visual programming tasks. We then report on a comparative study of languages with the same semantics but different syntax (visual and textual). Model-based time predictions of program entry tasks were compared to observed times from an empirical study. The time estimates for the visual condition greatly overestimated the observed times. The primary source of the overestimates appeared to be the time estimate for pointing with the mouse. We then look at three different approaches to improving program entry models. We report on a separate study to calibrate the mouse-pointing time estimate, and demonstrate improved correlation between predicted and observed times with the new estimate. We also apply task modeling to program editing activities, in order to model error recovery behavior during program entry. Finally, we discuss language-specific customization of the keystroke-level operator for mental preparation. We conclude that task modeling is a useful technique for studying differences in the usability of programming languages at the keystroke level.