SynShine: Improved Fixing of Syntax Errors

Abstract

Novice programmers struggle with the complex syntax of modern programming languages like <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Java</small> , and make lot of syntax errors. The diagnostic syntax error messages from compilers and IDEs are sometimes useful, but often the messages are cryptic and puzzling. Novices could be helped, and instructors' time saved, by automated repair suggestions when dealing with syntax errors. Large samples of novice errors and fixes are now available, offering the possibility of data-driven machine-learning approaches to help novices fix syntax errors. Current machine-learning approaches do a reasonable job fixing syntax errors in shorter programs, but don't work as well even for moderately longer programs. We introduce <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SynShine</small> , a machine-learning based tool that substantially improves on the state-of-the-art, by learning to use compiler diagnostics, employing a very large neural model that leverages unsupervised pre-training, and relying on multi-label classification rather than autoregressive synthesis to generate the (repaired) output. We describe <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SynShine</small> 's architecture in detail, and provide a detailed evaluation. We have built <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SynShine</small> into a free, open-source version of Visual Studio Code (VSCode); we make all our source code and models freely available.