Type sound syntactic language extension

Abstract

Syntactic programming language extension provides a high level of abstraction, considerably reduces boilerplate code, adapts a programming language to a specific application area, and is an essential tool to manage complexity and to increase the productivity of the software development process. Syntactic language extensions are commonly implemented by rewriting or desugaring them into code of the base language at compile time. To form a sustainable abstraction, the desugaring of extended code must not introduce type errors in the generated code. These type errors are a serious threat to the usability of syntactic abstraction: they are reported in terms of the generated code leading to a leaky abstraction, they are often hard to find, and it is often unclear if the use site or the definition site of the extension is defective. In this thesis, we develop the language extension system and method SoundX (Sound eXtensions) for statically typed programming languages. SoundX is language independent and follows the “extensions as libraries” approach of Sugar* [ER13]. It derives a syntactically extensible language dialect from a formal base language definition. A base language definition comprises the context-free syntax and the type system of the base language. The type system is specified by inductively defined judgements using inference rule schemata. Extensions consist of additional syntax, the desugaring into the base language, and new typing rules for the extended code that act as an interface of the extension. Extensions are free to add new phrases to all syntactic sorts of the base language like terms, types, or declarations. SoundX uses the typing rules of the base language and the extension to type check the extended program prior to desugaring. In this way, type errors are reported relative to the original sugared program and the desugaring can employ type information during the code generation. This allows the implementation of extensions that require type inference and significantly increases the expressiveness of SoundX compared to purely syntactical transformations. To process the type information in the code generator, SoundX maps typing derivations instead of plain expressions of the extended language to typing derivations of the base language using a novel derivation-centred desugaring procedure. SoundX guarantees that the desugaring of an extension results in welltyped code. To this end, it automatically verifies that the desugarings and the typing rules are sound with respect to each other. We formally prove that the SoundX verification procedure is sufficient to ensure that a valid derivation in the extended language is mapped to a valid derivation in the base language by establishing a Preservation theorem and characterise under which conditions such a resulting derivation exists using a Progress theorem. According to the “extensions as libraries” paradigm, extensions are scoped by the module system and activated by importing them. Independently defined extension are composed on import and it is possible to use one extension as the target of another extension enabling incremental extension [EGR12]. The SoundX verification procedure verifies an extension modularly and it is not necessary to re-verify the composition of extensions. SoundX is implemented as a plugin for the Sugar* framework providing an integration into the Eclipse IDE. The implementation contains a pragmatically inspired but principled solution to the practically relevant problem of fresh name generation during code generation and a variation of the resolution algorithm to obtain an explicit representation of a derivation tree. Moreover, it includes a general and type system independent procedure to locate and report type errors, which returns adequate and understandable error message in practice. We demonstrate the applicability and expressiveness of SoundX by several case studies using a base language built on the Simply-Typed λ-Calculus.