Executable Domain-specific Languages Research Articles

A language-parametric test coverage framework for executable domain-specific languages

Test coverage is an effective technique to measure test case quality and to enable fault localization. However, for Executable Domain-Specific Languages (xDSLs), coverage metrics and associated tools are currently manually defined for each xDSL resulting in costly, error-prone, and non-reusable work. To address this problem, we propose a novel language-parametric test coverage framework for xDSLs. We define two coverage metrics adapted to xDSLs: model element and branch coverage. For performing coverage measurements, we propose a generic technique which can be used out-of-the-box by domain experts using any xDSL to define, execute, and test models. In addition, the coverage of model elements and branches can be parameterized for a given xDSL through the definition of coverage rules using a dedicated language. We showcase two test coverage applications for xDSLs: measuring the quality of test suites for executable models and localizing the models’ faults using Spectrum-Based Fault Localization techniques. We evaluate our approach using four different xDSLs. Results show that (i) we can generate meaningful coverage measurements for all investigated models, (ii) the provided coverage rule language enables framework parameterization for all xDSLs, and (iii) the computed coverage measurements are useful in identifying defects of the models.

Advanced testing and debugging support for reactive executable DSLs

Executable Domain-Specific Languages (xDSLs) allow the definition and the execution of behavioral models. Some behavioral models are reactive, meaning that during their execution, they accept external events and react by exposing events to the external environment. Since complex interaction may occur between the reactive model and the external environment, they should be tested as early as possible to ensure the correctness of their behavior. In this paper, we propose a set of generic testing facilities for reactive xDSLs using the standardized Test Description Language (TDL). Given a reactive xDSL, we generate a TDL library enabling the domain experts to write and run event-driven TDL test cases for conforming reactive models. To further support the domain expert, the approach integrates interactive debugging to help in localizing defects, and mutation analysis to measure the quality of test cases. We evaluate the level of genericity of the approach by successfully writing, executing, and analyzing 247 event-driven TDL test cases for 70 models conforming to two different reactive xDSLs.

Adapting TDL to Provide Testing Support for Executable DSLs.

Testing is one of the most prevalent and successful verification and validation (V&V) techniques used in the field of software engineering. While a large number of testing frameworks exist for general-purpose programming languages, providing testing facilities for any given executable Domain Specific Language (xDSL) remains a costly and challenging task. In this context, a standard such as the Test Description Language (TDL) appears as a suitable foundation for the definition of a generic testing approach for xDSLs. Unfortunately, TDL does not provide the domain-specific concepts required to write test cases for a given xDSL and does not include any model execution facilities. Our contribution addresses these limitations and thereby provides a fully generic testing approach for xDSLs based on TDL. Required TDL domain-specific concepts are automatically inferred from the xDSL definition through a model transformation into TDL. Model execution facilities are provided through the definition of a refined operational semantics for TDL. The application of our approach on 5 different xDSLs shows its generality and that it can successfully be used for testing executable models.

Behavioral interfaces for executable DSLs

Executable domain-specific languages (DSLs) enable the execution of behavioral models. While an execution is mostly driven by the model content (e.g., control structures), many use cases require interacting with the running model, such as simulating scenarios in an automated or interactive way, or coupling the model with other models of the system or environment. The management of these interactions is usually hardcoded into the semantics of the DSL, which prevents its reuse for other DSLs and the provision of generic interaction-centric tools (e.g., event injector). In this paper, we propose a metalanguage for complementing the definition of executable DSLs with explicit behavioral interfaces to enable external tools to interact with executed models in a unified way. We implemented the proposed metalanguage in the GEMOC Studio and show how behavioral interfaces enable the realization of tools that are generic and thus usable for different executable DSLs.

Runtime Monitoring for Executable DSLs.

Runtime monitoring is a fundamental technique used throughout the lifecycle of a system for many purposes, such as debugging, testing, or live analytics. While runtime monitoring for general purpose programming languages has seen a great amount of research, developing such complex facilities for any executable Domain Specific Language (DSL) remains a challenging, reoccurring and error prone task. A generic solution must both support a wide range of executable DSLs (xDSLs) and induce as little execution time overhead as possible. Our contribution is a fully generic approach based on a temporal property language with a semantics tailored for runtime verification. Properties can be compiled to efficient runtime monitors that can be attached to any kind of executable discrete event model within an integrated development environment. Efficiency is bolstered using a novel combination of structural model queries and complex event processing. Our evaluation on 3 xDSLs shows that the approach is applicable with an execution time overhead of 121% (on executions shorter than 1s), to 79% (on executions shorter than 20s) making it suitable for model testing and debugging.

Meta3: a code generator framework for domain-specific languages

In software development, domain-specific languages (DSLs) are often applied for specific or repetitive tasks. For executable DSLs, a model interpreter can be developed to run DSL programs. Nevertheless, it is more widespread to generate code in a general-purpose programming language. A properly chosen DSL expresses the original problem more naturally for both domain and information technology experts, and thus, this approach makes the whole development process, especially requirements engineering and requirements analysis, more efficient and less prone to human errors. There are code generator frameworks and so-called language workbenches available that make the development of code generators for DSLs easier. In this paper, we report on our own code generator framework, called Meta3. Meta3 is based on our code generator development experience. We believe that this experience report will be useful for developers of code generators and language workbenches interested in building more flexible and robust code generators as well as better tools that support the construction of the latter.

Omniscient debugging for executable DSLs

Omniscient debugging is a promising technique that relies on execution traces to enable free traversal of the states reached by a model (or program) during an execution. While a few General-Purpose Languages (GPLs) already have support for omniscient debugging, developing such a complex tool for any executable Domain Specific Language (DSL) remains a challenging and error prone task. A generic solution must: support a wide range of executable DSLs independently of the metaprogramming approaches used for implementing their semantics; be efficient for good responsiveness. Our contribution relies on a generic omniscient debugger supported by efficient generic trace management facilities. To support a wide range of executable DSLs, the debugger provides a common set of debugging facilities, and is based on a pattern to define runtime services independently of metaprogramming approaches. Results show that our debugger can be used with various executable DSLs implemented with different metaprogramming approaches. As compared to a solution that copies the model at each step, it is on average sixtimes more efficient in memory, and at least 2.2 faster when exploring past execution states, while only slowing down the execution 1.6 times on average.