C Preprocessor Research Articles

Preprocessor-based variability in open-source and industrial software systems: An empirical study

Almost every sufficiently complex software system today is configurable. Conditional compilation is a simple variability-implementation mechanism that is widely used in open-source projects and industry. Especially, the C preprocessor (CPP) is very popular in practice, but it is also gaining (again) interest in academia. Although there have been several attempts to understand and improve CPP, there is a lack of understanding of how it is used in open-source and industrial systems and whether different usage patterns have emerged. The background is that much research on configurable systems and product lines concentrates on open-source systems, simply because they are available for study in the first place. This leads to the potentially problematic situation that it is unclear whether the results obtained from these studies are transferable to industrial systems. We aim at lowering this gap by comparing the use of CPP in open-source projects and industry—especially from the embedded-systems domain—based on a substantial set of subject systems and well-known variability metrics, including size, scattering, and tangling metrics. A key result of our empirical study is that, regarding almost all aspects we studied, the analyzed open-source systems and the considered embedded systems from industry are similar regarding most metrics, including systems that have been developed in industry and made open source at some point. So, our study indicates that, regarding CPP as variability-implementation mechanism, insights, methods, and tools developed based on studies of open-source systems are transferable to industrial systems—at least, with respect to the metrics we considered.

A Catalogue of Refactorings to Remove Incomplete Annotations

Developers use the C Preprocessor (CPP) to handle portability and vari- ability in program families of different sizes and domains. However, despite the widely use of the CPP in practice, it is often criticised due to its negative impact on code quality and maintainability, tool development, and its error-prone characteristics. In particular, developers aggravate these problems when using incomplete annotations, i.e., directives encompassing only parts of syntactical units. In a previous work, we performed an empirical study on 41 C program family releases and found that almost 90% of syntax errors occur in incomplete annotations. There are some refactorings to remove incomplete annotations proposed in the literature. However, they clone code and increase Lines of Code (LOC). To avoid incomplete annotations and their intrinsic problems, in this article we propose a catalogue of refactorings that converts incomplete annotations into complete ones without cloning code. We implement an Eclipse plug-in to help developers applying our refactorings automatically. To evaluate our catalogue, we performed a study to analyse questions related to code cloning, LOC, and number of directives. To answer our research questions, we analyse releases of 12 C program families of different domains ranging from 4.9 thousand to 1.5 million LOC. The results show that our catalogue can remove all incomplete annotations without cloning code, and increasing only in 0.04% the LOC and in 2.10% the number of directives.

Does the discipline of preprocessor annotations matter?

The C preprocessor ( CPP ) is a simple and language-independent tool, widely used to implement variable software systems using conditional compilation (i.e., by including or excluding annotated code). Although CPP provides powerful means to express variability, it has been criticized for allowing arbitrary annotations that break the underlying structure of the source code. We distinguish between disciplined annotations, which align with the structure of the source code, and undisciplined annotations, which do not. Several studies suggest that especially the latter type of annotations makes it hard to (automatically) analyze the code. However, little is known about whether the type of annotations has an effect on program comprehension. We address this issue by means of a controlled experiment with human subjects. We designed similar tasks for both, disciplined and undisciplined annotations, to measure program comprehension. Then, we measured the performance of the subjects regarding correctness and response time for solving the tasks. Our results suggest that there are no differences between disciplined and undisciplined annotations from a program-comprehension perspective. Nevertheless, we observed that finding and correcting errors is a time-consuming and tedious task in the presence of preprocessor annotations.

Revealing and repairing configuration inconsistencies in large-scale system software

System software typically offers a large amount of compile-time options and variability. A good example is the Linux kernel, which provides more than 10,000 configurable features, growing rapidly. This allows users to tailor it with respect to a broad range of supported hardware architectures and application domains. From the maintenance point of view, compile-time configurability poses big challenges. The configuration model (the selectable features and their constraints as presented to the user) and the configurability that is actually implemented in the code have to be kept in sync, which, if performed manually, is a tedious and error-prone task. In the case of Linux, this has led to numerous defects in the source code, many of which are actual bugs. In order to ensure consistency between the variability expressed in the code and the configuration models, we propose an approach that extracts variability from both into propositional logic. This reveals inconsistencies between variability as expressed by the C Preprocessor (CPP) and an explicit variability model, which manifest themselves in seemingly conditional code that is in fact unconditional. We evaluate our approach with the Linux, for which our tool detects 1,766 configurability defects, which turned out as dead/superfluous source code and bugs. Our findings have led to numerous source-code improvements and bug fixes in Linux: 123 patches (49 merged) fix 364 defects, 147 of which have been confirmed by the corresponding Linux developers and 20 as fixing a previously unknown bug.

Efficient extraction and analysis of preprocessor-based variability

The C Preprocessor (CPP) is the tool of choice for the implementation of variability in many large-scale configurable software projects. Linux, probably the most-configurable piece of software ever, employs more than 10,000 preprocessor variables for this purpose. However, this de-facto variability tends to be "hidden in the code"; which on the long term leads to variability defects, such as dead code or inconsistencies with respect to the intended (modeled) variability of the software. This calls for tool support for the efficient extraction of (and reasoning over) CPP-based variability. We suggest a novel approach to extract CPP-based variability. Our tool transforms CPP-based variability in O(n) complexity into a propositional formula that "mimics" all valid effects of conditional compilation and can be analyzed with standard SAT or BDD packages. Our evaluation results demonstrate the scalability and practicability of the approach. A dead-block-analysis on the complete Linux source tree takes less than 30 minutes; we thereby have revealed 60 dead blocks, 2 of which meanwhile have been confirmed as new (and long-lasting) bugs; the rest is still under investigation.

Unified versioning through feature logic

Software configuration management (SCM) suffers from tight coupling between SCM version-ing models and the imposed SCM processes. In order to adapt SCM tools to SCM processes, rather than vice versa, we propose a unified versioning model, the version set model . Version sets denote versions, components, and configurations by feature terms , that is, Boolean terms over ( feature : value )-attributions. Through feature logic , we deduce consistency of abstract configurations as well as features of derived components and describe how features propagate in the SCM process; using feature implications , we integrate change-oriented and version-oriented SCM models. We have implemented the version set model in an SCM system called ICE, for Incremental Configuration Environment . ICE is based on a featured file system (FFS) , where version sets are accessed as virtual files and directories. Using the well-known C preprocessor (CPP) representation, users can view and edit multiple versions simultaneously, while only the differences between versions are stored. It turns out that all major SCM models can be realized and integrated efficiently on top of the FFS, demonstrating the flexible and unifying nature of the version set model.

Programmable syntax macros

Lisp has shown that a programmable syntax macro system acts as an adjunct to the compiler that gives the programmer important and powerful abstraction facilities not provided by the language. Unlike simple token substitution macros, such as are provided by CPP (the C preprocessor), syntax macros operate on Abstract Syntax Trees (ASTs). Programmable syntax macro systems have not yet been developed for syntactically rich languages such as C because rich concrete syntax requires the manual construction of syntactically valid program fragments, which is a tedious, difficult, and error prone process. Also, using two languages, one for writing the program, and one for writing macros, is another source of complexity. This research solves these problems by having the macro language be a minimal extension of the programming language, by introducing explicit code template operators into the macro language, and by using a type system to guarantee, at macro definition time, that all macros and macro functions only produce syntactically valid program fragments. The code template operators make the language context sensitive, which requires changes to the parser. The parser must perform type analysis in order to parse macro definitions, or to parse user code that invokes macros.