Support Program Comprehension Research Articles

Trace visualization within the Software City metaphor: Controlled experiments on program comprehension

Especially with the rise of microservice architectures, software is hard to understand when just the static dependencies are known. The actual call paths and the dynamic behavior of the application are hidden behind network communication. To comprehend what is going on in the software the vast amount of runtime data (traces) needs to be reduced and visualized. This work explores more effective visualizations to support program comprehension based on runtime data. The pure DynaCity visualization supports understanding normal behavior, while DynaCity rc supports the comprehension of faulty behavior. DynaCity uses the city metaphor for visualization. Its novel trace visualization displays dynamic dependencies as arcs atop the city. To reduce the number of traces, DynaCity aggregates all requests between the same two components into one arc whose brightness reflects both the number and the total duration of the requests. DynaCity also encodes dynamic trace data in a heatmap that it uses to light up the building: the brighter a building is, the more active it is, i.e., the more and the longer the requests are that it receives and/or spawns. An additional color scheme reflects any error/status codes among the aggregated traces. In a controlled experiment, we compare our approach with a traditional trace visualization built into the same Software City but showing all dependencies (without aggregation) as individual arcs and also disabling the heatmap. We also report on a second study that evaluates if an error-based coloring of only the arcs is sufficient or if the buildings should also be colored. We call this extension DynaCity rc as it is meant to support r oot c ause analyses. The source code and the raw data of the quantitative evaluations are available from https://github.com/qaware/dynacity . We show quantitatively that a group of professional software developers who participated in a controlled experiment solve typical software comprehension tasks more correctly (11.7%) and also saved 5.83% of the total allotted time with the help of DynaCity and that they prefer it over the more traditional dynamic trace visualization. The color scheme based on HTTP error codes in DynaCity rc supports developers when performing root cause analyses, as the median of them stated that the visualization helped them much in solving the tasks. The evaluation also shows that subjects using DynaCity rc with colored arcs and buildings find the responsible component 26.2% and the underlying root cause 33.3% more correctly than the group with just colored arcs. They also ranked it 40% more helpful to color both. The DynaCity visualization helps professional software engineers to understand the dynamic behavior of a software system better and faster. The color encoding of error codes in DynaCity rc also helps them with root cause analyses. • DynaCity helps developers in software comprehension. • View is less cluttered by aggregating traces into representative arcs and by mapping activities to the brightness of both arcs and buildings. • Extension DynaCity rc supports developers in root cause analyses. • Better to map error codes to the colors of arcs and also buildings. • Controlled experiments with up to 30 professionals justify claims quantitatively.

What do class comments tell us? An investigation of comment evolution and practices in Pharo Smalltalk

ContextPrevious studies have characterized code comments in various programming languages, showing how high quality of code comments is crucial to support program comprehension activities, and to improve the effectiveness of maintenance tasks. However, very few studies have focused on understanding developer practices to write comments. None of them has compared such developer practices to the standard comment guidelines to study the extent to which developers follow the guidelines.ObjectiveTherefore, our goal is to investigate developer commenting practices and compare them to the comment guidelines.MethodThis paper reports the first empirical study investigating commenting practices in Pharo Smalltalk. First, we analyze class comment evolution over seven Pharo versions. Then, we quantitatively and qualitatively investigate the information types embedded in class comments. Finally, we study the adherence of developer commenting practices to the official class comment template over Pharo versions.ResultsOur results show that there is a rapid increase in class comments in the initial three Pharo versions, while in subsequent versions developers added comments to both new and old classes, thus maintaining a similar code to comment ratio. We furthermore found three times as many information types in class comments as those suggested by the template. However, the information types suggested by the template tend to be present more often than other types of information. Additionally, we find that a substantial proportion of comments follow the writing style of the template in writing these information types, but they are written and formatted in a non-uniform way.ConclusionThe results suggest the need to standardize the commenting guidelines for formatting the text, and to provide headers for the different information types to ensure a consistent style and to identify the information easily. Given the importance of high-quality code comments, we draw numerous implications for developers and researchers to improve the support for comment quality assessment tools.

ProMeTA: a taxonomy for program metamodels in program reverse engineering

To support program comprehension, maintenance, and evolution, metamodels are frequently used during program reverse engineering activities to describe and analyze constituents of a program and their relations. Reverse engineering tools often define their own metamodels according to the intended purposes and features. Although each metamodel has its own advantages, its limitations may be addressed by other metamodels. Existing works have evaluated and compared metamodels and tools, but none have considered all the possible characteristics and limitations to provide a comprehensive guideline for classifying, comparing, reusing, and extending program metamodels. To aid practitioners and researchers in classifying, comparing, reusing, and extending program metamodels and their corresponding reverse engineering tools according to the intended goals, we establish a conceptual framework with definitions of program metamodels and related concepts. We confirmed that any reverse engineering activity can be clearly described as a pattern based on the framework from the viewpoint of program metamodels. Then the framework is used to provide a comprehensive taxonomy, named Program Metamodel TAxonomy (ProMeTA), which incorporates newly identified characteristics into those stated in previous works, which were identified via a systematic literature review (SLR) on program metamodels, while keeping the orthogonality of the entire taxonomy. Additionally, we validate the taxonomy in terms of its orthogonality and usefulness through the classification of popular metamodels.

Recovery of User Interface Web Design Patterns using Regular Expressions

User Interface Web Design Patterns are standard solutions for the development of web applications. The recovery of these patterns from web applications supports program comprehension, reusability, reverse engineering, re-engineering, and maintenance of legacy web applications. The recovery of patterns from web applications becomes arduous due to the heterogeneous nature of web applications. Authors presented different catalogs and recovery approaches for extracting User Interface Web Design Patterns from source code in last one and half decade. There is still a lack of formal specifications for web design patterns, which are important for their recovery from source code. The objective of this paper is to specify User Interface Web Design Patterns (UIWDP) using semiformal specification technique and use these specifications for the recovery of patterns from the source code of web applications using regular expressions. 55 feature types are identified for the specification of 15 UIWDPs. We evaluated our approach on 75 randomly selected web applications and recovered 15 UIWDPs. The standard deviation, precision, recall and F-score measures are used to evaluate the accuracy of our approach.

Pattern detection for conceptual schema recovery in data-intensive systems

In this paper, an approach for information systems reverse engineering is proposed and applied. The aim is to support a unified perspective to the reverse engineering process of both data and software. At the state of the art, indeed, many methods, techniques, and tools for software reverse engineering have been proposed to support program comprehension, software maintenance, and software evolution. Other approaches and tools have been proposed for data reverse engineering, with the aim, for example, to provide complete and up-to-date documentation of legacy databases. However, the two engineering communities often worked independently, and very few approaches addressed the reverse engineering of both data and software as information system's constituencies. Hence, a higher integration is needed to support a better co-evolution of databases and programs, in an environment often characterized by high availability of data and volatility of information flows. Accordingly, the approach we propose leverages the detection of object-relational mapping design patterns to build a conceptual schema of the software under analysis. Then, the conceptual schema is mapped to the domain model of the system, to support the design of the evolution of the information system itself. The approach is evaluated on two large-scale open-source enterprise applications. Copyright © 2014 John Wiley & Sons, Ltd.

设计模式指导的软件分簇方法

Program comprehension is the process of achieving knowledge information from computer program. It aims to gain all the information from the program in order to understand the intention of the developers so as to support reviewing, defect localization, reverse-engineering, re-engineering, porting, maintenance, reuse and other software engineering activities of existing or legacy systems. This paper proposes a design pattern directed software clustering approach to support program comprehension, which employs design pattern information hidden in programs to direct software clustering processes, helping program developers and maintainers rapidly and conveniently comprehend open source code using the divide and conquer strategy. Firstly, we utilize existing design pattern detection techniques to identify design pattern information from source code. Then, we direct software clustering using the identi ed design pattern information based on the inter-role relationship among design pattern roles. Last, we visualize clustering result to support program understanding. We also developed a prototype tool, TasteJ, and conducted controlled human subject experiments. The results of our experiments demonstrate the feasibility of our approach.

Labeling source code with information retrieval methods: an empirical study

To support program comprehension, software artifacts can be labeled--for example within software visualization tools--with a set of representative words, hereby referred to as labels. Such labels can be obtained using various approaches, including Information Retrieval (IR) methods or other simple heuristics. They provide a bird-eye's view of the source code, allowing developers to look over software components fast and make more informed decisions on which parts of the source code they need to analyze in detail. However, few empirical studies have been conducted to verify whether the extracted labels make sense to software developers. This paper investigates (i) to what extent various IR techniques and other simple heuristics overlap with (and differ from) labeling performed by humans; (ii) what kinds of source code terms do humans use when labeling software artifacts; and (iii) what factors--in particular what characteristics of the artifacts to be labeled--influence the performance of automatic labeling techniques. We conducted two experiments in which we asked a group of students (38 in total) to label 20 classes from two Java software systems, JHotDraw and eXVantage. Then, we analyzed to what extent the words identified with an automated technique--including Vector Space Models, Latent Semantic Indexing (LSI), latent Dirichlet allocation (LDA), as well as customized heuristics extracting words from specific source code elements--overlap with those identified by humans. Results indicate that, in most cases, simpler automatic labeling techniques--based on the use of words extracted from class and method names as well as from class comments--better reflect human-based labeling. Indeed, clustering-based approaches (LSI and LDA) are more worthwhile to be used for source code artifacts having a high verbosity, as well as for artifacts requiring more effort to be manually labeled. The obtained results help to define guidelines on how to build effective automatic labeling techniques, and provide some insights on the actual usefulness of automatic labeling techniques during program comprehension tasks.

The Solid* toolset for software visual analytics of program structure and metrics comprehension: From research prototype to product

Software visual analytics (SVA) tools combine static program analysis and fact extraction with information visualization to support program comprehension. However, building efficient and effective SVA tools is highly challenging, as it involves extensive software development in program analysis, graphics, information visualization, and interaction. We present a SVA toolset for software maintenance, and detail two of its components which target software structure, metrics and code duplication. We illustrate the toolset’s usage for constructing software visualizations with examples in education, research, and industrial contexts. We discuss the design evolution from research prototypes to integrated, scalable, and easy-to-use products, and present several guidelines for the development of efficient and effective SVA solutions.

Scenario-based and value-based specification mining: better together

Specification mining takes execution traces as input and extracts likely program invariants, which can be used for comprehension, verification, and evolution related tasks. In this work we integrate scenario-based specification mining, which uses a data-mining algorithm to suggest ordering constraints in the form of live sequence charts, an inter-object, visual, modal, scenario-based specification language, with mining of value-based invariants, which detects likely invariants holding at specific program points. The key to the integration is a technique we call scenario-based slicing, running on top of the mining algorithms to distinguish the scenario-specific invariants from the general ones. The resulting suggested specifications are rich, consisting of modal scenarios annotated with scenario-specific value-based invariants, referring to event parameters and participating object properties.We have implemented the mining algorithm and the visual presentation of the mined scenarios within a standard development environment. An evaluation of our work over a number of case studies shows promising results in extracting expressive specifications from real programs, which could not be extracted previously. The more expressive the mined specifications, the higher their potential to support program comprehension and testing.

Supporting program comprehension in large preprocessor-based software product lines

Software product line (SPL) engineering provides an effective mechanism to implement variable software. However, using preprocessors to realise variability, which is typical in industry, is heavily criticised, because it often leads to obfuscated code. Using background colours to highlight code annotated with preprocessor statements to support comprehensibility has proved to be effective, however, scalability to large SPLs is questionable. The authors’ aim is to implement and evaluate scalable usage of background colours for industrial-sized SPLs. They designed and implemented scalable concepts in a tool called FeatureCommander. To evaluate its effectiveness, the authors conducted a controlled experiment with a large real-world SPL with over 99 000 lines of code and 340 features. They used a within-subjects design with treatment colours and no colours. They compared correctness and response time of tasks for both treatments. For certain kinds of tasks, background colours improve program comprehension. Furthermore, the subjects generally favour background colours compared with no background colours. In addition, the subjects who worked with background colours had to use the search functions less frequently. The authors show that background colours can improve program comprehension in large SPLs. Based on these encouraging results, they continue their work on improving program comprehension in large SPLs.

Spy: A flexible code profiling framework

Code profiling is an essential activity to increase software quality. It is commonly employed in a wide variety of tasks, such as supporting program comprehension, determining execution bottlenecks, and assessing code coverage by unit tests. Spy is an innovative framework to easily build profilers and visualize profiling information. The profiling information is obtained by inserting dedicated code before or after method execution. The gathered profiling information is structured in line with the application structure in terms of packages, classes, and methods. Spy has been instantiated on four occasions so far. We created profilers dedicated to test coverage, time execution, type feedback, and profiling evolution across version. We also integrated Spy in the Pharo IDE. Spy has been implemented in the Pharo Smalltalk programming language and is available under the MIT license.

Understanding the relevance of micro-structures for design patterns detection

One important issue concerning design patterns in reverse engineering is their detection to support program comprehension, design recovery, system (re-)documentation, and software evolution. The objectives of this paper are to identify and analyze different types of building blocks of design patterns and to evaluate if the detection of these building blocks (called micro-structures) is relevant for the detection of occurrences of the design patterns. This analysis is useful to understand how the different types of micro-structures can be combined to better comprehend design patterns and to improve their detection. To achieve the objectives, the paper provides a description of different micro-structures, an analysis of their relevance in different design motifs, and a statistical analysis on the number and types of micro-structures present in different design patterns. Finally, we investigate if the detection of some design patterns can be performed only through the detection of a combined set of micro-structures, or other techniques should be exploited.

Introduction to the special issue on program comprehension through dynamic analysis (PCODA)

Introduction to the special issue on program comprehension through dynamic analysis (PCODA)

The impacts of function extraction technology on program comprehension: A controlled experiment

Program comprehension is a critical, time-consuming, and highly error-prone task for software developers. Function extraction (FX) is a theory and technology that automates and supports program comprehension by calculating the semantic behaviors of programs at many levels of abstraction and displaying those behaviors in a standard, readable format in terms of the “as-built” specification of the program. In this experimental study, developers using an FX prototype tool to assist them in determining the behavior of software modules have significantly more effective program comprehension, in both increased accuracy of understanding and reduced time on task. Moreover, developers have a positive reaction toward the use of the FX technology, and use of FX does not reduce their overall comprehension of the program.

Structure and Properties of Traces for Functional Programs

The tracer Hat records in a detailed trace the computation of a program written in the lazy functional language Haskell. The trace can then be viewed in various ways to support program comprehension and debugging. The trace was named the augmented redex trail. Its structure was inspired by standard graph rewriting implementations of functional languages. Here we describe a model of the trace that captures its essential properties and allows formal reasoning. The trace is a graph constructed by graph rewriting but goes beyond simple term graphs. Although the trace is a graph whose structure is independent of any rewriting strategy, we define the trace inductively, thus giving us a powerful method for proving its properties.

Recovering UML class models from C++: A detailed explanation

An approach to recovering design-level UML class models from C++ source code to support program comprehension is presented. A set of mappings are given that focus on accurately identifying such elements as relationship types, multiplicities, and aggregation semantics. These mappings are based on domain knowledge of the C++ language and common programming conventions and idioms. Additionally, formal concept analysis is used to detect design-level attributes of UML classes. An application implementing these mappings is used to reverse engineer a moderately sized, open-source application and the resultant class model is compared against those produced by other UML reverse engineering tools. This comparison shows that the presented mapping rules effectively produce meaningful and semantically accurate UML models.

Theories, tools and research methods in program comprehension: past, present and future

Program comprehension research can be characterized by both the theories that provide rich explanations about how programmers understand software, as well as the tools that are used to assist in comprehension tasks. In this paper, I review some of the key cognitive theories of program comprehension that have emerged over the past thirty years. Using these theories as a canvas, I then explore how tools that are commonly used today have evolved to support program comprehension. Specifically, I discuss how the theories and tools are related and reflect on the research methods that were used to construct the theories and evaluate the tools. The reviewed theories and tools are distinguished according to human characteristics, program characteristics, and the context for the various comprehension tasks. Finally, I predict how these characteristics will change in the future and speculate on how a number of important research directions could lead to improvements in program comprehension tool development and research methods.

Web application comprehension based on dependence analysis

Many research indicate a lot of money and time are spent on maintaining and modifying program delivered. So the policies to support program comprehension are very important. Program comprehension is a crucial and difficult task. Insufficient design, illogical code structure, short documents will enhance the comprehensive difficulty. Developing Web application is usually a process with quick implementation and delivery. In addition, generally a Web application is coded by combining mark language statements with some embedded applets. Such programming mode affects comprehension of Web applications disadvantageously. This paper proposes a method to improving understanding Web by dependence analysis and slice technology.

Supporting program comprehension using dominance trees

This paper will address how program comprehension can be aided within software maintenance through the use of high level representations of code, specifically the use of dominance trees. The paper proposes that dominance trees should be used as a standard approach within the program comprehension process. A number of ways in which the dominance trees are supportive of the comprehension process are highlighted. The evaluation considers if dominance trees are supportive of standard program comprehension strategies such as an asdneeded strategy, whether they provide an early identification of the consequences of changes and highlight areas where preventative maintenance is necessary. Their actual usefulness in achieving these goals is evaluated through the use of a number of case studies. The results have shown that there are a number of specific tasks performed during program comprehension that can be aided through the use of dominance trees.