Software Engineering Artifacts Research Articles

Artificial Intelligence for Software Engineering: An Initial Review on Software Bug Detection and Prediction

The need for speed and quality in delivering all software engineering artifacts has inevitably remained the biggest challenge in today’s software development environment. While everyone caters to complex software engineering processes, new releases are expected by the market on almost a daily basis. Thus, several Artificial Intelligence (AI) techniques have been introduced that are intensively used in the modern software engineering industry to fulfill market needs. This paper presents the initial results of our review work on software bug detection and prediction studies using AI techniques. Our focus is to (i) identify factors affecting the effectiveness of current software bug detection and prediction techniques and (ii) identify the effectiveness of AI techniques in improving current software bug detection and prediction techniques. The evidence showed that the software engineering domain has utilized artificial intelligence approaches and techniques to facilitate the complex tasks of software bug detection and bug prediction. It mainly demonstrates the significance of merging artificial intelligence with the software engineering domain in terms of reduced overhead and efficient results to enhance the quality of software products.

Exploring the efficacy of transfer learning in mining image-based software artifacts

BackgroundTransfer learning allows us to train deep architectures requiring a large number of learned parameters, even if the amount of available data is limited, by leveraging existing models previously trained for another task. In previous attempts to classify image-based software artifacts in the absence of big data, it was noted that standard off-the-shelf deep architectures such as VGG could not be utilized due to their large parameter space and therefore had to be replaced by customized architectures with fewer layers. This proves to be challenging to empirical software engineers who would like to make use of existing architectures without the need for customization.FindingsHere we explore the applicability of transfer learning utilizing models pre-trained on non-software engineering data applied to the problem of classifying software unified modeling language (UML) diagrams. Our experimental results show training reacts positively to transfer learning as related to sample size, even though the pre-trained model was not exposed to training instances from the software domain. We contrast the transferred network with other networks to show its advantage on different sized training sets, which indicates that transfer learning is equally effective to custom deep architectures in respect to classification accuracy when large amounts of training data is not available.ConclusionOur findings suggest that transfer learning, even when based on models that do not contain software engineering artifacts, can provide a pathway for using off-the-shelf deep architectures without customization. This provides an alternative to practitioners who want to apply deep learning to image-based classification but do not have the expertise or comfort to define their own network architectures.

The Impact of Lexicon Adaptation on the Emotion Mining From Software Engineering Artifacts

Sentiment analysis and emotion mining techniques are increasingly being used in the field of software engineering. However, the experiments conducted so far have not yielded high accuracy results. Researchers indicate a lack of adaptation of the methods of emotion mining to the specific context of the domain as the main cause of this situation. The article describes research aimed at examining whether the adaptation of the lexicon with emotional intensity of words in the context of software engineering improves the reliability of sentiment analysis. For this purpose, a new lexicon is developed in which words are evaluated as if they were used in the field of software engineering. A comparative experiment of emotion mining based on a generic and a software engineering specific lexicon does not reveal any significant differences in the results.

Semi-automatic rule-based domain terminology and software feature-relevant information extraction from natural language user manuals

Mature software systems comprise a vast number of heterogeneous system capabilities which are usually requested by different groups of stakeholders and which evolve over time. Software features describe and bundle low level capabilities logically on an abstract level and thus provide a structured and comprehensive overview of the entire capabilities of a software system. Software features are often not explicitly managed. Quite the contrary, feature-relevant information is often spread across several software engineering artifacts (e.g., user manual, issue tracking systems). It requires huge manual effort to identify and extract feature-relevant information from these artifacts in order to make feature knowledge explicit. In this paper we present a two-step-approach to extract feature-relevant information from a user manual: First we semi-automatically extract a domain terminology from a natural language user manual based on linguistic patterns. Then, we apply natural language processing techniques based on the extracted domain terminology and structural sentence information. Our approach is able to extract atomic feature-relevant information with an F1-score of at least 92.00%. We describe the implementation of the approach as well as evaluations based on example sections of a user manual taken from industry.

Software product lines traceability: A systematic mapping study

Context: Traceability in Software Product Lines (SPL) is the ability to interrelate software engineering artifacts through required links to answer specific questions related to the families of products and underlying development processes. Despite the existence of studies to map out available evidence on traceability for single systems development, there is a lack of understanding on common strategies, activities, artifacts, and research gaps for SPL traceability.Objective: This paper analyzes 62 studies dating from 2001 to 2015 and discusses seven aspects of SPL traceability: main goals, strategies, application domains, research intensity, research challenges, rigor, and industrial relevance. In addition to the analysis, this paper also synthesizes the available evidence, identifies open issues and points out areas calling for further research.Method: To gather evidence, we defined a mapping study process adapted from existing guidelines. Driven by a set of research questions, this process comprises three major phases: planning, conducting, and documenting the review.Results: This work provides a structured understanding of SPL traceability, indicating areas for further research. The lack of evidence regarding the application of research methods indicates the need for more rigorous SPL traceability studies with better description of context, study design, and limitations. For practitioners, although most identified studies have low industrial relevance, a few of them have high relevance and thus could provide some decision making support for application of SPL traceability in practice.Conclusions: This work concludes that SPL traceability is maturing and pinpoints areas where further investigation should be performed. As future work, we intend to improve the comparison between traceability proposals for SPL and single-system development.

Effective use of analysts’ effort in automated tracing

Because of the large amount of effort it takes to manually trace requirements, automated traceability methods for mapping textual software engineering artifacts to each other and generating candidate links have received increased attention over the past 15 years. Automatically generated links, however, are viewed as candidates until human analysts confirm/reject them for the final requirements traceability matrix. Studies have shown that analysts are a fallible, but necessary, participant in the tracing process. There are two key measures guiding analyst work on the evaluation of candidate links: accuracy of analyst decision and efficiency of their work. Intuitively, it is expected that the more effort the analyst spends on candidate link validation, the more accurate the final traceability matrix is likely to be, although the exact nature of this relationship may be difficult to gauge outright. To assist analysts in making the best use of their time when reviewing candidate links, prior work simulated four possible behaviors and showed that more structured approaches save the analysts’ time/effort required to achieve certain levels of accuracy. However, these behavioral simulations are complex to run and their results difficult to interpret and use in practice. In this paper, we present a mathematical model for evaluating analyst effort during requirements tracing tasks. We apply this model to a simulation study of 12 candidate link validation approaches. The simulation study is conducted on a number of different datasets. In each study, we assume perfect analyst behavior (i.e., analyst always being correct when making a determination about a link). Under this assumption, we evaluate the estimated effort for the analyst and plot it against the accuracy of the recovered traceability matrix. The effort estimation model is guided by a parameter specifying the relationship between the time it takes an analyst to evaluate a presented link and the time it takes an analyst to discover a link not presented to her. We construct a series of effort estimations based on different values of the model parameter. We found that the analysts’ approach to candidate link validation—essentially the order in which the analyst examines presented candidate links—does impact the effort. We also found that the lowest ratio of the cost of finding a correct link from scratch over the cost of recognizing a correct link yields the lowest effort for all datasets, but that the lowest effort does not always yield the highest quality matrix. We finally observed that effort varies by dataset. We conclude that the link evaluation approach we call “Top 1 Not Yet Examined Feedback Pruning” was the overall winner in terms of effort and highest quality and, thus, should be followed by human analysts if possible.

HOD2MLC

Ontologies have gained increasing usage to augment an application with domain knowledge, particularly in healthcare, where they represent knowledge ranging from: bioinformatics data such as protein, gene, etc. to biomedical informatics such as diseases, diagnosis, symptoms, etc. However, the current ontology development efforts and process are data intensive and construction based, creating ontologies for specific applications/requirements, rather than designing an abstract ontological solution(s) that can be reusable across the domain using a well-defined design process. To address this deficiency, the work presented herein positions ontologies as software engineering artifact that allows them to be placed into the position to share the captured domain conceptualization and its vocabulary involving disparate domain backgrounds, that can then be created, imported, exported and re-used using different frameworks, tools and techniques. Towards this end, the authors propose an agile software process for ontologies referred to as the Hybrid Ontology Design & Development Model with Lifecycle,HOD2MLC. To place HOD2MLC into a proper perspective, they explore, compare, and contrast it to existing ontology design and development alternatives with respect their various phases as related to the authors' work and phases in varied SDP models.

Assessing the quality factors found in in-line documentation written in natural language: The JavadocMiner

An important software engineering artifact used by developers and maintainers to assist in software comprehension and maintenance is source code documentation. It provides the insight needed by software engineers when performing a task, and therefore ensuring the quality of this documentation is extremely important. In-line documentation is at the forefront of explaining a programmer's original intentions for a given implementation. Since this documentation is written in natural language, ensuring its quality so far needed to be performed manually. In this paper, we present an effective and automated approach for assessing the quality of in-line documentation using a set of heuristics, targeting both the quality of language and consistency between the source code and its comments. Our evaluation is made up of two parts: We first apply the JavadocMiner tool to the different modules of two open source applications (ArgoUML and Eclipse) in order to automatically assess their intrinsic comment quality. In the second part of our evaluation, we correlate the results returned by the analysis with bug defects reported for the individual modules in order to examine connections between natural language documentation and source code quality.

Embracing Composite Metrics in Software Experiments

Traditionally most Software Engineering experiments tend to formulate hypotheses and analyze an independent variable or a series of independent variables. This approach greatly reduces the type of research questions which can be explored. In addition, most Software Engineering situations are highly complex with many intertwined or ill-defined concepts, processes and “objects”. Hence, the question arises: are independent variables really sufficient for describing Software Engineering situations? This paper argues that the community needs to consider fuzzier models for Software Engineering artifacts, especially it recommends using composite indices as a mechanism to allow the greater exploration of the experimental design space. However, this extension is not without its risks; and hence in conjunction, it explains how to utilize analysis safeguards (sensitivity and uncertainty analysis) to explore any effects introduced when utilizing experimental formulations with composite metrics.

Automatized Checking of Business Rules for Activity Execution Sequence in Workflows

In the present context of high competition, the enterprises are investing in the improvement of the efficiency of their production and management processes. To accomplish such tasks, one viable way is to formalize those processes into business models, so it is possible to evaluate and improve them. As important as the analysis of software engineering artifacts prior to building information systems, also is important the analysis of the definitions existing in the business models, to ensure that they reflect the reality of the process and its demands. This paper aims to develop an approach for the execution of validations of execution rules incorporated to workflows, using model checking to ensure the exactitude of the analysis.

Automated Abstractions for Contract Validation

Pre/postcondition-based specifications are commonplace in a variety of software engineering activities that range from requirements through to design and implementation. The fragmented nature of these specifications can hinder validation as it is difficult to understand if the specifications for the various operations fit together well. In this paper, we propose a novel technique for automatically constructing abstractions in the form of behavior models from pre/postcondition-based specifications. Abstraction techniques have been used successfully for addressing the complexity of formal artifacts in software engineering; however, the focus has been, up to now, on abstractions for verification. Our aim is abstraction for validation and hence, different and novel trade-offs between precision and tractability are required. More specifically, in this paper, we define and study enabledness-preserving abstractions, that is, models in which concrete states are grouped according to the set of operations that they enable. The abstraction results in a finite model that is intuitive to validate and which facilitates tracing back to the specification for debugging. The paper also reports on the application of the approach to two industrial strength protocol specifications in which concerns were identified.

Assessing traceability of software engineering artifacts

The generation of traceability links or traceability matrices is vital to many software engineering activities. It is also person-power intensive, time-consuming, error-prone, and lacks tool support. The activities that require traceability information include, but are not limited to, risk analysis, impact analysis, criticality assessment, test coverage analysis, and verification and validation of software systems. Information Retrieval (IR) techniques have been shown to assist with the automated generation of traceability links by reducing the time it takes to generate the traceability mapping. Researchers have applied techniques such as Latent Semantic Indexing (LSI), vector space retrieval, and probabilistic IR and have enjoyed some success. This paper concentrates on examining issues not previously widely studied in the context of traceability: the importance of the vocabulary base used for tracing and the evaluation and assessment of traceability mappings and methods using secondary measures. We examine these areas and perform empirical studies to understand the importance of each to the traceability of software engineering artifacts.

A survey on model versioning approaches

PurposeThe purpose of this paper is to provide a feature‐based characterization of version control systems (VCSs), providing an overview about the state‐of‐the‐art of versioning systems dedicated to modeling artifacts.Design/methodology/approachBased on a literature study of existing approaches, a description of the features of versioning systems is established. Special focus is set on three‐way merging which is an integral component of optimistic versioning. This characterization is employed on current model versioning systems, which allows the derivation of challenges in this research area.FindingsThe results of the evaluation show that several challenges need to be addressed in future developments of VCSs and merging tools in order to allow the parallel development of model artifacts.Practical implicationsMaking model‐driven engineering (MDE) a success requires supporting the parallel development of model artifacts as is done nowadays for text‐based artifacts. Therefore, model versioning capabilities are a must for leveraging MDE in practice.Originality/valueThe paper gives a comprehensive overview of collaboration features of VCSs for software engineering artifacts in general, discusses the state‐of‐the‐art of systems for model artifacts, and finally, lists urgent challenges, which have to be considered in future model versioning system for realizing MDE in practice.

Protocols in the use of empirical software engineering artifacts

If empirical software engineering is to grow as a valid scientific endeavor, the ability to acquire, use, share, and compare data collected from a variety of sources must be encouraged. This is nec...

A design for team peer code review

The software industry needs our graduates to have significant and meaningful experiences with teamwork. A new design has been developed for a teamwork exercise based on peer code review. This design uses the three Ss of building assignments for cooperative learning: Same problem, Specific choices, and Simultaneous report. Students perform peer code review individually, and within and between stable small groups. The code can be sanitized student work or may be altered by the instructor to meet specific course learning objectives. The review is done in three phases. First, individuals answer yes or no to questions that evaluate the quality of the design and code. Then each group answers the same set of questions. When the groups have completed their evaluation, the class joins together to reveal and discuss the answers. This design was successfully implemented during the 2003-2004 academic year in an introductory programming class. This design is applicable to more advanced classes with significant programming assignments. Future work includes extending this structure to give students experience with other software engineering artifacts early in their academic careers.

A comprehensive evaluation of capture-recapture models for estimating software defect content

An important requirement to control the inspection of software artifacts is to be able to decide, based on more objective information, whether the inspection can stop or whether it should continue to achieve a suitable level of artifact quality. A prediction of the number of remaining defects in an inspected artifact can be used for decision making. Several studies in software engineering have considered capture-recapture models to make a prediction. However, few studies compare the actual number of remaining defects to the one predicted by a capture-recapture model on real software engineering artifacts. The authors focus on traditional inspections and estimate, based on actual inspections data, the degree of accuracy of relevant state-of-the-art capture-recapture models for which statistical estimators exist. In order to assess their robustness, we look at the impact of the number of inspectors and the number of actual defects on the estimators' accuracy based on actual inspection data. Our results show that models are strongly affected by the number of inspectors, and therefore one must consider this factor before using capture-recapture models. When the number of inspectors is too small, no model is sufficiently accurate and underestimation may be substantial. In addition, some models perform better than others in a large number of conditions and plausible reasons are discussed. Based on our analyses, we recommend using a model taking into account that defects have different probabilities of being detected and the corresponding Jackknife Estimator. Furthermore, we calibrate the prediction models based on their relative error, as previously computed on other inspections. We identified theoretical limitations to this approach which were then confirmed by the data.

IESEM: INTEGRATED ENVIRONMENT FOR SOFTWARE EVOLUTION MANAGEMENT

Software evolution has no common paradigm which practitioners can adhere to. On the contrary, there is a wide range of models, methods, techniques, and tools which are selected according to the specific task, the application domain, the professional experience, and the organizational culture. We argue that different approaches and technologies may be combined into a unique platform to satisfy the needs of software systems which evolve over long periods of time. This paper presents the Integrated Environment for Software Evolution Management (IESEM) which includes software repositories, reverse engineering tools, rationale capture tools, software measurement tools, and a user-friendly interface. It can manage heterogeneous systems characterized by various design methods and programming languages. IESEM is based on a central repository which stores software engineering artifacts, program code, design, and implementation decisions in the form of a traceability graph. The repository stores also software measures computed both from programs and external CASE repositories. Measures are used to control software degradation during its evolution and to support decisions based on quality factors. The key concepts of IESEM, its design, and implementation are presented. The use of IESEM during development and maintenance is discussed. A case study shows IESEM's effectiveness in performing maintenance tasks.