Code Artifacts Research Articles

Sharing practices of software artefacts and source code for reproducible research

AbstractWhile source code of software and algorithms depicts an essential component in all fields of modern research involving data analysis and processing steps, it is uncommonly shared upon publication of results throughout disciplines. Simple guidelines to generate reproducible source code have been published. Still, code optimization supporting its repurposing to different settings is often neglected and even less thought of to be registered in catalogues for a public reuse. Though all research output should be reasonably curated in terms of reproducibility, it has been shown that researchers are frequently non-compliant with availability statements in their publications. These do not even include the use of persistent unique identifiers that would allow referencing archives of code artefacts at certain versions and time for long-lasting links to research articles. In this work, we provide an analysis on current practices of authors in open scientific journals in regard to code availability indications, FAIR principles applied to code and algorithms. We present common repositories of choice among authors. Results further show disciplinary differences of code availability in scholarly publications over the past years. We advocate proper description, archiving and referencing of source code and methods as part of the scientific knowledge, also appealing to editorial boards and reviewers for supervision.

An empirical study on the state-of-the-art methods for requirement-to-code traceability link recovery

Requirements-to-code traceability link recovery (RC-TLR) can establish connections between requirements and target code artifacts, which is critical for the maintenance and evolution of large software systems. However, to the best of our knowledge, there is no existing experimental study focused on state-of-the-art (SOTA) methods for the RC-TLR problem, and there is also a lack of uniform benchmarks for evaluating new methods in the field. We developed a framework to identify SOTA methods using the Systematic Literature Review method and applied it to research in the RC-TLR field from 2018 to 2023. Through experiments replication on 13 datasets using 6 methods, we observed that for information retrieval-based methods, Close Relations between Target artifacts-based method (CRT), TraceAbility Recovery by Consensual biTerms (TAROT), and Fine-grained TLR (FTLR) performed well on COEST dataset, while Combining Part-Of-Speech with information-retrieval techniques (Conpos) and TAROT achieve promising results in large datasets. As concerns machine learning-based methods, Random Forest consistently exhibits strong performances on all datasets. We hope that this study can provide a comparative benchmark for performance evaluation in the RC-TLR field. The resource repository that we have established is expected to alleviate the workload of researchers in performance analysis, and promote progress of the field.

On Formal Methods Thinking in Computer Science Education

Formal Methods (FM) radically improve the quality of the code artefacts they help to produce. They are simple, probably accessible to first-year undergraduate students and certainly to second-year students and beyond. Nevertheless, in many cases, they are not part of a general recommendation for course curricula, i.e., they are not taught — and yet they are valuable. One reason for this is that teaching “Formal Methods” is often confused with teaching logic and theory. This paper advocates what we call FM thinking : the application of ideas from Formal Methods applied in informal, lightweight, practical and accessible ways. And we will argue here that FM thinking should be part of the recommended curriculum for every Computer Science student. For even students who train only in that “thinking” will become much better programmers. But there will be others who, exposed to those ideas, will be ideally positioned to go further into the more theoretical background: why the techniques work; how they can be automated; and how new ones can be developed. Those students would follow subsequently a specialised, more theoretical stream, including topics such as semantics, logics, verification and proof-automation techniques.

Mining for cost awareness in the infrastructure as code artifacts of cloud-based applications: An exploratory study

Context:Cloud computing’s rise as the primary platform for software development and delivery is largely driven by the potential cost savings. However, it is surprising that no empirical evidence has been collected to determine whether cost awareness permeates the development process and how it manifests in practice. Objective:This study aims to provide empirical evidence of cost awareness by mining open source repositories of cloud-based applications. The focus is on Infrastructure-as-Code artifacts that automate software (re)deployment on the cloud. Methods:A systematic examination of 152735 repositories yielded 2010 relevant hits. We then analyzed 538 relevant commits and 208 relevant issues using inductive and deductive coding and corroborated findings with discussions from Stack Overflow. Results:The findings indicate that developers are not only concerned with the cost of their application deployments but also take actions to reduce these costs beyond selecting cheaper cloud services. We also identify research areas for future consideration. Conclusion:Although we focus on a particular Infrastructure-as-Code technology (Terraform), the findings can be applicable to cloud-based application development in general. The provided empirical grounding can serve developers seeking to reduce costs through service selection, resource allocation, deployment optimization, and other techniques.

LPMSAEF: Lightweight process mining-based software architecture evaluation framework for security and performance analysis

The article discusses the need for a lightweight software architecture evaluation framework that can address practitioners' concerns. Specifically, the proposed framework uses process mining and Petri nets to analyze security and performance in software development's early and late stages. Moreover, the framework has been implemented in six case studies, and the results show that it is a feasible and effective solution that can detect security and performance issues in complex and heterogeneous architecture with less time and effort. Furthermore, the article provides a detailed explanation of the framework's features, factors, and evaluation criteria. Additionally, this article discusses the challenges associated with traditional software architecture documentation methods using Unified Modeling Language diagrams and the limitations of code alone for creating comprehensive Software Architecture models. Various methods have been developed to extract implicit Software Architecture from code artifacts, but they tend to produce code-oriented diagrams instead of Software Architecture diagrams. Therefore, to bridge the model-code gap, the article proposes a framework that considers existing Software Architecture in the source code as architectural components and focuses on Software Architecture behaviors for analyzing performance and security. The proposed framework also suggests comparing Software Architecture extracted by different Process Mining algorithms to achieve consensus on architecture descriptions, using visualizations to understand differences and similarities. Finally, the article suggests that analyzing the previous version of a system's Software Architecture can lead to improvements and deviations from planned Software Architecture can be detected using traceability approaches to aid software architects in detecting inconsistencies.

Variability modeling of products, processes, and resources in cyber–physical production systems engineering

Cyber-Physical Production Systems (CPPSs), such as automated car manufacturing plants, execute a configurable sequence of production steps to manufacture products from a product portfolio. In CPPS engineering, domain experts start with manually determining feasible production step sequences and resources based on implicit knowledge. This process is hard to reproduce and highly inefficient. In this paper, we present the Extended Iterative Process Sequence Exploration (eIPSE) approach to derive variability models for products, processes, and resources from a domain-specific description. To automate the integrated exploration and configuration process for a CPPS, we provide a toolchain which automatically reduces the configuration space and allows to generate CPPS artifacts, such as control code for resources. We evaluate the approach with four real-world use cases, including the generation of control code artifacts, and an observational user study to collect feedback from engineers with different backgrounds. The results confirm the usefulness of the eIPSE approach and accompanying prototype to straightforwardly configure a desired CPPS.

Improvised Software Code Comprehension Using Data Mining

Millions of lines of code are used to create the modern software applications, which are more complicated in terms of their structure, behaviour, and functionality. The rapid advancement of supporting and enabling technologies, for example, is one reason why the development life cycles of these applications show a propensity to get shorter. As a result, a growing amount of the expense associated with software development moves from the generation of new artefacts to their adaption. Understanding the layout, functionality, and behaviour of current code artefacts is essential to this activity. The task of understanding code is crucial to software maintenance. We employed data mining techniques including clustering, classification, and associative rules to improvise software code comprehension

Decomposing Monolithic to Microservices: Keyword Extraction and BFS Combination Method to Cluster Monolithic’s Classes

Abstract 
 Microservices architecture is widely used because of the ease of maintaining its microservices, as opposed to encapsulating functionality in a monolithic, which may negatively impact the development process when the application continues to grow. The migration process from a monolithic architecture to microservices became necessary, but it often relies on the architect's intuition only, which may cost many resources. A method to assist developers in decomposing monolithic into microservices is proposed to address that problem. Unlike the existing methods that often rely on non-source code artifacts which may lead into inaccurate decomposition if the artifacts do not reflect the latest condition of the monolith, the proposed method relies on analyzing the application source code to produce a grouping recommendation for building microservices. By using specific keyword extraction followed by Breadth First Search traversal with certain rules, the proposed method decomposed the monolith's component into several cluster whose majority of cluster’s members have uniform business domain. Based on the experiment, the proposed method got an 0.81 accuracy mean in grouping monolithic's components with similar business domain, higher than the existing decomposition method's score. Further research is recommended to be done to increase the availability of the proposed method.

ROMEO: A binary vulnerability detection dataset for exploring Juliet through the lens of assembly language

ContextAutomatic vulnerability detection on C/C++ source code has benefitted from the introduction of machine learning to the field, with many recent publications targeting this combination. In contrast, assembly language or machine code artifacts receive less attention, although there are compelling reasons to study them. They are more representative of what is executed, more easily incorporated in dynamic analysis, and in the case of closed-source code, there is no alternative. ObjectiveWe evaluate the representative capability of assembly language compared to C/C++ source code for vulnerability detection. Furthermore, we investigate the role of call graph context in detecting function-spanning vulnerabilities. Finally, we verify whether compiling a benchmark dataset compromises an experiment’s soundness by inadvertently leaking label information. MethodWe propose ROMEO, a publicly available, reproducible and reusable binary vulnerability detection benchmark dataset derived from the synthetic Juliet test suite. Alongside, we introduce a simple text-based assembly language representation that includes context for function-spanning vulnerability detection and semantics to detect high-level vulnerabilities. It is constructed by disassembling the .text segment of the respective binaries. ResultsWe evaluate an x86 assembly language representation of the compiled dataset, combined with an off-the-shelf classifier. It compares favorably to state-of-the-art methods, including those operating on the full C/C++ code. Including context information using the call graph improves detection of function-spanning vulnerabilities. There is no label information leaked during the compilation process. ConclusionPerforming vulnerability detection on a compiled program instead of the source code is a worthwhile tradeoff. While certain information is lost, e.g., comments and certain identifiers, other valuable information is gained, e.g., about compiler optimizations.

Software Defect Prediction and Analysis Using Enhanced Random Forest (extRF) Technique: A Business Process Management and Improvement Concept in IOT-Based Application Processing Environment

Software defect prediction is a thriving study area in the realm of software engineering and processing in the IOT-based environment. Defect prediction creates a list of defective source code artifacts so that quality assurance companies may successfully assign limited methods for certifying programming things by investing more effort into the bad source code. Defect prediction can assist estimate maintenance times, which can help with quality assurance, dependability, security, and cost reduction. Many predictions in IOT-based processing environment and business process management and enhancement challenges still exist in defect expectation ponders, and there are various noteworthy concerns. In addition, it is difficult to apply these methodologies practically because most of the investigations verified in open-source programming ventures with the goal that present forecast models might not work for other programming items including business programming. Investigating security issues in cross-project deformity expectation is required since if we have more accessible restrictive datasets, the assessment of forecast models will be more stable. In general, every defect is essential regarding quality, reliability, security, and cost-effectiveness. Therefore, an enhanced and improved maintenance schedule is required to acknowledge forecasting techniques. Therefore, in this article, we have evaluated different Semi-Supervised Learning (SSL) techniques, among which Extended Random Forest (extRF) technique is one for defective system prediction. The Extended Random Forest (extRF) technique is the extended form of the Random Forest (RF), which is a supervised learning technique into semi-supervised learning getting the hang of refining every arbitrary tree given an individual-training worldview. An enhancing technique is recommended, and a weighted mixture of irregular trees creates the final forecast results.

TCTracer: Establishing test-to-code traceability links using dynamic and static techniques

Test-to-code traceability links model the relationships between test artefacts and code artefacts. When utilised during the development process, these links help developers to keep test code in sync with tested code, reducing the rate of test failures and missed faults. Test-to-code traceability links can also help developers to maintain an accurate mental model of the system, reducing the risk of architectural degradation when making changes. However, establishing and maintaining these links manually places an extra burden on developers and is error-prone. This paper presents TCTracer, an approach and implementation for the automatic establishment of test-to-code traceability links. Unlike existing work, TCTracer operates at both the method level and the class level, allowing us to establish links between tests and functions, as well as between test classes and tested classes. We improve over existing techniques by combining an ensemble of new and existing techniques that utilise both dynamic and static information and exploiting a synergistic flow of information between the method and class levels. An evaluation of TCTracer using five large, well-studied open source systems demonstrates that, on average, we can establish test-to-function links with a mean average precision (MAP) of 85% and test-class-to-class links with an MAP of 92%.

Further investigation of the survivability of code technical debt items

AbstractContext: Technical debt (TD) discusses the negative impact of sub‐optimal decisions to cope with the need‐for‐speed in software development. Code technical debt items (TDI) are atomic elements of TD that can be observed in code artifacts. Empirical results on open‐source systems demonstrated how code‐smells, which are just one type of TDIs, are introduced and “survive” during release cycles. However, little is known about whether the results on the survivability of code‐smells hold for other types of code TDIs (i.e., bugs and vulnerabilities) and in industrial settings. Goal: Understanding the survivability of code TDIs by conducting an empirical study analyzing two industrial cases and 31 open‐source systems from Apache Foundation. Method: We analyzed 133,670 code TDIs (35,703 from the industrial systems) detected by SonarQube (in 193,196 commits) to assess their survivability using survivability models. Results: In general, code TDIs tend to remain and linger for long periods in open‐source systems, whereas they are removed faster in industrial systems. Code TDIs that survive over a certain threshold tend to remain much longer, which confirms previous results. Our results also suggest that bugs tend to be removed faster, while code smells and vulnerabilities tend to survive longer.

Towards Balanced Defect Prediction with Better Information Propagation

Defect prediction, the task of predicting the presence of defects in source code artifacts, has broad application in software development. Defect prediction faces two major challenges, label scarcity, where only a small percentage of code artifacts are labeled, and data imbalance, where the majority of labeled artifacts are non-defective. Moreover, current defect prediction methods ignore the impact of information propagation among code artifacts and this negligence leads to performance degradation. In this paper, we propose DPCAG, a novel model to address the above three issues. We treat code artifacts as nodes in a graph, and learn to propagate influence among neighboring nodes iteratively in an EM framework. DPCAG dynamically adjusts the contributions of each node and selects high-confidence nodes for data augmentation. Experimental results on real-world benchmark datasets show that DPCAG improves performance compare to the state-of-the-art models. In particular, DPCAG achieves substantial performance superiority when measured by Matthews Correlation Coefficient (MCC), a metric that is widely acknowledged to be the most suitable for imbalanced data.

Guiding feature model evolution by lifting code-level dependencies

Feature models are a de facto standard for representing the commonalities and variability of product lines and configurable software systems. Requirements-level features are commonly implemented in multiple source code artifacts, resulting in complex dependencies at the code level. As developers add and evolve features frequently, it is challenging to keep feature models consistent with their implementation. This article thus presents an approach combining feature-to-code mappings and code dependency analyses to inform engineers about possible inconsistencies. We focus on code-level changes requiring updates in feature dependencies and constraints. Our approach uses static code analysis and a variation control system to lift complex code-level dependencies to feature models. We present the suggested dependencies to the engineer in two ways: directly as links between features in a feature model and as a heatmap visualizing the dependency changes of all features in a model. We evaluate our approach by analyzing the evolution history of two case study systems: the Pick-and-Place Unit and the KePlast product line of our industry partner KEBA AG. We present results of evaluating the utility (RQ1) and performance (RQ2) of our approach, as well as the quality of the suggestions (RQ3).

Within-Project Defect Prediction of Infrastructure-as-Code Using Product and Process Metrics

Infrastructure-as-code (IaC) is the DevOps practice enabling management and provisioning of infrastructure through the definition of machine-readable files, hereinafter referred to as <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">IaC scripts</i> . Similarly to other source code artefacts, these files may contain defects that can preclude their correct functioning. In this paper, we aim at assessing the role of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">product</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">process</i> metrics when predicting defective IaC scripts. We propose a fully integrated machine-learning framework for IaC Defect Prediction, that allows for repository crawling, metrics collection, model building, and evaluation. To evaluate it, we analyzed 104 projects and employed five machine-learning classifiers to compare their performance in flagging suspicious defective IaC scripts. The key results of the study report <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Random Forest</small> as the best-performing model, with a median AUC-PR of 0.93 and MCC of 0.80. Furthermore, at least for the collected projects, product metrics identify defective IaC scripts more accurately than process metrics. Our findings put a baseline for investigating IaC Defect Prediction and the relationship between the product and process metrics, and IaC scripts’ quality.

An Empirical Study on Type Annotations

Type annotations connect variables to domain-specific types. They enable the power of type checking and can detect faults early. In practice, type annotations have a reputation of being burdensome to developers. We lack, however, an empirical understanding of how and why they are burdensome. Hence, we seek to measure the baseline accuracy and speed for developers making type annotations to previously unseen code. We also study the impact of one or more type suggestions. We conduct an empirical study of 97 developers using 20 randomly selected code artifacts from the robotics domain containing physical unit types. We find that subjects select the correct physical type with just 51% accuracy, and a single correct annotation takes about 2 minutes on average. Showing subjects a single suggestion has a strong and significant impact on accuracy both when correct and incorrect, while showing three suggestions retains the significant benefits without the negative effects. We also find that suggestions do not come with a time penalty. We require subjects to explain their annotation choices, and we qualitatively analyze their explanations. We find that identifier names and reasoning about code operations are the primary clues for selecting a type. We also examine two state-of-the-art automated type annotation systems and find opportunities for their improvement.

Survey on software defect prediction techniques

ABSTRACT Recent advancements in technology have emerged the requirements of hardware and software applications. Along with this technical growth, software industries also have faced drastic growth in the demand of software for several applications. For any software industry, developing good quality software and maintaining its eminence for user end is considered as most important task for software industrial growth. In order to achieve this, software engineering plays an important role for software industries. Software applications are developed with the help of computer programming where codes are written for desired task. Generally, these codes contain some faulty instances which may lead to the buggy software development cause due to software defects. In the field of software engineering, software defect prediction is considered as most important task which can be used for maintaining the quality of software. Defect prediction results provide the list of defect-prone source code artefacts so that quality assurance team scan effectively allocate limited resources for validating software products by putting more effort on the defect-prone source code. As the size of software projects becomes larger, defect prediction techniques will play an important role to support developers as well as to speed up time to market with more reliable software products. One of the most exhaustive and pricey part of embedded software development is consider as the process of finding and fixing the defects. Due to complex infrastructure, magnitude, cost and time limitations, monitoring and fulfilling the quality is a big challenge, especially in automotive embedded systems. However, meeting the superior product quality and reliability is mandatory. Hence, higher importance is given to V&V (Verification & Validation). Software testing is an integral part of software V&V, which is focused on promising accurate functionality and long-term reliability of software systems. Simultaneously, software testing requires much effort, cost, infrastructure and expertise as the development. The costs and efforts elevate in safety critical software systems. Therefore, it is essential to have a good testing strategy for any industry with high software development costs. In this work, we are planning to develop an efficient approach for software defect prediction by using soft computing based machine learning techniques which helps to predict optimize the features and efficiently learn the features.

RSTrace+: Reviewer suggestion using software artifact traceability graphs

Various types of artifacts (requirements, source code, test cases, documents, etc.) are produced throughout the lifecycle of a software. These artifacts are connected with each other via traceability links that are stored in modern application lifecycle management repositories. Throughout the lifecycle of a software, various types of changes can arise in any one of these artifacts. It is important to review such changes to minimize their potential negative impacts. To make sure the review is conducted properly, the reviewer(s) should be chosen appropriately. We previously introduced a novel approach, named RSTrace, to automatically recommend reviewers that are best suited based on their familiarity with a given artifact. In this study, we introduce an advanced version of RSTrace, named RSTrace+ that accounts for recency information of traceability links including practical tool support for GitHub. In this study, we conducted a series of experiments on finding the appropriate code reviewer(s) using RSTrace+ and provided a comparison with the other code reviewer recommendation approaches. We had initially tested RSTrace+ on an open source project (Qt 3D Studio) and achieved a top-3 accuracy of 0.89 with an MRR (mean reciprocal ranking) of 0.81. In a further empirical evaluation of 40 open source projects, we compared RSTrace+ with Naive-Bayes, RevFinder and Profile based approach, and observed higher accuracies on the average. We confirmed that the proposed reviewer recommendation approach yields promising top-k and MRR scores on the average compared to the existing reviewer recommendation approaches. Unlike other code reviewer recommendation approaches, RSTrace+ is not limited to recommending reviewers for source code artifacts and can potentially be used for recommending reviewers for other types of artifacts. Our approach can also visualize the affected artifacts and help the developer to make assessments of the potential impacts of change to the reviewed artifact.

A feature location approach for mapping application features extracted from crowd-based screencasts to source code

Crowd-based multimedia documents such as screencasts have emerged as a source for documenting requirements, the workflow and implementation issues of open source and agile software projects. For example, users can show and narrate how they manipulate an application’s GUI to perform a certain functionality, or a bug reporter could visually explain how to trigger a bug or a security vulnerability. Unfortunately, the streaming nature of programming screencasts and their binary format limit how developers can interact with a screencast’s content. In this research, we present an automated approach for mining and linking the multimedia content found in screencasts to their relevant software artifacts and, more specifically, to source code. We apply LDA-based mining approaches that take as input a set of screencast artifacts, such as GUI text and spoken word, to make the screencast content accessible and searchable to users and to link it to their relevant source code artifacts. To evaluate the applicability of our approach, we report on results from case studies that we conducted on existing WordPress and Mozilla Firefox screencasts. We found that our automated approach can significantly speed up the feature location process. For WordPress, we find that our approach using screencast speech and GUI text can successfully link relevant source code files within the top 10 hits of the result set with median Reciprocal Rank (RR) of 50% (rank 2) and 100% (rank 1). In the case of Firefox, our approach can identify relevant source code directories within the top 100 hits using screencast speech and GUI text with the median RR = 20%, meaning that the first true positive is ranked 5 or higher in more than 50% of the cases. Also, source code related to the frontend implementation that handles high-level or GUI-related aspects of an application is located with higher accuracy. We also found that term frequency rebalancing can further improve the linking results when using less noisy scenarios or locating less technical implementation of scenarios. Investigating the results of using original and weighted screencast data sources (speech, GUI, speech and GUI) that can result in having the highest median RR values in both case studies shows that speech data is an important information source that can result in having RR of 100%.

Supporting making online: the role of artifact, teacher and peer interactions in crafting electronic textiles

PurposeThe purpose of this paper is to report changes when a classroom-based makerspace moved from face-to-face to an online setting.Design/methodology/approachTo better understand changes in teaching maker activities, as they move from face-to-face to online contexts, the authors analyzed video and interview data from six weeks of an introductory computer science high school classroom (38 youth) that was implementing an electronic textiles unit, shifting to asynchronous online teaching and learning during the March 2020 state-wide school closure because of the pandemic. The authors analyzed field notes and videos of face-to-face and online interactions between the teacher and his students in learning to craft and code their electronic textiles projects.FindingsThe analysis revealed changes in the role of physical and code artifacts, in improvising teaching, and channels for communication between the teacher and students.Research limitations/implicationsThis study discusses the implications for future pedagogical design and research efforts, as the authors continue to engage youth and work toward designing equitable learning opportunities with maker activities online.Originality/valueIn maker activities such as electronic textiles, youth design, sew and program circuits to make personalized three-dimensional, textile artifacts. However, nearly all research on supporting and teaching making has been conducted in face-to-face settings.