Traditional Software Engineering Research Articles

Comparative analysis of real issues in open-source machine learning projects

ContextIn the last decade of data-driven decision-making, Machine Learning (ML) systems reign supreme. Because of the different characteristics between ML and traditional Software Engineering systems, we do not know to what extent the issue-reporting needs are different, and to what extent these differences impact the issue resolution process.ObjectiveWe aim to compare the differences between ML and non-ML issues in open-source applied AI projects in terms of resolution time and size of fix. This research aims to enhance the predictability of maintenance tasks by providing valuable insights for issue reporting and task scheduling activities.MethodWe collect issue reports from Github repositories of open-source ML projects using an automatic approach, filter them using ML keywords and libraries, manually categorize them using an adapted deep learning bug taxonomy, and compare resolution time and fix size for ML and non-ML issues in a controlled sample.Result147 ML issues and 147 non-ML issues are collected for analysis. We found that ML issues take more time to resolve than non-ML issues, the median difference is 14 days. There is no significant difference in terms of size of fix between ML and non-ML issues. No significant differences are found between different ML issue categories in terms of resolution time and size of fix.ConclusionOur study provided evidence that the life cycle for ML issues is stretched, and thus further work is required to identify the reason. The results also highlighted the need for future work to design custom tooling to support faster resolution of ML issues.

Research on the Application of Project Teaching Method in the New Model of Software Engineering Course

Software engineering course is one of the core courses of computer science. The students trained in the current teaching mode can no longer meet the market demand for high-technology talents. Based on this, the research attempts to optimize the traditional software engineering teaching mode by using the project teaching method (PTM). According to the basic concept of PTM and course characteristics, the reform path of PTM in software engineering course is explored in the experiment. Then in the experiment, the indicators that affect teaching reform effect is selected and a evaluation model is built. And GA-BP algorithm is used to evaluate the effect of the evaluation model. To verify the performance of the built model and the final evaluation effect, the research results are tested from the fitness of the algorithm, error performance, prediction in the data set and other aspects. GA-BP algorithm converged when it iterated to the 18th generation, and the final fitness value was 0.61. The average error square value of GA-BP was 0.35 and the minimum error square sum of GA-BP was 0.48. Its prediction accuracy in test set and training set was 93.4% and 94.1% respectively. The maximum prediction error in the training sample was only 0.015, and the performance of the above data was better than the other three algorithms. To sum up, applying PTM to software engineering curriculum reform can achieve better teaching results.

GraphPrior: Mutation-based Test Input Prioritization for Graph Neural Networks

Graph Neural Networks (GNNs) have achieved promising performance in a variety of practical applications. Similar to traditional DNNs, GNNs could exhibit incorrect behavior that may lead to severe consequences, and thus testing is necessary and crucial. However, labeling all the test inputs for GNNs can be costly and time-consuming, especially when dealing with large and complex graphs, which seriously affects the efficiency of GNN testing. Existing studies have focused on test prioritization for DNNs, which aims to identify and prioritize fault-revealing tests (i.e., test inputs that are more likely to be misclassified) to detect system bugs earlier in a limited time. Although some DNN prioritization approaches have been demonstrated effective, there is a significant problem when applying them to GNNs: They do not take into account the connections (edges) between GNN test inputs (nodes), which play a significant role in GNN inference. In general, DNN test inputs are independent of each other, while GNN test inputs are usually represented as a graph with complex relationships between each test. In this article, we propose GraphPrior ( GNN -oriented Test Prior itization), a set of approaches to prioritize test inputs specifically for GNNs via mutation analysis. Inspired by mutation testing in traditional software engineering, in which test suites are evaluated based on the mutants they kill, GraphPrior generates mutated models for GNNs and regards test inputs that kill many mutated models as more likely to be misclassified. Then, GraphPrior leverages the mutation results in two ways, killing-based and feature-based methods. When scoring a test input, the killing-based method considers each mutated model equally important, while feature-based methods learn different importance for each mutated model through ranking models. Finally, GraphPrior ranks all the test inputs based on their scores. We conducted an extensive study based on 604 subjects to evaluate GraphPrior on both natural and adversarial test inputs. The results demonstrate that KMGP, the killing-based GraphPrior approach, outperforms the compared approaches in a majority of cases, with an average improvement of 4.76% ~49.60% in terms of APFD. Furthermore, the feature-based GraphPrior approach, RFGP, performs the best among all the GraphPrior approaches. On adversarial test inputs, RFGP outperforms the compared approaches across different adversarial attacks, with the average improvement of 2.95% ~46.69%.

Capstone Projects and their Transition into the Software Development Industry: A 10 Year Systematic Review of Literature

This study sought to examine the existing software engineering and Information Technology practices covered in final-year capstone projects in higher institutions of learning around the world, and the possibility of commercializing products developed from these projects. The researchers undertook a Systematic Literature Review methodology. The researchers used Artificial Intelligence (A.I.) tools, including Litmaps and Connected Papers for literature identification, while Quillbot and Scholarly A.I. were used for summary writing. Manual full paper review, data extraction and synthesis, and quantitative analyses were done on the final dataset using Microsoft Word, Excel, and Rayyan.ai. Eighty-eight papers from ACM Library, IEEE, Science Direct, EBSCOHost, Emerald Insight, Taylor & Francis, and Google Scholar were reviewed. Most papers were from Western countries, with only a few from Africa. Findings indicate a remarkable movement from traditional software engineering practices such as Waterfall to modern agile methods such as Scrum and Kaban. There was an increased use of Generative A.I. and Machine learning platforms such as GitHub and Arduino in projects profiled. Common scripting languages used by students were Java and Python. Most projects were undertaken in groups, with no prospective client. Most capstone courses were purely academic endeavours, and only a few were ever commercialized and transformed into industry products. One limitation of the review is that it includes grey literature from conferences and student theses. The study provides insights into State-of-the-Art approaches that can be adopted by higher institutions of learning to improve capstone courses.

Software Requirement Engineering: Traceability Techniques and Tools

Requirement Traceability is one of the activities in managing requirements. It is important for software projects and is affecting the quality of software products. Requirement Traceability is a method to analyze the effect of changes among various software development lifecycle parts. Agile methodologies have been presented as an alternative to traditional software engineering methodologies. The transformation between traditional and agile methodologies is a hard task so the need for traceability grows. This paper introduces traceability research at the requirement engineering on the traceability literature published during the last years. It also investigates and discusses the requirements for traceability issues. It finally presents several requirement traceability techniques and tools to support traceability.

The impact of security and privacy threat modeling on blockchain-enabled-electronic voting system

Blockchain is a decentralized, distributed ledger that records transactions between two parties. A blockchain-based software system is a new and innovative approach to software engineering that uses blockchain technology. This approach has several advantages over traditional software engineering approaches, including improved security and transparency. The most common software engineering approaches are waterfall, agile and hybrid models. Each of these has its strengths and weakness. A blockchain-based system has the advantage of being more secure and transparent than any of these approaches. It also can track changes more accurately, which can improve quality control. Blockchain technologies have incredible potential but also have some problems. One problem is security and privacy issues, which brings into question the resilience of existing security and trust mechanisms. The distributed application (dApp) framework for the proposed electronic voting system is built with the help of blockchain technology in this proposal. As a result, fewer crimes has committed against sensitive data during the electoral process because of immutability, transparency and privacy. Ganache, Metamask and hashing algorithms are used to develop the dApp. The paper's strengths lie in its ability to create and analyze threat models for blockchain-enabled-electronic voting systems and to identify the types of threats using Microsoft STRIDE.

Software Engineering for Responsible AI

The unique characteristics of artificial intelligence (AI) systems pose new challenges to traditional software engineering approaches. Thus, new software engineering approaches are required to develop AI systems in a responsible manner.

A probabilistic framework for mutation testing in deep neural networks

Context: Mutation Testing (MT) is an important tool in traditional Software Engineering (SE) white-box testing. It aims to artificially inject faults in a system to evaluate a test suite's capability to detect them, assuming that the test suite defects finding capability will then translate to real faults. If MT has long been used in SE, it is only recently that it started gaining the attention of the Deep Learning (DL) community, with researchers adapting it to improve the testability of DL models and improve the trustworthiness of DL systems. Objective: If several techniques have been proposed for MT, most of them neglected the stochasticity inherent to DL resulting from the training phase. Even the latest MT approaches in DL, which propose to tackle MT through a statistical approach, might give inconsistent results. Indeed, as their statistic is based on a fixed set of sampled training instances, it can lead to different results across instances set when results should be consistent for any instance. Methods: In this work, we propose a Probabilistic Mutation Testing (PMT) approach that alleviates the inconsistency problem and allows for a more consistent decision on whether a mutant is killed or not. Results: We show that PMT effectively allows a more consistent and informed decision on mutations through evaluation using three models and eight mutation operators used in previously proposed MT methods. We also analyze the trade-off between the approximation error and the cost of our method, showing that relatively small error can be achieved for a manageable cost. Conclusion: Our results showed the limitation of current MT practices in DNN and the need to rethink them. We believe PMT is the first step in that direction which effectively removes the lack of consistency across test executions of previous methods caused by the stochasticity of DNN training.

Asset Management in Machine Learning: State-of-research and State-of-practice

Machine learning components are essential for today’s software systems, causing a need to adapt traditional software engineering practices when developing machine-learning-based systems. This need is pronounced due to many development-related challenges of machine learning components such as asset, experiment, and dependency management. Recently, many asset management tools addressing these challenges have become available. It is essential to understand the support such tools offer to facilitate research and practice on building new management tools with native supports for machine learning and software engineering assets. This article positions machine learning asset management as a discipline that provides improved methods and tools for performing operations on machine learning assets. We present a feature-based survey of 18 state-of-practice and 12 state-of-research tools supporting machine-learning asset management. We overview their features for managing the types of assets used in machine learning experiments. Most state-of-research tools focus on tracking, exploring, and retrieving assets to address development concerns such as reproducibility, while the state-of-practice tools also offer collaboration and workflow-execution-related operations. In addition, assets are primarily tracked intrusively from the source code through APIs and managed via web dashboards or command-line interfaces (CLIs). We identify asynchronous collaboration and asset reusability as directions for new tools and techniques.

Uses of business process modeling in agile software development projects

Agile methodologies and frameworks are widely used in software development projects because of their support for continuous change and delivery. Agile software development advocates de-prioritizing aspects such as processes and documentation. In traditional software engineering methodologies, however, business process models have been extensively used to support these aspects. Up until now, it is unclear to what extent recommendations to focus on code imply that conceptual modeling should be discontinued.

Technology readiness levels for machine learning systems

The development and deployment of machine learning systems can be executed easily with modern tools, but the process is typically rushed and means-to-an-end. Lack of diligence can lead to technical debt, scope creep and misaligned objectives, model misuse and failures, and expensive consequences. Engineering systems, on the other hand, follow well-defined processes and testing standards to streamline development for high-quality, reliable results. The extreme is spacecraft systems, with mission critical measures and robustness throughout the process. Drawing on experience in both spacecraft engineering and machine learning (research through product across domain areas), we’ve developed a proven systems engineering approach for machine learning and artificial intelligence: the Machine Learning Technology Readiness Levels framework defines a principled process to ensure robust, reliable, and responsible systems while being streamlined for machine learning workflows, including key distinctions from traditional software engineering, and a lingua franca for people across teams and organizations to work collaboratively on machine learning and artificial intelligence technologies. Here we describe the framework and elucidate with use-cases from physics research to computer vision apps to medical diagnostics.

Editorial to theme section on open environmental software systems modeling

This theme section aims to disseminate the latest research results in the area of open environmental software systems modeling. Software-intensive systems, such as cyber-physical systems and self-adaptive systems, need to be able to constantly operate in open and dynamic environments, which requires them to continuously evolve, handle internal and external uncertainties, and so on. Traditional software engineering methodologies need to be extended for tackling these new challenges, and modeling is a promising technique to do that. This theme section contains two papers that tackle these issues, that were accepted after a thorough peer-reviewing process. Moreover, it contains an “Expert’s Voice” contribution on the uncertainty interaction problem, which is a relevant issue in the systems considered in this theme section.

Feature Interactions on Steroids: On the Composition of ML Models

One of the key differences between traditional software engineering and machine learning (ML) is the lack of specifications for ML models. Traditionally, specifications provide a cornerstone for compositional reasoning and for the divide-and-conquer strategy of how we build large and complex systems from components, but these are hard to come by for machine learned components. While the lack of specification seems like a fundamental new problem at first sight, in fact, software engineers routinely deal with iffy specifications in practice. We face weak specifications, wrong specifications, and unanticipated interactions among specifications. ML may push us further, but the problems are not fundamentally new. Rethinking ML model composition from the perspective of the feature-interaction problem highlights the importance of software design.

Fuzzing-based hard-label black-box attacks against machine learning models

Machine learning models are vulnerable to adversarial examples. We study the most realistic hard-label black-box attacks in this paper. The main limitation of the existing attacks is that they need a large number of model queries, making them inefficient and even infeasible in practice. Inspired by the very successful fuzz testing approach in traditional software engineering and computer security domains, we propose fuzzing-based hard-label black-box attacks against machine learning models. We design an AdvFuzzer to explore multiple paths between a source image and a guidance image, and design a LocalFuzzer to explore the nearby space around a given input for identifying potential adversarial examples. We demonstrate that our fuzzing attacks are feasible and effective in generating successful adversarial examples with significantly reduced number of model queries and L0 distance. More interestingly, given a successful example generated by either our or other attacks, LocalFuzzer can immediately generate more successful adversarial examples even with smaller L2 distance from the source example.

Leveraging Social Network Analysis for Crowdsourced Software Engineering Research

Crowdsourced software engineering (CSE) is an emerging area that has been gaining much attention in the last few years. It refers to the use of crowdsourcing techniques in software engineering activities, including requirements engineering, implementation, design, testing, and verification. CSE is an alternative to traditional software engineering and uses an open call to which online developers can respond to and obtain work on various tasks, as opposed to the assigning of tasks to in-house developers. The great benefits of CSE have attracted the attention of many researchers, and many studies have recently been carried out in the field. This research aims to analyze publications on CSE using social network analysis (SNA). A total of 509 CSE publications from six popular databases were analyzed to determine the characteristics of the collaborative networks of co-authorship of the research (i.e., the co-authors, institutions involved in co-authorship, and countries involved in co-authorship) and of the citation networks on which the publications of the studies are listed. The findings help identify CSE research productivity, trends, performances, community structures, and relationships between various collaborative patterns to provide a more complete picture of CSE research.

Decentralized Machine Autonomy for Manufacturing Servitization.

Blockchain ecosystems are rapidly maturing and meeting the needs of business environments (e.g., industry, manufacturing, and robotics). The decentralized approaches in industries enable novel business concepts, such as machine autonomy and servitization of manufacturing environments. Introducing the distributed ledger technology principles into the machine sharing and servitization economy faces several challenges, and the integration opens new interesting research questions. Our research focuses on data and event models and secure upgradeable smart contract platforms for machine servitization. Our research indicates that with the proposed approaches, we can efficiently separate on- and off-chain data and assure scalability of the DApp without compromising the trust. We demonstrate that the secure upgradeable smart contract platform, which was adapted for machine servitization, supports the business workflow and, at the same time, assures common identification and authorization of all the participants in the system, including people, devices, and legal entities. We present a hybrid decentralized application (DApp) for the servitization of 3D printing. The solution can be used for or easily adapted to other manufacturing domains. It comprises a modular, upgradeable smart contract platform and off-chain machine, customer and web management, and monitoring interfaces. We pay special attention to the data and event models during the design, which are fundamental for the hybrid data storage and DApp architecture and the responsiveness of off-chain interfaces. The smart contract platform uses a proxy contract to control the access of smart contracts and role-based access control in function calls for blockchain users. We deploy and evaluate the DApp in a consortium blockchain network for performance and privacy. All the actors in the solution, including the machines, are identified by their blockchain accounts and are compeers. Our solution thus facilitates integration with the traditional information-communication systems in terms of the hybrid architectures and security standards for smart contract design comparable to those in traditional software engineering.

Synergistically Employing User Stories and Use Cases in the Practice and Teaching of Systems Analysis and Design

Over the past three decades, user stories and use cases have become increasingly dominant systems analysis and design (SA&D) requirements techniques. Both support articulating functional requirements for software projects, although they evolved within different software development approaches—user stories from agile development and use cases from traditional software engineering—and they differ significantly in terms of their grammar, syntax, and the level of detail they can capture. As such, user stories and use cases are neither synonyms nor mutually exclusive alternatives. Rather, they can and should be complementary in the systems requirements process. Unfortunately, this mix of similarities and differences—coupled with a lack of formal standards for either—make understanding and synergistically employing user stories with use cases confusing and challenging for practitioners and students alike. To address this, this paper first provides a descriptive overview of the evolution of user stories, use cases, and their interrelationship. Second, it fills a gap in the literature by providing a prescriptive, detailed approach to employing user stories and use cases together. This prescriptive approach is illustrated via a comprehensive tutorial example, providing practitioners with actionable skills and SA&D teachers and students with a new pedagogical tool.

A Domain-Specific Language for Modeling IoT System Architectures That Support Monitoring

The Internet of Things (IoT) is a technological paradigm involved in a diversity of domains with favorable impacts on people’s daily lives and the development of industry and cities. Nowadays, one of the most critical challenges is developing software for IoT systems since the traditional Software Engineering methodologies and tools are unproductive in the face of the complex requirements resulting from the highly distributed, heterogeneous, and dynamic scenarios in which these systems operate. Model-Driven Engineering (MDE) emerges as an appropriate approach to abstract the complexity of IoT systems. However, there are no domain-specific languages (DSLs) aligned to standardized reference architectures for IoT. Furthermore, existing DSLs have an incomplete language to represent the IoT entities that may be needed at the edge, fog, and cloud layers to monitor IoT environments. Therefore, this paper proposes a domain-specific language named Monitor-IoT, which supports developers in designing multi-layer monitoring architectures for IoT systems with high abstraction, expressiveness, and flexibility. Monitor-IoT consists of a high-level visual modeling language and a metamodel aligned with the ISO/IEC 30141:2018 reference architecture. In addition, it provides a language capable of modeling architectures with a wide variety of digital entities and dataflows (synchronous and asynchronous) between them across the edge, fog, and cloud layers to support the monitoring of a diversity of IoT scenarios. The empirical evaluation of Monitor-IoT through the application of an experiment, which contemplates the use of the Technology Acceptance Model (TAM), demonstrates the intention of the participants to use this tool in the future since they consider it easy to use and useful.

Harmonica: A Framework for Semi‐automated Design and Implementation of Blockchain Applications

ABSTRACTDesigning blockchain‐based applications is a tedious task. Compared to traditional software engineering, software architects cannot rely on previous experiences or proven practices, often formalized as software patterns. Also, the selection of an adequate blockchain technology is difficult without deep knowledge of the technology. This paper introduces Harmonica, a framework for the design and implementation of a blockchain‐based application. This framework is divided in three parts: a decision‐making engine to recommend a blockchain technology and blockchain‐based software patterns relying on requirements, a configurator to generate code stubs and configuration files, and a knowledge base to support those tools.

The entrepreneurial logic of startup software development: A study of 40 software startups

Software startups are an essential source of innovation and software-intensive products. The need to understand product development in startups and to provide relevant support are highlighted in software research. While state-of-the-art literature reveals how startups develop their software, the reasons why they adopt these activities are underexplored. This study investigates the tactics behind software engineering (SE) activities by analyzing key engineering events during startup journeys. We explore how entrepreneurial mindsets may be associated with SE knowledge areas and with each startup case. Our theoretical foundation is based on causation and effectuation models. We conducted semi-structured interviews with 40 software startups. We used two-round open coding and thematic analysis to describe and identify entrepreneurial software development patterns. Additionally, we calculated an effectuation index for each startup case. We identified 621 events merged into 32 codes of entrepreneurial logic in SE from the sample. We found a systemic occurrence of the logic in all areas of SE activities. Minimum Viable Product (MVP), Technical Debt (TD), and Customer Involvement (CI) tend to be associated with effectual logic, while testing activities at different levels are associated with causal logic. The effectuation index revealed that startups are either effectuation-driven or mixed-logics-driven. Software startups fall into two types that differentiate between how traditional SE approaches may apply to them. Effectuation seems the most relevant and essential model for explaining and developing suitable SE practices for software startups.