Software Engineering Practices Research Articles

Thoughts on applicability

The gulf between industry and academic interests in Software Engineering appears to continue to widen. With software taking on an ever-growing role in society, it is somewhat concerning that the interests and research focuses of academic software engineers seem to rarely overlap with industry interests. In this editorial, I hypothesize four major concerns I have with the applicability of much of the research in this area. I also surface the important meta question: What is Software Engineering Research? Is this the study of software engineers, or the study of best practices in software engineering? Does academic SE research owe more to anthropology, or the study of what makes software successful?

WASMICO: Micro-containers in microcontrollers with WebAssembly

The Internet-of-Things (IoT) has created a complex environment where hardware and software interact in complex ways. Despite being a prime candidate for applying well-established software engineering practices, IoT has not seen the same level of adoption as other areas, such as cloud development. This discrepancy is even more evident in the case of edge devices, where programming and managing applications can be challenging due to their heterogeneous nature and dependence on specific toolchains and languages. However, the emergence of WebAssembly as a viable solution for running high-level languages on some devices presents an opportunity to streamline development practices, such as DevOps. In this paper, we present WASMICO — a firmware and command-line utility that allows for the execution and management of application lifecycles in microcontrollers. Our solution has been benchmarked against other state-of-the-art tools, demonstrating its feasibility, novel features, and empirical evidence that it outperforms some of the most widely used solutions for running high-level code on these devices. Overall, our work aims to promote the use of well-established software engineering practices in the IoT domain, helping to bridge the gap between cloud and edge development.

The never-ending story–How companies transition to and sustain continuous software engineering practices

The never-ending story–How companies transition to and sustain continuous software engineering practices

Taxonomy of inline code comment smells

Code comments play a vital role in source code comprehension and software maintainability. It is common for developers to write comments to explain a code snippet, and commenting code is generally considered a good practice in software engineering. However, low-quality comments can have a detrimental effect on software quality or be ineffective for code understanding. This study aims to create a taxonomy of inline code comment smells and determine how frequently each smell type occurs in software projects. We conducted a multivocal literature review to define the initial taxonomy of inline comment smells. Afterward, we manually labeled 2447 inline comments from eight open-source projects where half of them were Java, and another half were Python projects. We created a taxonomy of 11 inline code comment smell types and found out that the smells exist in both Java and Python projects with varying degrees. Moreover, we conducted an online survey with 41 software practitioners to learn their opinions on these smells and their impact on code comprehension and software maintainability. The survey respondents generally agreed with the taxonomy; however, they reported that some smell types might have a positive effect on code comprehension in certain scenarios. We also opened pull requests and issues fixing the comment smells in the sampled projects, where we got a 27% acceptance rate. We share our manually labeled dataset online and provide implications for software engineering practitioners, researchers, and educators.

On the accessibility of the grayscale version of a PoN‐enabled use case notation

AbstractThe physics of notations (PoN) is a framework for designing cognitively effective notations. Cognitive effectiveness refers to the ease (speed and accuracy) by which model readers read models. In 2019, a PoN‐Enabled notation for use case diagrams was proposed. It has been empirically proven that the cognitive effectiveness of this new notation is superior to the original use case notation. The new notation however relies in part on the use of color. The accessibility of this new design has not been validated for use by the colorblind community, or any users who need to use a grayscale version of the models created. The PoN framework was not designed to explicitly account for the colorblind community or users who can only access grayscale versions of models. Inclusivity should not be an afterthought in software engineering practice or research. To this end, this paper empirically evaluates the cognitive effectiveness of the PoN‐enabled use case notation in situations where it is viewed in grayscale.

Detecting and resolving feature envy through automated machine learning and move method refactoring

Efficiently identifying and resolving code smells enhances software project quality. This paper presents a novel solution, utilizing automated machine learning (AutoML) techniques, to detect code smells and apply move method refactoring. By evaluating code metrics before and after refactoring, we assessed its impact on coupling, complexity, and cohesion. Key contributions of this research include a unique dataset for code smell classification and the development of models using AutoGluon for optimal performance. Furthermore, the study identifies the top 20 influential features in classifying feature envy, a well-known code smell, stemming from excessive reliance on external classes. We also explored how move method refactoring addresses feature envy, revealing reduced coupling and complexity, and improved cohesion, ultimately enhancing code quality. In summary, this research offers an empirical, data-driven approach, integrating AutoML and move method refactoring to optimize software project quality. Insights gained shed light on the benefits of refactoring on code quality and the significance of specific features in detecting feature envy. Future research can expand to explore additional refactoring techniques and a broader range of code metrics, advancing software engineering practices and standards.

The work of art in the age of artificial intelligibility

AbstractThe emergence of complex deep-learning models capable of producing novel images on a practically innumerable number of subjects and in an equally wide variety of artistic styles is beginning to highlight serious inadequacies in the ethical, aesthetic, epistemological and legal frameworks we have so far used to categorise art. To begin tackling these issues and identifying a role for AI in the production and protection of human artwork, it is necessary to take a multidisciplinary approach which considers current legal precedents, the practice of software engineering, historical attitudes towards technological innovation and a sustained technical analysis of the models themselves. This paper queries the location and nature of substantive artistic work in the developmental stages of an AI-generated image, offering critiques of existing assumptions and posing questions for future research. The emergence of convincing AI creative output, artistic or literary, has significant long-term implications for the humanities, including the need for re-appraisal of foundational ideas about authorship and creativity in general. The effects of artificial intelligence, whether generalised or task-specific, cannot be ignored or displaced now that easy-access, scalable image and text production is a reality.

Security for Machine Learning-based Software Systems: A Survey of Threats, Practices, and Challenges

The rapid development of Machine Learning (ML) has demonstrated superior performance in many areas, such as computer vision and video and speech recognition. It has now been increasingly leveraged in software systems to automate the core tasks. However, how to securely develop the machine learning-based modern software systems (MLBSS) remains a big challenge, for which the insufficient consideration will largely limit its application in safety-critical domains. One concern is that the present MLBSS development tends to be rushed, and the latent vulnerabilities and privacy issues exposed to external users and attackers will be largely neglected and hard to be identified. Additionally, machine learning-based software systems exhibit different liabilities towards novel vulnerabilities at different development stages from requirement analysis to system maintenance, due to its inherent limitations from the model and data and the external adversary capabilities. The successful generation of such intelligent systems will thus solicit dedicated efforts jointly from different research areas, i.e., software engineering, system security, and machine learning. Most of the recent works regarding the security issues for ML have a strong focus on the data and models, which has brought adversarial attacks into consideration. In this work, we consider that security for machine learning-based software systems may arise from inherent system defects or external adversarial attacks, and the secure development practices should be taken throughout the whole lifecycle. While machine learning has become a new threat domain for existing software engineering practices, there is no such review work covering the topic. Overall, we present a holistic review regarding the security for MLBSS, which covers a systematic understanding from a structure review of three distinct aspects in terms of security threats. Moreover, it provides a thorough state-of-the-practice for MLBSS secure development. Finally, we summarize the literature for system security assurance and motivate the future research directions with open challenges. We anticipate this work provides sufficient discussion and novel insights to incorporate system security engineering for future exploration.

Eliciting and modeling emotional requirements: a systematic mapping review.

Considering users' emotions plays an extremely crucial role in the adoption and acceptance of recent technology by the end user. User emotions can also help to identify unknown requirements, saving resources that would otherwise be wasted if discovered later. However, eliciting and modeling users' emotional requirements in software engineering is still an open research area. This systematic mapping review analyzes emotional requirements (ER) practices in software engineering from two perspectives: elicitation and modeling. For elicitation techniques, we investigate the techniques, evaluation methods, limitations, and application domains. For modeling techniques, we examine the modeling languages, analyses, limitations, and domains. We systematically reviewed studies on emotional requirements engineering published between 1993-2023 and identified 46 relevant primary studies. A total of 34 studies investigated ER elicitation techniques, five examined modeling techniques, and seven covered both. Illustrative case studies were the main evaluation method for proposed elicitation techniques. Identified limitations include time consumption and extensive human involvement. The dominant application domains were healthcare and well-being, and game development. This review summarizes the current landscape of emotional requirements research, highlighting key elicitation and modeling techniques, evaluations, limitations, and domains. Further research can build on these findings to advance emotional requirements practices in software engineering. Future research may address (1) managing conflicting emotional requirements across users, (2) evaluating the value and impact of considering emotional requirements during the development and (3) Modeling and analyzing emotional requirements in relation to other requirements.

Continuous Software Engineering Practices in {AI/ML} Development Past the Narrow Lens of {MLOps}: {A}doption Challenges

Background: Continuous software engineering practices are currently considered state of the art in Software Engineering (SE). Recently, this interest in continuous SE has extended to ML system development as well, primarily through MLOps. However, little is known about continuous SE in ML development outside the specific continuous practices present in MLOps. Aim: In this paper, we explored continuous SE in ML development more generally, outside the specific scope of MLOps. We sought to understand what challenges organizations face in adopting all the 13 continuous SE practices identified in existing literature. Method: We conducted a multiple case study of organizations developing ML systems. Data from the cases was collected through thematic interviews. The interview instrument focused on different aspects of continuous SE, as well as the use of relevant tools and methods. Results: We interviewed 8 ML experts from different organizations. Based on the data, we identified various challenges associated with the adoption of continuous SE practices in ML development. Our results are summarized through 7 key findings. Conclusion: The largest challenges we identified seem to stem from communication issues. ML experts seem to continue to work in silos, detached from both the rest of the project and the customers.

Process mining software engineering practices: A case study for deployment pipelines

Context:In mature software development organizations the ci/cd pipeline is the only route to deploy software into production. While the workflow of this process seems straightforward, the reality is different since exceptions and deviations are the norm in actual industry practice. In this context, Process Mining appears as a promising technique to uncover deviations and check compliance with standardized DevOps processes, and highlight bottlenecks and potential improvement areas. Objective:This paper presents a case study designed to assess the potential of using Process Mining techniques to provide visibility into the deployment pipeline. Method:This research uses raw event data extracted from the continuous practices toolchain, which is then used to compute a comprehensive set of DevOps-specific metrics, thus supporting objective monitoring of the quality and efficiency of the deployment workflow. The study focuses on different development units in the Engineering team, each working in a distinct business context but sharing standard practices. Results:We verified that even though there are standards for the deployment pipelines, each team’s workflow denotes local variations with unique points for improvement that are highly coupled to their business unit context. We observed that each team’s pipeline has different temporal profiles that reflect their context and work practices. Additionally, we identified a set of deployment pipeline metrics focusing on process compliance, efficiency, and deployment stability. Conclusion:The main contributions of this paper include (1) the description of an actual application of Process Mining to the deployment pipeline of a highly complex e-commerce platform, (2) how this approach provided an objective understanding of the efficiency and quality of the development workflow, (3) how this process-centric view, combined with domain-specific DevOps metrics, supports continuous practices, and (4) how Developers can analyse their workflows by applying Process Mining while using standard tools like GitLab and PM4Py.

Design and Evaluation using Technology Acceptance Model of an Architecture Conceptualization Framework System based on the ISO/IEC/IEEE 42020

Among the difficulties in developing software-intensive systems are the necessity of managing and controlling data that must be held for decades, as well as describing the needs and concerns of a variety of stakeholders. Therefore, one cannot neglect a good Software Engineering practice which is to develop software-intensive systems based on solid software architecture. However, the processes related to the software architecture of software-intensive systems are often considered only from a low level of abstraction. A recent architectural Standard, the ISO/IEC/IEEE 42020, defines 6 clauses for the architecture process, among them the Architecture Conceptual-ization process is the subject of this study. Considering that the ISO/IEC/IEEE 42020 has only recently been published, given the importance of establishing a well-defined software architecture, and considering the difficulties of understanding an architectural Standard, this work proposes a framework, and then the design and further evaluation of a web-based application to support soft-ware architects in using the activities and tasks of the Architecture Conceptualization clause based on the framework described. The ArchConcept was designed to address the high-level abstraction of the Standard ISO/IEC/IEEE 42020 and can be useful for software architects who want to follow ISO/IEC/IEEE 42020’s recommendation and achieve high-quality results in their work of software architecture conceptualization. A qualitative evaluation employing a questionnaire was carried out to obtain information about the perceptions of professionals regarding the ArchConcept, according to the Technology Acceptance Model (TAM). As ArchConcept is focused on activities of Archi-tecture Conceptualization, which is one of the early stages of a software project, the results found could be evidence of the short time dedicated to the initial phases of projects and their consequences.regarding the ArchConcept, according to the Technology Acceptance Model (TAM). As ArchConcept is focused on the early stages of the project (Architecture Conceptualization), the results found in this work could be evidence of the short time dedicated to the initial phase of projects and their consequences.

Assessing the utility of text-to-SQL approaches for satisfying software developer information needs

Software analytics integrated with complex databases can deliver project intelligence into the hands of software engineering (SE) experts for satisfying their information needs. A new and promising machine learning technique known as text-to-SQL automatically extracts information for users of complex databases without the need to fully understand the database structure nor the accompanying query language. Users pose their request as so-called natural language utterance, i.e., question. Our goal was evaluating the performance and applicability of text-to-SQL approaches on data derived from tools typically used in the workflow of software engineers for satisfying their information needs. We carefully selected and discussed five seminal as well as state-of-the-art text-to-SQL approaches and conducted a comparative assessment using the large-scale, cross-domain Spider dataset and the SE domain-specific SEOSS-Queries dataset. Furthermore, we study via a survey how SE professionals perform in satisfying their information needs and how they perceive text-to-SQL approaches. For the best performing approach, we observe a high accuracy of 94% in query prediction when training specifically on SE data. This accuracy is almost independent of the query’s complexity. At the same time, we observe that SE professionals have substantial deficits in satisfying their information needs directly via SQL queries. Furthermore, SE professionals are open for utilizing text-to-SQL approaches in their daily work, considering them less time-consuming and helpful. We conclude that state-of-the-art text-to-SQL approaches are applicable in SE practice for day-to-day information needs.

A KNOWLEDGE MANAGEMENT GUIDELINE: IDENTIFYING AND QUANTIFYING THE KNOWLEDGE LEVEL OF A TEAM THAT WORKS FOR A SOFTWARE ENGINEERING PROJECT

This scientific paper aims to explore the identification and quantification of the knowledge level within a team working on a software engineering project. The knowledge level of a team plays a crucial role in the success and efficiency of project outcomes. To achieve this objective, a comprehensive review of existing literature on knowledge management, team dynamics, and software engineering practices was conducted. This article reveals insights into the knowledge dynamics within software engineering teams. The research methodology consists of different quantitative and qualitative methods for data collection and analysis. The identified knowledge gaps and strengths can inform strategies for improving team performance, such as targeted training programs or knowledge sharing initiatives. Furthermore, the quantification of knowledge levels can serve as a benchmark for future projects, allowing for better resource allocation and team composition. Overall, this research is bringing benefits to the field of software engineering by providing a systematic approach to identify and quantify the knowledge level of teams, enabling organizations to optimize their team structures and enhance project outcomes.

PyPads

Despite algorithmic advancements in the field of machine learning, a need for improvement in the infrastructure supporting machine learning development and research has become increasingly apparent. Machine learning experiments usually tend to be more ad-hoc in nature, and results are communicated most often in the form of a publication. Experimental details are often omitted due to size or time constraints, or simply because the complexity in terms of technical setup or parametrization became intractable. Even access to code bases, disregard important properties of the environment and experimental setup, like for example random generators or computing infrastructure. At the same time, tracking and communicating an often inherently exploratory scientific process is a task with considerable effort. We explored different venues to tackle these issues from a data science engineering point of view. The efforts resulted in PyPads, a framework providing an infrastructure to extend experimental setups with logging, communication and analysis features in a mostly non-intrusive way. PyPads can be extended to different Python-based frameworks, utilizing community driven, descriptive metadata in an effort to harmonize library specific logs in an ontology. Meanwhile, we also try to emphasize similarities to practices in software engineering, which have turned out to be essential in practical applications.

DValidator: An approach for validating dependencies in build configurations

Reusing components is a well-established practice in modern software engineering and brings many advantages, such as a reduction of development costs and time. However, there are still several problems when reusing software components, such as the management of the dependencies of a project. Modern build systems provide dependency managers to support developers when dealing with dependencies. But even with this tool support, dependency management is an error-prone task which can lead to dependency hell if it gets out of control.In this paper, we propose DValidator, an approach that considers dependencies on project level and method call level for validating dependencies in build configurations. First, DValidator encodes a project’s dependency graph as specified in a build configuration and its call graph into a representation using Answer Set Programming (ASP). Then it uses Clingo as a solver to detect problems with the dependencies in that build configuration. In a preliminary evaluation with four open source Maven projects we show that our approach can detect selected dependency smells in less than eight seconds. Next steps concern the investigation of our approach for automatically improving dependency configurations, such as automatically repairing dependency smells and conflicts.Editor’s note: Open Science material was validated by the Journal of Systems and Software Open Science Board.

Software engineering practices for machine learning — Adoption, effects, and team assessment

Machine learning (ML) is extensively used in production-ready applications, calling for mature engineering techniques to ensure robust development, deployment and maintenance. Given the potential negative impact machine learning (ML) can have on people, society or the environment, engineering techniques that can ensure robustness against technical errors and adversarial attacks are of considerable importance. In this work, we investigate how teams of experts develop, deploy and maintain software with ML components. Moreover, we link what teams do to the effects they aim to achieve and provide means for improvement. Towards this goal, we performed a mixed-methods study with a sequential exploratory strategy. First, we performed a systematic literature review through which we mined both academic and grey literature, and compiled a catalogue of engineering practices for ML. Second, we validated this catalogue using a large-scale survey, which measured the degree of adoption of the practices and their perceived effects. Third, we ran validation interviews with practitioners to add depth to the survey results. The catalogue covers a broad range of practices for engineering software systems with ML components and for ensuring non-functional properties that fall under the umbrella of trustworthy ML, such as fairness, security or accountability. Here, we present the results of our study, which indicate, for example, that larger and more experienced teams tend to adopt more practices, but that trustworthiness practices tend to be neglected. Moreover, we show that the effects measured in our survey, such as team agility or accountability, can be predicted quite accurately from groups of practices. This allowed us to contrast the importance of the practices for these effects as well as adoption rates, revealing, for example, that widely adopted practices are, in reality, less important with respect to some effects. For instance, writing reusable scripts for data cleaning and merging is highly adopted, but has a limited impact on reproducibility. Overall, our study provides a quantitative assessment of ML engineering practices and their impact on desirable properties of software with ML components, by which we open multiple avenues for improving the adoption of useful practices.Editor’s note: Open Science material was validated by the Journal of Systems and Software Open Science Board.

Capstone Projects and their Transition into the Software Development Industry: A 10 Year Systematic Review of Literature

This study sought to examine the existing software engineering and Information Technology practices covered in final-year capstone projects in higher institutions of learning around the world, and the possibility of commercializing products developed from these projects. The researchers undertook a Systematic Literature Review methodology. The researchers used Artificial Intelligence (A.I.) tools, including Litmaps and Connected Papers for literature identification, while Quillbot and Scholarly A.I. were used for summary writing. Manual full paper review, data extraction and synthesis, and quantitative analyses were done on the final dataset using Microsoft Word, Excel, and Rayyan.ai. Eighty-eight papers from ACM Library, IEEE, Science Direct, EBSCOHost, Emerald Insight, Taylor & Francis, and Google Scholar were reviewed. Most papers were from Western countries, with only a few from Africa. Findings indicate a remarkable movement from traditional software engineering practices such as Waterfall to modern agile methods such as Scrum and Kaban. There was an increased use of Generative A.I. and Machine learning platforms such as GitHub and Arduino in projects profiled. Common scripting languages used by students were Java and Python. Most projects were undertaken in groups, with no prospective client. Most capstone courses were purely academic endeavours, and only a few were ever commercialized and transformed into industry products. One limitation of the review is that it includes grey literature from conferences and student theses. The study provides insights into State-of-the-Art approaches that can be adopted by higher institutions of learning to improve capstone courses.

Factors influencing sustainability aspects in crowdsourced software development: A systematic literature review

AbstractCrowdsource software development has become more and more popular in recent years in the software industry. Crowdsourcing is an open‐call technique for outsourcing tasks to a broad and undefined crowd. Crowdsourcing provides numerous advantages including reduced costs, fast project completion, talent identification, diversity of solutions, top‐quality, and access to problem‐solving creativity. Despite of the benefits gained from crowdsourcing, there are numerous issues like lack of experienced workers, lack of confidentiality, copyright issues, software sustainability, and so forth. There is also less focus on the long‐term sustainability of software development because of new ideas emerging in crowdsourcing software development. Furthermore, in literature, lack of guidelines towards sustainable software crowdsourcing is highlighted as one of the limitations in the software standards. This study aims to identify the factors that influence sustainability aspects in crowdsourced software development. We have conducted a systematic literature review for identification of these factors. In this paper, we present findings of the systematic literature review in the form of a list of 11 factors extracted from a sample of 45 finally selected papers. Among these factors, six of the factors are ranked as critical factors. These critical factors are “Lack of coding standard in documentation,” “Use of popular programming tools,” “Crowd Lack of knowledge and awareness about sustainability,” “Energy‐efficient coding,” “Lack of awareness about sustainable software engineering practices,” and “Lack of coordination/communication between client and crowd.”

Poison Attack and Poison Detection on Deep Source Code Processing Models

In the software engineering (SE) community, deep learning (DL) has recently been applied to many source code processing tasks, achieving state-of-the-art results. Due to the poor interpretability of DL models, their security vulnerabilities require scrutiny. Recently, researchers have identified an emergent security threat to DL models, namely poison attacks . The attackers aim to inject insidious backdoors into DL models by poisoning the training data with poison samples. The backdoors mean that poisoned models work normally with clean inputs but produce targeted erroneous results with inputs embedded with specific triggers. By using triggers to activate backdoors, attackers can manipulate poisoned models in security-related scenarios ( e.g., defect detection) and lead to severe consequences. To verify the vulnerability of deep source code processing models to poison attacks, we present a poison attack approach for source code named CodePoisoner as a strong imaginary enemy. CodePoisoner can produce compilable and functionality-preserving poison samples and effectively attack deep source code processing models by poisoning the training data with poison samples. To defend against poison attacks, we further propose an effective poison detection approach named CodeDetector . CodeDetector can automatically identify poison samples in the training data. We apply CodePoisoner and CodeDetector to six deep source code processing models, including defect detection, clone detection, and code repair models. The results show that ① CodePoisoner conducts successful poison attacks with a high attack success rate (avg: 98.3%, max: 100%). It validates that existing deep source code processing models have a strong vulnerability to poison attacks. ② CodeDetector effectively defends against multiple poison attack approaches by detecting (max: 100%) poison samples in the training data. We hope this work can help SE researchers and practitioners notice poison attacks and inspire the design of more advanced defense techniques.