Software Engineering Research Research Articles

Visualizing software refactoring using radar charts

Refactoring tools have advanced greatly and are being used in many large projects. As a result, a great deal of information is now available about past refactoring and its effects on the source code. However, when multiple refactoring is performed at once, it becomes more difficult to analyze their impact. Refactoring visualization can help developers create more maintainable code that is easier to understand and modify over time. Although there is an increasing interest in visualizing code changes in software engineering research, there has been relatively little research on visualizing the process of refactoring. In this paper, we propose a Radar Chart Refactoring Visualization (RcRV) approach to visualize software refactoring of source code across multiple software releases. Radar charts are a form of 2D visualization that can show multiple variables on a single chart. The RcRv receives input from developers or through refactoring identification tools, such as Ref-Finder, to generate charts. The generated charts can show the changes made during the refactoring process, highlighting areas of the trend of refactoring over evolution for multiple refactoring, multiple methods, and multiple classes. The evaluation study conducted to assess the usefulness of the RcRV tool has shown that the proposed tool is useful to developers, appealing, and easy to use. The proposed method of visualization can be beneficial for developers and maintainers to detect design violations and potential bugs in the code, thus saving time and effort during the development and maintenance process. Therefore, this research presents a significant contribution to the software engineering field by providing developers with an efficient tool to enhance code quality and maintainability.

Looking for related posts on GitHub discussions.

Software teams increasingly adopt different tools and communication channels to aid the software collaborative development model and coordinate tasks. Among such resources, software development forums have become widely used by developers. Such environments enable developers to get and share technical information quickly. In line with this trend, GitHub announced GitHub Discussions-a native forum to facilitate collaborative discussions between users and members of communities hosted on the platform. Since GitHub Discussions is a software development forum, it faces challenges similar to those faced by systems used for asynchronous communication, including the problems caused by related posts (duplicated and near-duplicated posts). These related posts can add noise to the platform and compromise project knowledge sharing. Hence, this article addresses the problem of detecting related posts on GitHub Discussions. To achieve this, we propose an approach based on a Sentence-BERT pre-trained general-purpose model: the RD-Detector. We evaluated RD-Detector using data from three communities hosted in GitHub. Our dataset comprises 16,048 discussion posts. Three maintainers and three Software Engineering (SE) researchers manually evaluated the RD-Detector results, achieving 77-100% of precision and 66% of recall. In addition, maintainers pointed out practical applications of the approach, such as providing knowledge to support merging the discussion posts and converting the posts to comments on other related posts. Maintainers can benefit from RD-Detector to address the labor-intensive task of manually detecting related posts.

FFEA software: Repeated early innovation

FFEA software: Repeated early innovation Dr Joanna Leng, Senior Research Software Engineering Fellow at the University of Leeds, discusses software with repeated early innovation using the example of the FFEA software. This article analyses the application of software engineering (SE) tools and practices and research software engineering (RSE) practices to the Fluctuating Finite Element Analysis (FFEA) software, a mesoscale modelling package. In 2016, I met the FFEA team to discuss improving their software. In 2020, I started working with them to develop a roadmap for the FFEA software by assessing their research interests and software.

Verification and validation of software process simulation models: A systematic mapping study

AbstractSoftware process simulation models (SPSMs) that are based on descriptive process models offer the executability that can demonstrate dynamic changes of software processes over time. Verification and validation (V&V) is critical in SPSMs for guaranteeing the quality and reliability of models. V&V of dynamic software process models is more complex and challenging than for static software process models. This work systematically summarizes and maps V&V studies in SPSM to provide guidelines for future research and practice. Specifically, this study aims at identifying the focus of research on V&V, the methods used for V&V, and how to implement V&V of SPSMs in software engineering research. We conducted a systematic mapping study on studies of SPSMs that report on their V&V activities. Under the guidance of a V&V meta‐model for SPSMs, we study four research questions about V&V process. We identified 107 primary studies from a pool of 313 papers on SPSMs until 2021. There are two main results of our study. The first one presents the relationship between quality aspects of SPSMs and the V&V methods to assure them. The second result reveals the relationships among the modeling process, three modeling steps, five quality aspects, and 10 V&V methods. Generally, researchers do not pay sufficient attention to V&V, as 65.8% ( ) failed to mention or elaborate on their V&V process. We systematically summarize and map the state‐of‐the‐art V&V research in software process modeling field to support modelers' practice and improve their V&V process.

Two Birds with One Stone: Boosting Code Generation and Code Search via a Generative Adversarial Network

Automatically transforming developers' natural language descriptions into source code has been a longstanding goal in software engineering research. Two types of approaches have been proposed in the literature to achieve this: code generation, which involves generating a new code snippet, and code search, which involves reusing existing code. However, despite existing efforts, the effectiveness of the state-of-the-art techniques remains limited. To seek for further advancement, our insight is that code generation and code search can help overcome the limitation of each other: the code generator can benefit from feedback on the quality of its generated code, which can be provided by the code searcher, while the code searcher can benefit from the additional training data augmented by the code generator to better understand code semantics. Drawing on this insight, we propose a novel approach that combines code generation and code search techniques using a generative adversarial network (GAN), enabling mutual improvement through the adversarial training. Specifically, we treat code generation and code search as the generator and discriminator in the GAN framework, respectively, and incorporate several customized designs for our tasks. We evaluate our approach in eight different settings, and consistently observe significant performance improvements for both code generation and code search. For instance, when using NatGen, a state-of-the-art code generator, as the generator and GraphCodeBERT, a state-of-the-art code searcher, as the discriminator, we achieve a 32% increase in CodeBLEU score for code generation, and a 12% increase in mean reciprocal rank for code search on a large-scale Python dataset, compared to their original performances.

Fostering Collaboration and Advancing Research in Software Engineering and Game Development for Serious Contexts

The potential benefits of using the engaging and interactive nature of games to achieve specific objectives have been recognized by researchers and professionals from numerous domains. Serious games have been developed to impart knowledge, skills, and awareness in areas such as education, healthcare and the environment, while gamification has been applied to enhance the engagement, motivation, and participation of users in non-game activities such as sustainability and learning. As a result, the fields of game engineering, software engineering, and user experience are increasingly converging to create innovative solutions that blend the strengths of games with real-world applications.

Induktyvinis metodas komponentinių programų sintezėje

The automatic programming and automatic software synthesis systems are relevant software engineering research objects. This article presents the prospects of induction as a component-based software synthesis method. The usage of induction in the component-based software synthesis system can solve main synthesis problems: problem of specification, problem of undefined components and problem of non-functional requirements. However using induction synthesis system can lose soundness provided by structural synhesis of programs method. To avoid this any inductive conclusion should be verified.

Qualitative software engineering research: Reflections and guidelines

AbstractResearchers are increasingly recognizing the importance of human aspects in software development. Because qualitative methods are used to explore human behavior in‐depth, we believe that studies using such methods will become more common. Existing qualitative software engineering guidelines do not cover the full breadth of qualitative methods and the knowledge on how to use them like in social sciences. The purpose of this study was to extend the software engineering community's current body of knowledge regarding available qualitative methods and their quality assurance frameworks and to provide recommendations and guidelines for their use. With the support of an epistemological argument and a survey of the literature, we suggest that future research would benefit from (1) utilizing a broader set of research methods, (2) more strongly emphasizing reflexivity, and (3) employing qualitative guidelines and quality criteria. We present an overview of three qualitative methods commonly used in social sciences but rarely seen in software engineering research, namely interpretative phenomenological analysis, narrative analysis, and discourse analysis. Furthermore, we discuss the meaning of reflexivity in relation to the software engineering context and suggest means of fostering it. Our paper will help software engineering researchers better select and then guide the application of a broader set of qualitative research methods.

Mining Github for Factors That Affect Open Source Software Sustainability

Open-Source Software (OSS) is everywhere. The availability of such software enables researchers in software engineering to have a deep insight into the factors that affect the success of the software. Some OSS repos get more popular and evolve over time, while others may only survive for a couple of months. This study aims to help developers identify internal factors that affect the sustainability of their software. Firstly, identify the most demanding application domain; secondly, observe the most popular repositories (for the demanded application domain) for about three years to identify the factors that help such repos to survive (sustainability factors). Dara mining algorithms (classification and regression) are used as a tool to find the GitHub factors that affect the sustainability of OSS.

Development of SkinTracker, an integrated dermatology mobile app and web portal enabling remote clinical research studies.

In-person dermatology clinical research studies often face recruitment and participation challenges due to travel-, time-, and cost-associated barriers. Studies incorporating virtual/asynchronous formats can potentially enhance research subject participation and satisfaction, but few mobile health tools are available to enable remote study conduct. We developed SkinTracker, a patient-facing mobile app and researcher-facing web platform, that enables longitudinal collection of skin photos, patient reported outcomes, and biometric health and environmental data. Eight design thinking sessions including dermatologists, clinical research staff, software engineers, and graphic designers were held to create the components of SkinTracker. Following iterative prototyping, SkinTracker was piloted across six adult and four pediatric subjects with atopic dermatitis (AD) of varying severity levels to test and provide feedback on SkinTracker for six months. The SkinTracker app enables collection of informed consent for study participation, baseline medical history, standardized skin photographs, patient-reported outcomes (e.g., Patient Oriented Eczema Measure (POEM), Pruritus Numerical Rating Scale (NRS), Dermatology Life Quality Index (DLQI)), medication use, adverse events, voice diary to document qualitative experiences, chat function for communication with research team, environmental and biometric data such as exercise and sleep metrics through integration with an Apple Watch. The researcher web portal allows for management and visualization of subject enrollment, skin photographs for examination and severity scoring, survey completion, and other patient modules. The pilot study requested that subjects complete surveys and photographs on a weekly to monthly basis via the SkinTracker app. Afterwards, participants rated their experience in a 7-item user experience survey covering app function, design, and desire for participation in future studies using SkinTracker. Almost all subjects agreed or strongly agreed that SkinTracker enabled more convenient participation in skin research studies compared to an in-person format. To our knowledge, SkinTracker is one of the first integrated app- and web-based platforms allowing collection and management of data commonly obtained in clinical research studies. SkinTracker enables detailed, frequent capture of data that may better reflect the fluctuating course of conditions such as AD, and can be modularly customized for different skin conditions to improve dermatologic research participation and patient access.

Case study identification: A trivial indicator outperforms human classifiers

The definition and term “case study” are not being applied consistently by software engineering researchers. We previously developed a trivial “smell indicator” to help detect the misclassification of primary studies as case studies. To evaluate the performance of the indicator. We compare the performance of the indicator against human classifiers for three datasets, two datasets comprising classifications by both authors of systematic literature studies and primary studies, and one dataset comprising only primary-study author classifications. The indicator outperforms the human classifiers for all datasets. The indicator is successful because human classifiers “fail” to properly classify their own, and others’, primary studies. Consequently, reviewers of primary studies and authors of systematic literature studies could use the classifier as a “sanity” check for primary studies. Moreover, authors might use the indicator to double-check how they classified a study, as part of their analysis, and prior to submitting their manuscript for publication. We challenge the research community to both beat the indicator, and to improve its ability to identify true case studies.

Function Call Graph Context Encoding for Neural Source Code Summarization

Source code summarization is the task of writing natural language descriptions of source code. The primary use of these descriptions is in documentation for programmers. Automatic generation of these descriptions is a high value research target due to the time cost to programmers of writing these descriptions themselves. In recent years, a confluence of software engineering and artificial intelligence research has made inroads into automatic source code summarization through applications of neural models of that source code. However, an Achilles' heel to a vast majority of approaches is that they tend to rely solely on the context provided by the source code being summarized. But empirical studies in program comprehension are quite clear that the information needed to describe code much more often resides in the context in the form of Function Call Graph surrounding that code. In this paper, we present a technique for encoding this call graph context for neural models of code summarization. We implement our approach as a supplement to existing approaches, and show statistically significant improvement over existing approaches. In a human study with 20 programmers, we show that programmers perceive generated summaries to generally be as accurate, readable, and concise as human-written summaries.

Embedding Software Engineering in Mixed Methods Research: Computationally Enhanced Risk Communication

Mixed methods research ameliorates many convergent research challenges within the contemporary sociotechnical landscape. We suggest the integration of software engineering in mixed methods research is a critical step to address some of the remaining and persistent challenges. One such research challenge where computational science is particularly well suited is in hazard preparedness—in particular, the creation of risk communication messages to mitigate or to prevent harm. Computationally enhanced risk communication is convergent research that integrates software engineering and social science research for the benefit of protecting humans and infrastructure. To this end, we developed a mixed methods framework for the efficient construction of risk communication messages. We call this the Domain Agnostic Risk Communication (DARC) framework and present it here. The DARC framework formalizes connections between computational science and social science methods. It incorporates the best available science in risk communication research and a cadre of natural language processing techniques to impart validity, reliability, and precision into resultant messages. The DARC framework is highly modular owing to the incorporation of the software engineering principles of abstraction, extensibility, and encapsulation. Although the focus of this position article is on risk communication, we encourage the incorporation of software engineering into mixed methods research and the incorporation of mixed methods more broadly into software engineering experimentation.

Combining Phenomenology and Grounded Theory in Software Engineering: An Experience

Phenomenology and grounded theory are two prominent qualitative methods, particularly used in social sciences research. Phenomenology is carried out to understand the individuals’ actual experience regarding a phenomenon. The method describes "what" individuals experience and "how" they experience it. The focus is on the meaning of the exposures experienced by individuals regarding the phenomenon. Grounded theory on the other hand allows researchers to explore a phenomenon in depth with individuals, by which a theory is then generated. The goal is to go beyond the understanding of phenomenon by producing a theory that describes comprehensively the problem being studied. Although these two methods are initiated by similar motivations, namely to understand a phenomenon, they however employ slightly different approaches during execution. The differences make them fit complementary together to produce a more concrete and holistic outcome. To research that occasionally use qualitative methods such as Software Engineering, these methods bring new and multifaceted experience. Software Engineering research opts for qualitative methods to promote understanding, as many phenomena in the field have yet to be understood by its community. In that respect, grounded theory is becoming quite a norm in Software Engineering research in recent years, phenomenology however is relatively sporadic. This paper shares the experience of employing as well as combining phenomenology and grounded theory in Software Engineering research. The sharing is intended to inspire future research in twofold: more technical fields such as Software Engineering to employ qualitative methods in research; and leveraging the benefits of combining two qualitative methods complementarily in one study.

A Manifesto for Rewarding and Recognizing Team Infrastructure Roles

The Scientific Reform Movement has highlighted the need for large research teams with diverse skills. This has necessitated the growth of professional team infrastructure roles (TIRs) who support research through specialised skills, but do not have primary responsibility for conceiving or leading research projects. TIRs such as Lab Technicians, Project Managers, Data Stewards, Community Managers, and Research Software Engineers all play an important role in ensuring the success of a research project, but are commonly neglected under current reward and recognition procedures, which focus on the individual academic researcher instead of the teams involved. Without meaningful identification and recognition of TIR contributions, we risk reinforcing the conceptual and practical division between academic researchers and TIRs. This situation is inequitable and detrimental to the research enterprise: the limited potential for career advancement for TIRs may cause them to leave for other occupations, ultimately leading to a loss of institutional skill, expertise, and memory. This contribution explores the evolution of specialist TIRs and the status of these positions in various settings. We provide three case study descriptions of TIR activities, so that readers may become more familiar with the breadth and depth of their work. We then propose system level changes designed to embed meaningful recognition of all contributions. Acknowledging the contributions of all research roles will help retain skill and expertise, and lead to collaborative research ecosystems that are well-positioned to address complex research challenges.

Better research software tools to elevate the rate of scientific discovery or why we need to invest in research software engineering.

Better research software tools to elevate the rate of scientific discovery or why we need to invest in research software engineering.

What’s (Not) Working in Programmer User Studies?

A key goal of software engineering research is to improve the environments, tools, languages, and techniques programmers use to efficiently create quality software. Successfully designing these tools and demonstrating their effectiveness involves engaging with tool users—software engineers. Researchers often want to conduct user studies of software engineers to collect direct evidence. However, running user studies can be difficult, and researchers may lack solution strategies to overcome the barriers, so they may avoid user studies. To understand the challenges researchers face when conducting programmer user studies, we interviewed 26 researchers. Based on the analysis of interview data, we contribute (i) a taxonomy of 18 barriers researchers encounter; (ii) 23 solution strategies some researchers use to address 8 of the 18 barriers in their own studies; and (iii) 4 design ideas, which we adapted from the behavioral science community, that may lower 8 additional barriers. To validate the design ideas, we held an in-person all-day focus group with 16 researchers.

The development of research software engineering as a profession

The development of research software engineering as a profession Dr Joanna Leng at the School of Computing explores Research Software Engineering (RSE) as an emerging profession, and how computing technology is core to many professions. Research Software Engineering is a young and emerging profession. This article looks at how it compares to other more mature professions and how all these professions handle two factors that are particularly important to the RSE profession, the stability of computing technologies and the adoption of these computing technologies by each profession.

Understanding the Role of Stack Overflow in Supporting Software Development Tasks: A Research Perspective

Stack Overflow is a Q&A website that is popular among developers and extensively used in software engineering (SE) research. A significant body of research has examined how Stack Overflow can assist with software development tasks, such as recommending APIs. However, while researchers have recognized the importance of Stack Overflow in SE research related to software development tasks, the specific ways in which it is utilized and the reasons for its widespread usage in research have not been thoroughly explored. To address these knowledge gaps, we conducted the first study to understand the role of Stack Overflow in assisting with SE research regarding software development tasks by systematically examining relevant and high-quality research works. Meanwhile, we carried out a qualitative survey to gain insight into why researchers choose to utilize Stack Overflow in SE research and to solicit suggestions for the better use of Stack Overflow in research. The study identifies trends in the research area, prominent researchers and organizations, and the types of tasks that utilize Stack Overflow in research, with coding and debugging being the most common. Moreover, it examines how Stack Overflow data is utilized in SE research regarding software development tasks, including searching, training models, and mining associations. Our qualitative survey of researchers indicates that the popularity of Stack Overflow stems from its comprehensive explanations of technical topics that are often not found in documentation or manuals. The findings provide a comprehensive understanding of the role of Stack Overflow in SE research regarding software development tasks, and offer actionable implications for both researchers and stakeholders of Stack Overflow to facilitate future research and improvements.

“We do not appreciate being experimented on”: Developer and researcher views on the ethics of experiments on open-source projects

A tenet of open source software development is to accept contributions from users–developers (typically after appropriate vetting). But should this also include interventions done as part of research on open source development? Following an incident in which buggy code was submitted to the Linux kernel to see whether it would be caught, we conduct a survey among open source developers and empirical software engineering researchers to see what behaviors they think are acceptable. This covers two main issues: the use of publicly accessible information, and conducting active experimentation. The survey had 224 respondents. The results indicate that open-source developers are largely open to research, provided it is done transparently. In other words, many would agree to experiments on open-source projects if the subjects were notified and provided informed consent, and in special cases also if only the project leaders agree. While researchers generally hold similar opinions, they sometimes fail to appreciate certain nuances that are important to developers. Examples include observing license restrictions on publishing open-source code and safeguarding the code. Conversely, researchers seem to be more concerned than developers about privacy issues. Based on these results, it is recommended that open source repositories and projects address use for research in their access guidelines, and that researchers take care to ask permission also when not formally required to do so. We note too that the open source community wants to be heard, so professional societies and IRBs should consult with them when formulating ethics codes.