Software Engineering Publications Research Articles

Research artifacts in software engineering publications: Status and trends

The Software Engineering (SE) community has been embracing the open science policy and encouraging researchers to disclose artifacts in their publications. However, the status and trends of artifact practice and quality remain unclear, lacking insights on further improvement. In this paper, we present an empirical study to characterize the research artifacts in SE publications. Specifically, we manually collect 1,487 artifacts from all 2,196 papers published in top-tier SE conferences (ASE, FSE, ICSE, and ISSTA) from 2017 to 2022. We investigate the common practices (e.g., URL location and format, storage websites), maintenance activities (e.g., last update time and URL validity), popularity (e.g., the number of stars on GitHub and characteristics), and quality (e.g., documentation and code smell) of these artifacts. Based on our analysis, we reveal a rise in publications providing artifacts. The usage of Zenodo for sharing artifacts has significantly increased. However, artifacts stored in GitHub tend to receive few stars, indicating a limited influence on real-world SE applications. We summarize the results and provide suggestions to different stakeholders in conjunction with current guidelines.

Ethics: What is the Brazilian Software Engineering Research Scenario?

Background: Ethics is the theory or science of the moral behavior of humans in society. Traditionally, we value “unethical” actions that go against determining morality in a specific context. One of the sub-domains of Ethics is Computational Ethics, which deals with ethical dilemmas that are strictly related to computational issues. Dilemmas in this area involve privacy, improper access, intellectual property, digital norms and laws, power, socio-technical aspects (such as gender discrimination), and robotics, among others. In this context, “Software Engineering” and “Software” are different objects. Engineering is an act, a practice, as also coding, programming, and software reuse. As with any act, moral and subsequent ethical considerations are appropriate. We characterize software as an object of concrete reality, as a sociotechnical system formed by a technical artifact, human aspect, and procedural aspect. This assumption will form the main base discussion of ethics and morals in Software Engineering in this paper. Objective: The goal of this paper is to unveil the Brazilian Software Engineering ethics panorama. Method: We follow the rigor of, and inspired by, a Systematic Literature Review (SLR) protocol to answer the question: how does ethics explicitly permeate the Brazilian Software Engineering publications between the last thirteen years (2010 and 2022)? Results: After analyzing 1529 papers through the research protocol, 175 (≈11%) presented some explicit occurrence of ethical aspects. The occurrence was relevant in only 7 papers (≈0.4%), exposing a shallow scenario on ethical or moral aspects. Conclusions: If Ethics is a topic considered important to deliberate, research or discuss, this did not occur significantly in the Brazilian Software Engineering research scenario since 2010. With this result in mind, we discussed parallel terms and concepts to enrich the contribution of the qualitative synthesis.

Understanding and improving artifact sharing in software engineering research

In recent years, many software engineering researchers have begun to include artifacts alongside their research papers. Ideally, artifacts, including tools, benchmarks, and data, support the dissemination of ideas, provide evidence for research claims, and serve as a starting point for future research. However, in practice, artifacts suffer from a variety of issues that prevent the realization of their full potential. To help the software engineering community realize the full potential of artifacts, we seek to understand the challenges involved in the creation, sharing, and use of artifacts. To that end, we perform a mixed-methods study including a survey of artifacts in software engineering publications, and an online survey of 153 software engineering researchers. By analyzing the perspectives of artifact creators, users, and reviewers, we identify several high-level challenges that affect the quality of artifacts including mismatched expectations between these groups, and a lack of sufficient reward for both creators and reviewers. Using Diffusion of Innovations as an analytical framework, we examine how these challenges relate to one another, and build an understanding of the factors that affect the sharing and success of artifacts. Finally, we make recommendations to improve the quality of artifacts based on our results and existing best practices.

How software engineering research aligns with design science: a review

BackgroundAssessing and communicating software engineering research can be challenging. Design science is recognized as an appropriate research paradigm for applied research, but is rarely explicitly used as a way to present planned or achieved research contributions in software engineering. Applying the design science lens to software engineering research may improve the assessment and communication of research contributions.AimThe aim of this study is 1) to understand whether the design science lens helps summarize and assess software engineering research contributions, and 2) to characterize different types of design science contributions in the software engineering literature.MethodIn previous research, we developed a visual abstract template, summarizing the core constructs of the design science paradigm. In this study, we use this template in a review of a set of 38 award winning software engineering publications to extract, analyze and characterize their design science contributions.ResultsWe identified five clusters of papers, classifying them according to their different types of design science contributions.ConclusionsThe design science lens helps emphasize the theoretical contribution of research output—in terms of technological rules—and reflect on the practical relevance, novelty and rigor of the rules proposed by the research.

Analyzing Citation Frequencies of Leading Software Engineering Scholars

It is understandable that sponsors of research activities are interested in assessing the work of scholars they (plan to) support although this is not a simple undertaking. Until today, there is no obvious approach for objectively measuring and comparing the quality of research results from different disciplines. Hence, counting publications has been used for long time as a substitute to deal with this challenge and only recent technological advances have fostered the usage of so-called citation indices (such as the h-index) for this purpose. Although this approach is as disputed as all previous ideas in this context, we feel it is about time to investigate the expressiveness of modern citation analysis approaches in computer science more closely. In order to do that, we have chosen the area of software engineering and created a first comprehensive ranking, illustrating citation values of world class scholars by analyzing the work of almost 700 researchers in this field. We have found that top h-index scores in software engineering are around 60 while top-notch g-indices start at around 130 when Google Scholar and Publish or Perish, the quasi standard tools for this purpose are used. Clearly, the results of our study are influenced by the coverage of these tools so that we have also analyzed Google Scholar and found it having a very high coverage of software engineering publications. Hence we are convinced to have collected good quality results that will allow our community to better judge and use citation numbers in the future.

Participation of women in software engineering publications

To the best of our knowledge, no works analyzing the participation of women as authors and editors in software engineering research publications currently exist. We have therefore followed a well-defined procedure in order to conduct an empirical study of female participation in 12 leading software engineering journals. We have analyzed the gender of the authors, editorial board members, associate editors and editors-in-chief over a two-year period in order to analyze, on the one hand, the rate of participation of women as authors and as editors in software engineering publications, and on the other, whether women are underrepresented. We have also analyzed the female distribution of authors and editors according to the geographical location of their institutions. This was done by first selecting the journals to be used as the population for data collection which then allowed us to identify female authors of papers and female editors, including the country in which their institutions are located. This eventually led to an analysis of female participation in order to understand representation rates. We analyzed 3,546 authors of 1,266 papers in 61 different countries, and 363 members of editorial boards in 30 different countries. The results of this analysis provide quantitative data concerning the participation of women as authors and editors in major software engineering journals including their distribution per country, in which important differences have been found. The results obtained were first used to compare the participation of women as authors and editors and were then used to carry out a series of simulations in order to be able to statistically confirm whether women are underrepresented. The study shows, amongst other things, that women are not underrepresented as editorial boards members and as editors-in-chief of the journals studied, although their representation as editors-in-chief is low.