GitHub Projects Research Articles

PI-Link: A Ground-Truth Dataset of Links Between Pull-Requests and Issues in GitHub

GitHub hosts Git repositories and provides issues-tracking services to provide a better collaboration environment for software developers. Issues and Pull-Requests are frequently used in GitHub to discuss and review the software requirements (new features, bugs, etc.) and software solutions (source code, test cases, etc.) respectively. The links between Issues and their corresponding Pull-Requests comprise valuable information to keep tracking current development as well as documenting knowledge for future development. Considering a large number of links, such information can be used to train machine learning models for several purposes such as feature location, bug prediction and localization, recommendation systems and documentation generation. To the best of our knowledge, no dataset has been proposed as a ground-truth of links between Issues and Pull-Requests. In this paper, we propose, PI-Link, a new significant and reliable ground-truth dataset composed of 50369 links that explicitly connect 34732 Issues with 50369 Pull-Requests. These links are automatically extracted from all (907,139) Android projects in GitHub created between January 1, 2011 and January 1, 2021. To better organize and store the collected data, we propose a metamodel based on the concepts of Issues and Pull Requests. Moreover, we analyze the relationships between Issues and their linked Pull Requests based on four features related to their titles, bodies, labels and comments. The selected features are analyzed in terms of their lengths and similarities based on three lexical and one semantic similarity metrics. The results showed promising similarities between Issues and their linked PRs at the lexical and semantic levels. In addition, some feature similarities are sensitive to the text length, whereas other feature similarities are sensitive to the term frequency.

Predicting Integer Overflow Errors via Supervised Learning

An integer overflow error occurs when an integer operation in computer software evaluates a value out of the integer range. It can lead to a fatal system failure. The existing approaches to detecting integer overflow errors rely on data/control-flow analysis of the code or execution of the code with test cases. This paper presents a supervised learning approach to predicting whether each method in a given Java program has an integer overflow error by treating the source code as text. Built upon real-world programs, our Java dataset covers all integer data types and operations in Java, the methods for preventing integer overflow errors, and adversarial samples. We have evaluated six classification models, BERT, DistilBERT, codeBERT, Code2Vec, fastText, and NBSVM. They represent different text embedding techniques for dealing with source code. The experiment results show that BERT and its variants have outperformed other models. We have applied the resultant BERT model to eleven real-world projects, including JDK13.0 and ten top-ranked GitHub projects, and revealed 181 integer overflow errors. In addition, we have evaluated the classification models with a C/C++ dataset. The result is similar to that of the Java dataset.

Opportunities and Challenges in Repeated Revisions to Pull-Requests: An Empirical Study

Background: The Pull-Request (PR) model is a widespread approach adopted by open source software (OSS) projects to support collaborative software development. However, it is often challenging to continuously evaluate and revise PRs in several iterations of code reviewsinvolving technical and social aspects. Aim: Our objective is twofold: identifying best practices for effective collaboration in continuous PR improvement and uncovering problems that deserve special attention to improve collaboration efficiency and productivity. Method: We conducted a mixed-methods empirical study of repeatedly revised PRs (i.e. those that have undergone a high number of revisions). Historical trace data of five long-lived popular GitHub projects were used for manual investigation of practices for requesting changes to PRs and reasons for nonacceptance of repeatedly revised PRs. Surveys of OSS practitioners were conducted to evaluate the results of manual analysis and to provide additional insights into developers' willingness regarding PR revisions and factors causing avoidable revisions in practice. Results: The main results of our research were as follows: (1) We identified 15 code review practices for requesting changes to PRs, among which practices with respect to explaining the reasoning behind requested changes and tracking the progress of PR review and revision were undervalued by reviewers; (2) While submitters can in general undergo 1-5 rounds of revisions, they are willing to offer more revisions when they are in a friendly community and receive helpful feedback; (3) We revealed 11 factors causing avoidable revisions regarding to reviewers' feedback, code review policy, pre-submission issues, and implementation of new revisions; and (4) Nonacceptance of repeatedly revised PRs was due mainly to inactivity of submitters or reviewers and being superseded for better maintenance. Finally, based on these findings, we proposed recommendations and implications for OSS practitioners and tool designers to facilitate efficient collaboration in PR revisions.

Open-Source Software in the Classroom: Empowering Students to Self-Select Projects to Contribute

Contribution: Instructors are leveraging open-source software (OSS) as a way to experience authentic examples of software problems with their students. Recommender engines might assist students in selecting the right project based on metrics mined from project repositories (e.g., GitHub). This vision is realized through GitMate, a GitHub-based recommender for supporting students in their OSS selection. Background: Contributing to OSS is a valuable way to immerse students into the realities of software development. When it comes to OSS selection, self-selection seems to be the most engaging alternative. Yet, students lack the time (and skills) to analyze project facets and draw comparisons among OSS projects. Research Questions: How can students be assisted to select a good OSS project to contribute to? Specifically, how would a recommender system might help? The envisioned intervention should be useful not only in finding the right project but also challenging students’ initial selections with other alternatives, spurring reflection. Methodology: The aim is to act upon a dependent variable (mind changing in project selection) through an independent variable (project comparison). This is achieved through GitMate, a recommender system on top of GitHub. Its search facilities are used for students to locate three projects at their wish. Next, GitMate recommends similar projects based on the project facets (e.g., number of committers, commits, and stars), mined from GitHub. Pondering the importance of distinct facets, students can now tradeoff different projects. The experiment checks whether students change their first choice. Findings: The results indicate that GitMate helps students compare GitHub projects to the extent of making them change their first choice. Nearly, 80% of the students changed at least one project as a result of using GitMate. This seems to suggest GitMate being effective on its goal: facet-based comparison thinking during OSS selection.

Natural Language Processing Application on Commit Messages: A Case Study on HEP Software

Version Control and Source Code Management Systems, such as GitHub, contain a large amount of unstructured historical information of software projects. Recent studies have introduced Natural Language Processing (NLP) to help software engineers retrieve information from a very large collection of unstructured data. In this study, we have extended our previous study by increasing our datasets and machine learning and clustering techniques. We have followed a complex methodology made up of various steps. Starting from the raw commit messages we have employed NLP techniques to build a structured database. We have extracted their main features and used them as input of different clustering algorithms. Once each entry was labelled, we applied supervised machine learning techniques to build a prediction and classification model. We have developed a machine learning-based model to automatically classify commit messages of a software project. Our model exploits a ground-truth dataset that includes commit messages obtained from various GitHub projects belonging to the High Energy Physics context. The contribution of this paper is two-fold: it proposes a ground-truth database and it provides a machine learning prediction model that automatically identifies the more change-prone areas of code. Our model has obtained a very high average accuracy (0.9590), precision (0.9448), recall (0.9382), and F1-score (0.9360).

SpotitPy: a semi-automated tool for object-based co-localization of fluorescent labels in microscopy images

BackgroundIn fluorescence microscopy, co-localization refers to the spatial overlap between different fluorescent labels in cells. The degree of overlap between two or more channels in a microscope may reveal a physical interaction or topological functional interconnection between molecules. Recent advances in the imaging field require the development of specialized computational analysis software for the unbiased assessment of fluorescently labelled microscopy images.ResultsHere we present SpotitPy, a semi-automated image analysis tool for 2D object-based co-localization. SpotitPy allows the user to select fluorescent labels and perform a semi-automated and robust segmentation of the region of interest in distinct cell types. The workflow integrates advanced pre-processing manipulations for de-noising and in-depth semi-automated quantification of the co-localized fluorescent labels in two different channels. We validated SpotitPy by quantitatively assessing the presence of cytoplasmic ribonucleoprotein granules, e.g. processing (P) bodies, under conditions that challenge mRNA translation, thus highlighting SpotitPy benefits for semi-automatic, accurate analysis of large image datasets in eukaryotic cells. SpotitPy comes in a command line interface or a simple graphical user interphase and can be used as a standalone application.ConclusionsOverall, we present a novel and user-friendly tool that performs a semi-automated image analysis for 2D object-based co-localization. SpotitPy can provide reproducible and robust quantifications for large datasets within a limited timeframe. The software is open-source and can be found in the GitHub project repository: (https://github.com/alexiaales/SpotitPy).

A method for identifying references between projects in GitHub

• Extract links on issues, pull requests, and commits using patterns URL, Num and SHA. • Identify the change of project names and delete references between the same project. • In comparison with previous works, our method IREL identifies many more references. In open source software platforms, software projects do not usually develop in isolation, and they depend on each other and develop together. It is important to identify references between projects in software development activities, which may help projects identify cross-project bugs or attract new contributors from related projects. In this paper, we propose a method IREL to I dentify R eferences between projects by E xtracting L inks. We first extract links from descriptions and comments on issues, pull requests, and commits with three matching patterns. Then we identify changes in project names and replace the original project names with their new project names. Finally, we identify references between projects by selecting links with different source projects and target projects. We evaluate the performance based on datasets with 20,347,228 projects. Our method IREL obtains 934,322 references, 26.461 times as many as the method Reference Coupling and 16.483 times as many as the method Issue Units. Project PageRank scores based on references identified by our method IREL are more correlated with the number of stars of projects. Our method supports researchers to identify references better.

A Deep Dive into the Impact of COVID-19 on Software Development

Context: COVID-19 pandemic has impacted different business sectors around the world. Objective. This study investigates the impact of COVID-19 on software projects and software development professionals. Method: We conducted a mining software repository study based on 100 GitHub projects developed in Java using ten different metrics. Next, we surveyed 279 software development professionals for better understanding the impact of COVID-19 on daily activities and wellbeing. Results: We identified 12 observations related to productivity, code quality, and wellbeing. Conclusions: Our findings highlight that the impact of COVID-19 is not binary (reduce productivity vs. increase productivity) but rather a spectrum. For many of our observations, substantial proportions of respondents have differing opinions from each other. We believe that more research is needed to uncover specific conditions that cause certain outcomes to be more prevalent.

Using textual bug reports to predict the fault category of software bugs

Debugging is a time-consuming and expensive process. Developers have to select appropriate tools, methods and approaches in order to efficiently reproduce, localize and fix bugs. These choices are based on the developers’ assessment of the type of fault for a given bug report. This paper proposes a machine learning (ML) based approach that predicts the fault type for a given textual bug report. We built a dataset from 70+ projects for training and evaluation of our approach. Further, we performed a user study to establish a baseline for non-expert human performance on this task. Our models, incorporating our custom preprocessing approaches, reach up to 0.69% macro average F1 score on this bug classification problem. We demonstrate inter-project transferability of our approach. Further, we identify and discuss issues and limitations of ML classification approaches applied on textual bug reports. Our models can support researchers in data collection efforts, as for example bug benchmark creation. In future, such models could aid inexperienced developers in debugging tool selection, helping save time and resources. • Natural language processing approach predicting the fault type from bug reports. • A data set of 762 labeled bug reports from 75 GitHub Projects. • A user survey to establish a performance baseline for fault type classification. • Machine learning and ensemble methods classifying bug reports.

Styler: learning formatting conventions to repair Checkstyle violations

Ensuring the consistent usage of formatting conventions is an important aspect of modern software quality assurance. To do so, the source code of a project should be checked against the formatting conventions (or rules) adopted by its development team, and then the detected violations should be repaired if any. While the former task can be automatically done by format checkers implemented in linters, there is no satisfactory solution for the latter. Manually fixing formatting convention violations is a waste of developer time and code formatters do not take into account the conventions adopted and configured by developers for the used linter. In this paper, we present Styler, a tool dedicated to fixing formatting rule violations raised by format checkers using a machine learning approach. For a given project, Styler first generates training data by injecting violations of the project-specific rules in violation-free source code files. Then, it learns fixes by feeding long short-term memory neural networks with the training data encoded into token sequences. Finally, it predicts fixes for real formatting violations with the trained models. Currently, Styler supports a single checker, Checkstyle, which is a highly configurable and popular format checker for Java. In an empirical evaluation, Styler repaired 41% of 26,791 Checkstyle violations mined from 104 GitHub projects. Moreover, we compared Styler with the IntelliJ plugin CheckStyle-IDEA and the machine-learning-based code formatters Naturalize and CodeBuff. We found out that Styler fixes violations of a diverse set of Checkstyle rules (24/25 rules), generates smaller repairs in comparison to the other systems, and predicts repairs in seconds once trained on a project. Through a manual analysis, we identified cases in which Styler does not succeed to generate correct repairs, which can guide further improvements in Styler. Finally, the results suggest that Styler can be useful to help developers repair Checkstyle formatting violations.

Release conventions of open‐source software: An exploratory study

AbstractSoftware engineering (SE) methodologies are widely used in both academia and industry to manage the software development life cycle. A number of studies of SE methodologies involve interviewing stakeholders to explore the real‐world practice. Although these interview‐based studies provide us with a user's perspective of an organization's practice, they do not describe the concrete summary of releases in open‐source social coding platforms. In particular, no existing studies investigated how releases are evolved in open‐source coding platforms, which assist release planners to a large extent. This study explores software development patterns followed in open‐source projects to see the overall management's reflection on software release decisions rather than concentrating on a particular methodology. Our experiments on 51 software origins (with 1777k revisions and 12k releases) from the Software Heritage Graph Dataset (SWHGD) and their GitHub project boards (with 23k cards) reveal reasonably active project management with phase simplicity can release software versions more frequently and can follow the small release conventions of Extreme Programming. Additionally, the study also reveals that a combination of development and management activities can be applied to predict the possible number of software releases in a month ( ).

Research Management in "Many Analysts" Projects

With the recent increase of large scientific collaborations in the form of "Many Analysts" projects, research project management has quickly moved from small/medium-scale coordination to large-scale planning. Project coordinators are faced with unprecedented challenges which make "Many Analysts" projects time consuming and resource demanding. With this poster, we would like to share our experience managing the ongoing Many Speech Analysis project (https://many-speech-analyses.github.io) and offer tips and logistic solutions to minimise the resource overhead required for such type of projects. We will also discuss which steps in the process didn't go as expected and how on hindsight we wished we approached them.
 
 More specifically we will discuss the following aspects:
 
 
 Overall design of the study in five phases: recruitment, teams' analyses, analyses peer-review, meta-analysis and write-up.
 Preparation of a Registered Report and process of getting an In Principle Acceptance prior to the beginning of the first phase (recruitment).
 Preparation and management of a project-wide OSF repository using the R package osfr, and integration with GitHub.
 Logistics of recruitment and team creation with social media, Eventbrite and Google Sheets.
 Development of questionnaires to gather background information on the recruited analysts and to collect reports on the the teams' analyses.
 General project management with GitHub projects, issues and GitHub actions.

Mutation testing in the wild: findings from GitHub

Mutation testing exploits artificial faults to measure the adequacy of test suites and guide their improvement. It has become an extremely popular testing technique as evidenced by the vast literature, numerous tools, and research events on the topic. Previous survey papers have successfully compiled the state of research, its evolution, problems, and challenges. However, the use of mutation testing in practice is still largely unexplored. In this paper, we report the results of a thorough study on the use of mutation testing in GitHub projects. Specifically, we first performed a search for mutation testing tools, 127 in total, and we automatically searched the GitHub repositories including evidence of their use. Then, we focused on the top ten most widely used tools, based on the previous results, and manually revised and classified over 3.5K GitHub active repositories importing them. Among other findings, we observed a recent upturn in interest and activity, with Infection (PHP), PIT (Java) and Humbug (PHP) being the most widely used mutation tools in recent years. The predominant use of mutation testing is development, followed by teaching and learning, and research projects, although with significant differences among mutation tools found in the literature—less adopted and largely used in teaching and research—and those found in GitHub only—more popular and more widely used in development. Our work provides a new and encouraging perspective on the state of practice of mutation testing.

Predicting health indicators for open source projects (using hyperparameter optimization)

Software developed on public platform is a source of data that can be used to make predictions about those projects. While the individual developing activity may be random and hard to predict, the developing behavior on project level can be predicted with good accuracy when large groups of developers work together on software projects. To demonstrate this, we use 64,181 months of data from 1,159 GitHub projects to make various predictions about the recent status of those projects (as of April 2020). We find that traditional estimation algorithms make many mistakes. Algorithms like k-nearest neighbors (KNN), support vector regression (SVR), random forest (RFT), linear regression (LNR), and regression trees (CART) have high error rates. But that error rate can be greatly reduced using hyperparameter optimization. To the best of our knowledge, this is the largest study yet conducted, using recent data for predicting multiple health indicators of open-source projects. To facilitate open science (and replications and extensions of this work), all our materials are available online at https://github.com/arennax/Health_Indicator_Prediction .

EXNVerify: coverage analysis for long and short-read sequencing data in clinical context.

Accurate identification of genetic variants to a large extent is based on the type of experimental technology, quality of the material and coverage of sequencing data obtained. The latter, coverage quality, highly influences variant calling accuracy and final diagnosis. Our motivation was to create a tool that will evaluate genome coverage and accelerate the introduction of long-read sequencing to medical diagnostics and clinical practice. The implementation was guided by the ease of use of the tool by users who are not proficient in using complex software. A Docker container is perfect for this purpose. Using Docker's advantages (flexibility, mobility and ease of use of the proposed tools), we created eXNVerify. This is a tool for inspection of clinical data in the context of pathogenic variants search. The tool calculates clinical depth coverage (CDC) - a measure of coverage which we introduce to evaluate loci with pathogenic germline and somatic variants reported in ClinVar. The tool additionally provides visualization options for user-defined genes of interest. Finally, we present examples of BRCA1, TP53, CFTR application and results of a test conducted in the Extensive Sequence Dataset of Gold-Standard Samples for Benchmarking and Development. eXNVerify improves the diagnostic process of patients related to important genetic diseases and facilitates the assessment of genetic samples by diagnosticians. The use of Docker allows to run an analysis package and does not require any special technical preparation. Detailed examples are included in the GitHub project documentation and the package can be downloaded directly from DockerHub using the command: docker pull porebskis/exnverify:1.0.

On the impact of security vulnerabilities in the npm and RubyGems dependency networks

The increasing interest in open source software has led to the emergence of large language-specific package distributions of reusable software libraries, such as npm and RubyGems. These software packages can be subject to vulnerabilities that may expose dependent packages through explicitly declared dependencies. Using Snyk’s vulnerability database, this article empirically studies vulnerabilities affecting npm and RubyGems packages. We analyse how and when these vulnerabilities are disclosed and fixed, and how their prevalence changes over time. We also analyse how vulnerable packages expose their direct and indirect dependents to vulnerabilities. We distinguish between two types of dependents: packages distributed via the package manager, and external GitHub projects depending on npm packages. We observe that the number of vulnerabilities in npm is increasing and being disclosed faster than vulnerabilities in RubyGems. For both package distributions, the time required to disclose vulnerabilities is increasing over time. Vulnerabilities in npm packages affect a median of 30 package releases, while this is 59 releases in RubyGems packages. A large proportion of external GitHub projects is exposed to vulnerabilities coming from direct or indirect dependencies. 33% and 40% of dependency vulnerabilities to which projects and packages are exposed, respectively, have their fixes in more recent releases within the same major release range of the used dependency. Our findings reveal that more effort is needed to better secure open source package distributions.

Automated evaluation of comments to aid software maintenance

AbstractApproaches to evaluate comments based on whether they increase code comprehensibility for software maintenance tasks are important, but largely missing. We proposeCommentfor automated classification and quality evaluation of code comments of C codebases based on how they can help to understand existing code. We conduct surveys and document developers' perceptions on the type of comments that prove useful to maintaining software in the form of comment categories. A total of 20,206 comments have been collected from open‐sourceGithubprojects and annotated with assistance from industry experts. We develop features to semantically analyze comments to locate concepts related to categories of usefulness. Additionally, features based on code and comment correlation are designed to infer whether the comment is also consistent and not superfluous. Using neural networks, comments are classified asuseful,partially useful, andnot usefulwith precision and recall scores of 86.27% and 86.42%, respectively. The proposed framework for comment quality evaluation incorporates industry practices and adds significant value to companies wanting to formulate better code commenting strategies. Furthermore, large codebases can be de‐cluttered by removing comments not helpful in maintaining code.

The penumbra of open source: projects outside of centralized platforms are longer maintained, more academic and more collaborative

GitHub has become the central online platform for much of open source, hosting most open source code repositories. With this popularity, the public digital traces of GitHub are now a valuable means to study teamwork and collaboration. In many ways, however, GitHub is a convenience sample, and may not be representative of open source development off the platform. Here we develop a novel, extensive sample of public open source project repositories outside of centralized platforms. We characterized these projects along a number of dimensions, and compare to a time-matched sample of corresponding GitHub projects. Our sample projects tend to have more collaborators, are maintained for longer periods, and tend to be more focused on academic and scientific problems.

An Empirical Analysis of Inferences From Commit, Fork, and Branch Rates of Top GitHub Projects

GitHub repositories have been used to understand some important parameters related to quantitative growth of software projects. The rate at which fork, commits and branches have been done in different repositories which are supposed to be the topmost ones when it comes to the number of contributors is the main point of study in this paper. The desirable goal is to find out some sort of similarity in the trends of all these software projects. Similar trends tend to lead to interesting inferences regarding the quantitative growth and even the quality of these software projects. Is the rate of forking, branching, committing anything to do with the success of software projects? The authors have taken up this question in the current study.

DeepReGraph co-clusters temporal gene expression and cis-regulatory elements through heterogeneous graph representation learning

This work presents DeepReGraph, a novel method for co-clustering genes and cis-regulatory elements (CREs) into candidate regulatory networks. Gene expression data, as well as data from three CRE activity markers from a publicly available dataset of mouse fetal heart tissue, were used for DeepReGraph concept proofing. In this study we used open chromatin accessibility from ATAC-seq experiments, as well as H3K27ac and H3K27me3 histone marks as CREs activity markers. However, this method can be executed with other sets of markers. We modelled all data sources as a heterogeneous graph and adapted a state-of-the-art representation learning algorithm to produce a low-dimensional and easy-to-cluster embedding of genes and CREs. Deep graph auto-encoders and an adaptive-sparsity generative model are the algorithmic core of DeepReGraph. The main contribution of our work is the design of proper combination rules for the heterogeneous gene expression and CRE activity data and the computational encoding of well-known gene expression regulatory mechanisms into a suitable objective function for graph embedding. We showed that the co-clusters of genes and CREs in the final embedding shed light on developmental regulatory mechanisms in mouse fetal-heart tissue. Such clustering could not be achieved by using only gene expression data. Function enrichment analysis proves that the genes in the co-clusters are involved in distinct biological processes. The enriched transcription factor binding sites in CREs prioritize the candidate transcript factors which drive the temporal changes in gene expression. Consequently, we conclude that DeepReGraph could foster hypothesis-driven tissue development research from high-throughput expression and epigenomic data. Full source code and data are available on the DeepReGraph GitHub project.