API comparison based on the non-functional information mined from Stack Overflow

Abstract

When comparing similar APIs, developers tend to distinguish them from the aspects of functional details. At the same time, some important non-functional factors (such as performance, usability, and security) may be ignored or noticed after using the API in the project. This may result in unnecessary errors or extra costs. API-related questions are common on Stack Overflow, and they can give a well-rounded picture of the APIs. This provides us with a rich resource for API comparison. However, although many methods are offered for mining Questions and Answers (Q&As) automatically, they often suffer from two main problems: 1) they only focus on the functional information of APIs; 2) they analyze each text in isolation but ignore the correlations among them. In this paper, we propose an approach based on the pre-training model BERT to mine the non-functional information of APIs from Stack Overflow: we first tease out the correlations among questions, answers as well as corresponding reviews, so that one Q&A can be analyzed as a whole; then, an information extraction model is constructed by fine-tuning BERT with three subtasks—entity identification, aspect classification, and sentiment analysis separately, and we use it to mine the texts in Q&As step by step; finally, we summarize and visualize the results in a user-friendly way, so that developers can understand the information intuitively at the beginning of API selection. We evaluate our approach on 4,456 Q&As collected from Stack Overflow. The results show our approach can identify the correlations among reviews with 90.1% precision, and such information can improve the performance of the data mining process. In addition, the survey on maturers and novices indicates the understandability and helpfulness of our method. Moreover, compared with language models, our method can provide more intuitive and brief information for API comparison in non-functional aspects.