Stack Overflow Questions Research Articles

On the Usage of New JavaScript Features Through Transpilers: The Babel Case

JavaScript applications have long been dealing with browser compatibility issues. Due to the variety of browsers, operational systems, and versions, not all new JavaScript features are available to users. Fortunately transpilers transform the new code syntax into an older version, making the code compatible with a large number of browsers. This study, which focuses on a largely used transpiler (Babel), aims to shed light on how new JavaScript features are used and what major challenges developers face when using them. We assess the top-1K JavaScript projects from GitHub, 108 Stack Overflow questions, and the browser compatibility related to 18 recent JavaScript features. We conclude that, without a transpiler, almost 14% of the web users could face a JavaScript bug when landing on a website that uses a new JavaScript feature.

A study on classifying Stack Overflow questions based on difficulty by utilizing contextual features

Technical question-answering sites like Stack Overflow are gaining enormous attention from practitioners of specialized fields looking to exchange their programming knowledge. They ask questions on different topics with varying degrees of complexity and difficulty. All practitioners do not have the same level of expertise on those topics to respond to such questions. However, the current approach used by Stack Overflow mostly filters questions based on topics alone and does not take difficulty into account. For this reason, a large percentage of questions fail to attract the attention of appropriate users, resulting in questions having no answer or a significant delay in response time. To address these limitations, we incorporate three models, TF-IDF, LDA, and Doc2Vec, to extract semantic and context-dependent features that can measure the difficulty of questions. Each of these models is paired with different classifiers along with other features to classify the questions based on difficulty. Extensive experiments on three different datasets exhibit the effectiveness of our models, and Doc2Vec outperforms the other models. We also identified that the contextual features are correlated with question difficulty, and one subset of features outperforms others. The proposed approach can be beneficial for building an automatic tagger based on question difficulty.

On the Effectiveness of Transfer Learning for Code Search

The Transformer architecture and transfer learning have marked a quantum leap in natural language processing, improving the state of the art across a range of text-based tasks. This paper examines how these advancements can be applied to and improve code search. To this end, we pre-train a BERT-based model on combinations of natural language and source code data and fine-tune it on pairs of StackOverflow question titles and code answers. Our results show that the pre-trained models consistently outperform the models that were not pre-trained. In cases where the model was pre-trained on natural language “and” source code data, it also outperforms an information retrieval baseline based on Lucene. Also, we demonstrated that the combined use of an information retrieval-based approach followed by a Transformer leads to the best results overall, especially when searching into a large search pool. Transfer learning is particularly effective when much pre-training data is available and fine-tuning data is limited. We demonstrate that natural language processing models based on the Transformer architecture can be directly applied to source code analysis tasks, such as code search. With the development of Transformer models designed more specifically for dealing with source code data, we believe the results of source code analysis tasks can be further improved.

Autonomous Tagging of Stack Overflow Questions

Educational resources like question-and-answer websites like Stack Exchange and Quora are growing in popularity online. A large number of these gatherings depend on labeling, which includes a part marking a post with a suitable assortment of subjects that depict the post and make it more straightforward to find and sort. We give a multi-name order framework that naturally distinguishes clients' requests to upgrade the client experience. A straight SVM and a carefully selected portion of the researched highlight set are used to create a one-versus-rest classifier for a Stack Overflow dataset. By utilizing a subsample of the initial data that is restricted to 100 labels and at least 500 events of each label throughout the data, our characterization framework achieves an ideal F1 score of 62.35 percent.

On the Variability of Software Engineering Needs for Deep Learning: Stages, Trends, and Application Types

The wide use of Deep learning (DL) has not been followed by the corresponding advances in software engineering (SE) for DL. Research shows that developers writing DL software have specific development stages (i.e., SE4DL stages) and face new DL-specific problems. Despite substantial research, it is unclear how DL developers’ SE needs for DL vary over stages, application types, or if they change over time. To help focus research and development efforts on DL-development challenges, we analyze 92,830 Stack Overflow (SO) questions and 227,756 READMEs of public repositories related to DL. Latent Dirichlet Allocation (LDA) reveals 27 topics for the SO questions where 19 (70.4%) topics mainly relate to a single SE4DL stage, and eight topics span multiple stages. Most questions concern <i>Data Preparation</i> and <i>Model Setup</i> stages. The relative rates of questions for 11 topics have increased, for eight topics decreased over time. Questions for the former 11 topics had a lower percentage of accepting an answer than the remaining questions. LDA on README files reveals 16 distinct application types for the 227k repositories. We apply the LDA model fitted on READMEs to the 92,830 SO questions and find that 27% of the questions are related to the 16 DL application types. The most asked question topic varies across application types, with half primarily relating to the second and third stages. Specifically, developers ask the most questions about topics primarily relating to <i>Data Preparation</i> (2nd) stage for four mature application types such as <inline-formula><tex-math notation="LaTeX">${{\sf Image\ Segmentation}}$</tex-math></inline-formula> , and topics primarily relating to <i>Model Setup</i> (3rd) stage for four application types concerning emerging methods such as <inline-formula><tex-math notation="LaTeX">${{\sf Transfer\ Learning}}$</tex-math></inline-formula> . Based on our findings, we distill several actionable insights for SE4DL research, practice, and education, such as better support for using trained models, application-type specific tools, and teaching materials.

Improving Stack Overflow question title generation with copying enhanced CodeBERT model and bi-modal information

Stack Overflow is very helpful for software developers who are seeking answers to programming problems. Previous studies have shown that a growing number of questions are of low quality and thus obtain less attention from potential answerers. Gao et al. proposed an LSTM-based model (i.e., BiLSTM-CC) to automatically generate question titles from the code snippets to improve the question quality. However, only using the code snippets in the question body cannot provide sufficient information for title generation, and LSTMs cannot capture the long-range dependencies between tokens. This paper proposes CCBERT, a deep learning based novel model to enhance the performance of question title generation by making full use of the bi-modal information of the entire question body. CCBERT follows the encoder–decoder paradigm and uses CodeBERT to encode the question body into hidden representations, a stacked Transformer decoder to generate predicted tokens, and an additional copy attention layer to refine the output distribution. Both the encoder and decoder perform the multi-head self-attention operation to better capture the long-range dependencies. This paper builds a dataset containing around 200,000 high-quality questions filtered from the data officially published by Stack Overflow to verify the effectiveness of the CCBERT model. CCBERT outperforms all the baseline models on the dataset. Experiments on both code-only and low-resource datasets show the superiority of CCBERT with less performance degradation. The human evaluation also shows the excellent performance of CCBERT concerning both readability and correlation criteria. CCBERT is capable of automatically capturing the bi-modal semantic information from the entire question body and parsing the long-range dependencies to achieve better performance. Therefore, CCBERT is an effective approach for generating Stack Overflow question titles.

Multi-view approach to suggest moderation actions in community question answering sites

With thousands of new questions posted every day on popular Q&A websites, there is a need for automated and accurate software solutions to replace manual moderation. In this paper, we address the critical drawbacks of crowdsourcing moderation actions in Q&A communities and demonstrate the ability to automate moderation using the latest machine learning models. From a technical point, we propose a multi-view approach that generates three distinct feature groups that examine a question from three different perspectives: 1) question-related features extracted using a BERT-based regression model; 2) context-related features extracted using a named-entity-recognition model; and 3) general lexical features derived using statistical and analytical methods. As a last step, we train a gradient boosting classifier to predict a moderation action. For evaluation purposes, we created a new dataset consisting of 60,000 Stack Overflow questions classified into three choices of moderation actions. Based on cross-validation on the novel dataset, our approach reaches 95.6% accuracy as a multiclass task and outperforms all state-of-the-art and previously-published models. Our results clearly demonstrate the high influence of our feature generation components on the overall success of the classifier.

The reproducibility of programming-related issues in Stack Overflow questions

Software developers often look for solutions to their code-level problems using the Stack Overflow Q&A website. To receive help, developers frequently submit questions that contain sample code segments along with the description of the programming issue. Unfortunately, it is not always possible to reproduce the issues from the code segments they provide. Issues that are not easily reproducible may impede questions from receiving prompt and appropriate solutions. We conducted an exploratory study on the reproducibility of issues discussed in 400 Java and 400 Python questions. We parsed, compiled, executed, and carefully examined the code segments from these questions to reproduce the reported programming issues, expending 300 person-hours of effort. The outcomes of our study are three-fold. First, we can reproduce the issues for approximately 68% of Java and 71% of Python code segments. In contrast, we were unable to reproduce approximately 22% of Java and 19% of Python issues. Of the reproducible issues, approximately 67% of the Java and 20% of the Python code segments required minor or major modifications to reproduce the issues. Second, we carefully investigated why programming issues could not be reproduced and provided evidence-based guidelines to write effective code examples for Stack Overflow questions. Third, we investigated the correlation between the issue reproducibility status of questions and the corresponding answer meta-data, such as the presence of an accepted answer. According to our analysis, a reproducible question has at least two times higher chance of receiving an accepted answer than an irreproducible question. Besides, the median time delay in receiving accepted answers is double if the issues reported in questions could not be reproduced. We also investigated the confounding factors (e.g., user reputation) that can affect questions receiving answers besides reproducibility. We found that such factors do not hurt the correlation between reproducibility status and answer meta-data.

Here we go again

Once a programmer knows one language, they can leverage concepts and knowledge already learned, and easily pick up another programming language. But is that always the case? To understand if programmers have difficulty learning additional programming languages, we conducted an empirical study of Stack Overflow questions across 18 different programming languages. We hypothesized that previous knowledge could potentially interfere with learning a new programming language. From our inspection of 450 Stack Overflow questions, we found 276 instances of interference that occurred due to faulty assumptions originating from knowledge about a different language. To understand why these difficulties occurred, we conducted semistructured interviews with 16 professional programmers. The interviews revealed that programmers make failed attempts to relate a new programming language with what they already know. Our findings inform design implications for technical authors, toolsmiths, and language designers, such as designing documentation and automated tools that reduce interference, anticipating uncommon language transitions during language design, and welcoming programmers not just into a language, but its entire ecosystem.

1+1$&gt;$2: Programming Know-What and Know-How Knowledge Fusion, Semantic Enrichment and Coherent Application

Software programming requires both API reference (know-what) knowledge and programming task (know-how) knowledge. Lots of programming know-what and know-how knowledge is documented in text, for example, API reference documentation and programming tutorials. To improve knowledge accessibility and usage, several recent studies use Natural Language Processing (NLP) methods to construct API know-what knowledge graph (API-KG) and programming task know-how knowledge graph (Task-KG) from software documentation. Although being promising, current API-KG and Task-KG are independent of each other, and thus are void of inherent connections between the two types of knowledge. Our empirical study on Stack Overflow questions confirms that only 36% of the API usage problems can be answered by the know-how or the know-what knowledge alone, while the rest questions requires a fusion of both. Inspired by this observation, we make the first attempt to fuse API-KG and Task-KG by API entity linking. This fusion creates nine categories of API semantic relations and two types of task semantic relations which are not present in the stand-alone API-KG or Task-KG. According to the definitions of these new API and task semantic relations, our approach dives deeper than surface-level API linking of API-KG and Task-KG, and infer nine categories of API semantic relations from task descriptions and two types of task semantic relations with the assistance of API-KG, which enrich the declaration or syntactic relations in the current API-KG and Task-KG. Our fused and semantically-enriched API-Task KG supports coherent API/Task-centric knowledge search by text or code queries. We have implemented our approach on Java programming documentation and built a web tool to search and explore API and programming task knowledge. Our evaluation confirms the high-accuracy of our knowledge extraction, fusion and enrichment methods, and the effectiveness and usefulness of our API-Task KG for answering Stack Overflow questions.

Reproducibility of Issues Reported in Stack Overflow Questions: Challenges, Impact &amp; Estimation

Reproducibility of Issues Reported in Stack Overflow Questions: Challenges, Impact &amp; Estimation

How are framework code samples maintained and used by developers? The case of Android and Spring Boot

Modern software systems are commonly built on top of frameworks. To accelerate the learning process of features provided by frameworks, code samples are made available to assist developers. However, we know little about how code samples are developed and consumed. In this paper, we aim to fill this gap by assessing the characteristics of framework code samples. We provide insights into how code samples are maintained and used by developers. We analyze over 230 code samples provided by Android and Spring Boot, and assess aspects related to their code, evolution, popularity, and client usage. We find that most code samples are small and simple, provide a working environment for the clients, and rely on automated build tools. They frequently change, for example, to adapt to new framework versions. We also detect that clients commonly fork the code samples, however, they rarely modify them. To further understand the problems faced by developers, we analyze 614 Stack Overflow questions about the code samples and 269 issues from code sample repositories. We find that developers face problems when trying to modify the code samples and the most common issue is related to improvement. Finally, we propose implications to creators and clients of code samples to improve maintenance and usage activities.

How do Virtual Badges Incentivize Voluntary Contributions to Online Communities?

Online communities rely heavily on badges to incentivize users to participate in voluntary activities. We examine the relationship between earned badges/pursued badges and contributions in the Stack Overflow question and answer community. Our main results reveal that different levels of earned and pursued badges increase the amount of subsequent answering activity. In addition, earned gold badges are the most influential in the motivation of users while earned bronze badges are the least impactful. Furthermore, the effect of badges involves positive spillovers from higher levels of earned badges to lower ones. Our findings offer important theoretical and managerial implications for online communities.

How do developers discuss and support new programming languages in technical Q&amp;A site? An empirical study of Go, Swift, and Rust in Stack Overflow

New programming languages (e.g., Swift, Go, Rust, etc.) are being introduced to provide a better opportunity for the developers to make software development robust and easy. At the early stage, a programming language is likely to have resource constraints that encourage the developers to seek help frequently from experienced peers active in QA sites such as Stack Overflow (SO). In this study, we have formally studied the discussions on three popular new languages introduced after the inception of SO (2008) and match those with the relevant activities in GitHub whenever appropriate. For that purpose, we have mined 4,17,82,536 questions and answers from SO and 7,846 issue information along with 6,60,965 repository information from GitHub. Initially, the development of new languages is relatively slow compared to mature languages (e.g., C, C++, Java). The expected outcome of this study is to reveal the difficulties and challenges faced by the developers working with these languages so that appropriate measures can be taken to expedite the generation of relevant resources. We have used the LDA method on SO's questions and answers to identify different topics of new languages. We have extracted several features of the answer pattern of the new languages from SO to study their characteristics. These attributes were used to identify difficult topics. We explored the background of developers who are contributing to these languages. We have created a model by combining Stack Overflow data and issues, repository, user data of GitHub. Finally, we have used that model to identify factors that affect language evolution. We believe that the outcome of our study is likely to help the owner/sponsor of these languages to design better features and documentation. It will also help the software developers or students to prepare themselves to work on these languages in an informed way.

Actor concurrency bugs: a comprehensive study on symptoms, root causes, API usages, and differences

Actor concurrency is becoming increasingly important in the development of real-world software systems. Although actor concurrency may be less susceptible to some multithreaded concurrency bugs, such as low-level data races and deadlocks, it comes with its own bugs that may be different. However, the fundamental characteristics of actor concurrency bugs, including their symptoms, root causes, API usages, examples, and differences when they come from different sources are still largely unknown. Actor software development can significantly benefit from a comprehensive qualitative and quantitative understanding of these characteristics, which is the focus of this work, to foster better API documentation, development practices, testing, debugging, repairing, and verification frameworks. To conduct this study, we take the following major steps. First, we construct a set of 186 real-world Akka actor bugs from Stack Overflow and GitHub via manual analysis of 3,924 Stack Overflow questions, answers, and comments and 3,315 GitHub commits, messages, original and modified code snippets, issues, and pull requests. Second, we manually study these actor bugs and their fixes to understand and classify their symptoms, root causes, and API usages. Third, we study the differences between the commonalities and distributions of symptoms, root causes, and API usages of our Stack Overflow and GitHub actor bugs. Fourth, we discuss real-world examples of our actor bugs with these symptoms and root causes. Finally, we investigate the relation of our findings with those of previous work and discuss their implications. A few findings of our study are: (1) symptoms of our actor bugs can be classified into five categories, with Error as the most common symptom and Incorrect Exceptions as the least common, (2) root causes of our actor bugs can be classified into ten categories, with Logic as the most common root cause and Untyped Communication as the least common, (3) a small number of Akka API packages are responsible for most of API usages by our actor bugs, and (4) our Stack Overflow and GitHub actor bugs can differ significantly in commonalities and distributions of their symptoms, root causes, and API usages. While some of our findings agree with those of previous work, others sharply contrast.

Automatic Voter Recommendation Method for Closing Questions in Stack Overflow

Stack Overflow is the most popular programming question and answer community that continuously receives a large number of questions every day. To ensure the quality of questions, the community grants privileges for the moderators and a group of experienced users to review the quality of questions and close the low-quality ones (e.g. duplicate or irrelevant questions). The review process is a typical crowdsourcing job that relies on users’ volunteer participation, and the current practices of closing questions in Stack Overflow face two aspects of challenges: (1) an obvious increase in both the absolute number and the percentage of “closed” questions; (2) a considerable decrease in participation willingness of experienced users to close questions. In order to solve the problem, we present a novel model of user willingness for reviewing and voting questions by incorporating four types of user activity history, including questions, answers, comments and votes of closing questions. Then we propose an automatic recommendation method based on the model to assign experienced users proper questions, to utilize the forces of them to close questions. The evaluation shows that the successful recommendation probability in the top 5, top 10, top 20, top 30, top 40, top 50 users are 48.23%, 58.93%, 68.83%, 74.27%, 78.13% and 81%, respectively.

Duplicate Question Detection With Deep Learning in Stack Overflow

Stack Overflow is a popular Community-based Question Answer (CQA) website focused on software programming and has attracted more and more users in recent years. However, duplicate questions frequently appear in Stack Overflow and they are manually marked by the users with high reputation. Automatic duplicate question detection alleviates labor and effort for users with high reputation. Although existing approaches extract textual features to automatically detect duplicate questions, these approaches are limited since semantic information could be lost. To tackle this problem, we explore the use of powerful deep learning techniques, including Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN) and Long Short-Term Memory (LSTM), to detect duplicate questions in Stack Overflow. In addition, we use Word2Vec to obtain the vector representations of words. They can fully capture semantic information at document-level and word-level respectively. Therefore, we construct three deep learning approaches WV-CNN, WV-RNN and WV-LSTM, which are based on Word2Vec, CNN, RNN and LSTM, to detect duplicate questions in Stack Overflow. Evaluation results show that WV-CNN and WV-LSTM have made significant improvements over four baseline approaches (i.e., DupPredictor, Dupe, DupPredictorRep-T, and DupeRep) and three deep learning approaches (i.e., DQ-CNN, DQ-RNN, and DQ-LSTM) in terms of recall-rate@5, recall-rate@10 and recall-rate@20. Furthermore, the experimental results indicate that our approaches WV-CNN, WV-RNN, and WV-LSTM outperform four machine learning approaches based on Support Vector Machine, Logic Regression, Random Forest and eXtreme Gradient Boosting in terms of recall-rate@5, recall-rate@10 and recall-rate@20.

Modeling stack overflow tags and topics as a hierarchy of concepts

Developers rely on online Q&A forums to look up technical solutions, to pose questions on implementation problems, and to enhance their community profile by contributing answers. Many popular developer communication platforms, such as the Stack Overflow Q&A forum, require threads of discussion to be tagged by their contributors for easier lookup in both asking and answering questions. In this paper, we propose to leverage Stack Overflow’s tags to create a hierarchical organization of concepts discussed on this platform. The resulting concept hierarchy couples tags with a model of their relevancy to prospective questions and answers. For this purpose, we configure and apply a supervised multi-label hierarchical topic model to Stack Overflow questions and demonstrate the quality of the model in several ways: by identifying tag synonyms, by tagging previously unseen Stack Overflow posts, and by exploring how the hierarchy could aid exploratory searches of the corpus. The results suggest that when traversing the inferred hierarchical concept model of Stack Overflow the questions become more specific as one explores down the hierarchy and more diverse as one jumps to different branches. The results also indicate that the model is an improvement over the baseline for the detection of tag synonyms and that the model could enhance existing ensemble methods for suggesting tags for new questions. The paper indicates that the concept hierarchy as a modeling imperative can create a useful representation of the Stack Overflow corpus. This hierarchy can be in turn integrated into development tools which rely on information retrieval and natural language processing, and thereby help developers more efficiently navigate crowd-sourced online documentation.

What’s Spain’s Paris? Mining analogical libraries from Q&amp;A discussions

Third-party libraries are an integral part of many software projects. It often happens that developers need to find analogical libraries that can provide comparable features to the libraries they are already familiar with for different programming languages or different mobile platforms. Existing methods to find analogical libraries are limited by the community-curated list of libraries, blogs, or Q&A posts, which often contain overwhelming or out-of-date information. In this paper, we present a new approach to recommend analogical libraries based on a knowledge base of analogical libraries mined from tags of millions of Stack Overflow questions. The novelty of our approach is to solve analogical-library questions by combining state-of-the-art word embedding technique and domain-specific relational and categorical knowledge mined from Stack Overflow. Given a library and a recommended analogical library, our approach further extracts questions and answer snippets in Stack Overflow about comparison of analogical libraries, which can potentially offer useful information scents for developers to further their investigation of the recommended analogical libraries. We implement our approach in a proof-of-concept web application and more than 34.8 thousands of users visited our website from November 2015 to August 2017. Our evaluation shows that our approach can make accurate recommendation of analogical libraries. We also demonstrate the usefulness of our analogical-library recommendations by using them to answer analogical-library questions in Stack Overflow. Google Analytics of our website traffic and analysis of the visitors’ interaction with website contents provide the insights into the usage patterns and the system design of our web application.

Supporting governance of mobile application developers from mining and analyzing technical questions in stack overflow

There is a need to improve the direct communication between large organizations that maintain mobile platforms (e.g. Apple, Google, and Microsoft) and third-party developers to solve technical questions that emerge during the project and development of developers’ contributions in a Mobile Software Ecosystem (MSECO). In this context, those organizations may not know how to define and evolve strategies to govern their developers towards achieving their organizational goals. Such organizations use an infrastructure to support developers, for example, questions and answers (Q&A) portals such as Stack Overflow. Interactions among developers in these portals feed a Q&A repository that can serve as a mechanism to understand and define strategies to support developers. In this paper, we mined 1,568,377 technical questions from Stack Overflow related to Android, iOS, and Windows Phone platforms. Next, we performed comparisons among those MSECO regarding: (i) developers’ activity intensity, (ii) hot-topics (using Latent Dirichlet allocation algorithm) from all and more commented/viewed questions, (iii) “What” and “How to” questions, (iv) hot-topics from more viewed unanswered questions, and (v) relationship among questions and official developer events. From the results, we identified four key insights: recruiting, educating, and monitoring strategies; barrier reduction; management of technology insertion; and fostering of relationships. The relevance of the four key insights to support developer governance was evaluated by practitioners through a survey. Finally, for each key insight we associated a total of 10 strategies to support developer governance activities. Such strategies were extracted from 65 studies identified through a systematic mapping of the literature.