Stack Exchange Research Articles

Capturing and querying fine-grained provenance of preprocessing pipelines in data science

Data processing pipelines that are designed to clean, transform and alter data in preparation for learning predictive models, have an impact on those models' accuracy and performance, as well on other properties, such as model fairness. It is therefore important to provide developers with the means to gain an in-depth understanding of how the pipeline steps affect the data, from the raw input to training sets ready to be used for learning. While other efforts track creation and changes of pipelines of relational operators, in this work we analyze the typical operations of data preparation within a machine learning process, and provide infrastructure for generating very granular provenance records from it, at the level of individual elements within a dataset. Our contributions include: (i) the formal definition of a core set of preprocessing operators, and the definition of provenance patterns for each of them, and (ii) a prototype implementation of an application-level provenance capture library that works alongside Python. We report on provenance processing and storage overhead and scalability experiments, carried out over both real ML benchmark pipelines and over TCP-DI, and show how the resulting provenance can be used to answer a suite of provenance benchmark queries that underpin some of the developers' debugging questions, as expressed on the Data Science Stack Exchange.

A novel community answer matching approach based on phrase fusion heterogeneous information network

Community Question Answering (CQA) allows users to ask or answer questions in a social way, so it is becoming the primary means for people acquiring knowledge. However, the asker must wait until a satisfactory answer appears, which reduces user activity. In this paper, we propose an innovative answering method that matches the most relevant answers for the new issue automatically. Firstly, we utilize phrases to represent the semantic of the posts (answers/questions) and construct a Phrase Fusion Heterogeneous Information Network, called PFHIN, to represent complex entity relationships in CQA. So, the answer selection is regarded as the related entity retrieval task. Then, we define the distance between entities in PFHIN, which is independent of the meta path. Finally, the Type-constrained Top-k Similarity Entity Finding Algorithm (TTSEF) is proposed for finding the nearest entities according to the known start entity and end-entity type, which can match the most relevant answers automatically.To the best of our knowledge, it is the first work to define the phrase information network for answer selection and provide a novel idea for the heterogeneous information network fusion. Experimental results on three large-scale datasets (Stack Overflow, Super User, and Mathematics) from Stack Exchange demonstrate that our proposed approaches significantly outperform the state-of-the-art answer retrieval methods. Moreover, we conduct an in-depth analysis of the meta path to the optimal answer and reveal the critical role of phrases in community answer matching.

One size does not fit all: A study of badge behavior in stack overflow

AbstractBadges are endemic to online interaction sites, from question and answer (Q&A) websites to ride sharing, as systems for rewarding participants for their contributions. This article studies how badge design affects people's contributions and behavior over time. Past work has shown that badges “steer” people's behavior toward substantially increasing the amount of contributions before obtaining the badge, and immediately decreasing their contributions thereafter, returning to their baseline contribution levels. In contrast, we find that the steering effect depends on the type of user, as modeled by the rate and intensity of the user's contributions. We use these measures to distinguish between different groups of user activity, including users who are not affected by the badge system despite being significant contributors to the site. We provide a predictive model of how users change their activity group over the course of their lifetime in the system. We demonstrate our approach empirically in three different Q&A sites on Stack Exchange with hundreds of thousands of users, for two types of activities (editing and voting on posts).

An Empirical Study of C++ Vulnerabilities in Crowd-Sourced Code Examples

Software developers share programming solutions in Q&A sites like Stack Overflow, Stack Exchange, Android forum, and so on. The reuse of crowd-sourced code snippets can facilitate rapid prototyping. However, recent research shows that the shared code snippets may be of low quality and can even contain vulnerabilities. This paper aims to understand the nature and the prevalence of security vulnerabilities in crowd-sourced code examples. To achieve this goal, we investigate security vulnerabilities in the C++ code snippets shared on Stack Overflow over a period of 10 years. In collaborative sessions involving multiple human coders, we manually assessed each code snippet for security vulnerabilities following CWE (Common Weakness Enumeration) guidelines. From the 72,483 reviewed code snippets used in at least one project hosted on GitHub, we found a total of 99 vulnerable code snippets categorized into 31 types. Many of the investigated code snippets are still not corrected on Stack Overflow. The 99 vulnerable code snippets found in Stack Overflow were reused in a total of 2859 GitHub projects. To help improve the quality of code snippets shared on Stack Overflow, we developed a browser extension that allows Stack Overflow users to be notified for vulnerabilities in code snippets when they see them on the platform.

Domain problem‐solving expert identification in community question answering

AbstractQuestion‐Answering (Q&A) services provide internet users with platforms to exchange knowledge and ideas. The development of Q&A sites, or Community Question Answering (CQA), mainly depends on the high‐quality content continuously contributed by users with high‐level expertise, who can be recognized as experts. Expert finding is an important task for the authorities of Q&A communities to encourage commitment. In a highly competitive market environment, CQA managers have to take measures to retain and nurture users, especially superior contributors. However, current expertise scoring techniques adopted in CQA often give much credit to very active users and fail to identify real experts. This study aims to develop a robust and practical expert identification framework for Q&A communities, by combining well‐designed expertise scoring technique and probabilistic clustering model. With regard to expert identification, a numerical metric of users' expertise is developed as the optimal expert finding strategy, and a clustering algorithm based on Gaussian‐Gamma mixture model (GGMM) is proposed to efficiently distinguish experts from nonexperts. In the experiments, the proposed method is applied to real‐world datasets collected from subcommunities of Stack Exchange Q&A networks. Results obtained from comparative experiments show that our method achieves better performance than the state‐of‐the‐art methods and demonstrate the effectiveness of the proposed framework. The analysis shows that the framework which combines the proposed expertise scoring technique and Gaussian–Gamma mixture clustering model is capable of detecting excellent domain problem‐solving experts who exhibit both domain interest and expertise.

Can Badges Foster a More Welcoming Culture on Q&A Boards?

Thriving online communities rely on a steady stream of newcomers to contribute new content. However, retaining newcomers has proven challenging. In this paper, we measure the success of an intervention used by Stack Exchange question-answering communities to create a more welcoming environment for newcomers. That intervention consisted in highlighting contributions by new users with a special indicator. We hypothesize that Stack Exchange's new policy would reduce negative reactions to new users and, ultimately, increase new user retention. We leverage causal modeling to assess the introduction of the so-called “new contributor indicator”, and we find it did not counter user retention decline in the short- and long-terms. However, our results indicate it did reduce unwelcoming reactions towards newcomers in the short-term. Our work has practical implications for online community managers aiming to improve their onboarding processes.

How Open Crowds Self-Organize

The ability of open crowds to produce useful and novel solutions is of significant research and practical interest. Much of the existing research has assumed that crowd participants approach problem-solving autonomously and independently of one another. In the Stack Exchange network, we explored problems posted by open crowds and the sequences of suggestions and comments leading up to a solution that was novel and judged by the solution seeker as useful for solving the problem. In the sequences of suggestions, we found indications of self-organization as the crowd emergently accumulated knowledge in specific patterns that were more likely to lead to accepted solutions. The patterns contained positive and negative forms of feedback ensuring that the crowd would, on its own and without external governance intervention, engage in actions that eventually led to accepted solutions. In total, based on the observed knowledge accumulation patterns, three self-organization principles were derived and incorporated into a theory of self-organization for creative solution generation in open crowds. Our theory development suggests implications for research on open crowds specifically, as well as for research on new organizational forms.

A Study of Bug Management Using the Stack Exchange Question and Answering Platform

Traditional bug management systems, like Bugzilla, are widely used in open source and commercial projects. Stack Exchange uses its online question and answer (Q&A) platform to collect and manage bugs, which brings several new unique features that are not offered in traditional bug management systems. Users can edit bug reports, use different communication channels, and vote on bug reports, answers, and their associated comments. Understanding how these features manage bug reports can provide insights to the designers of traditional bug management systems, like whether a feature should be introduced? and how would users leverage such a feature? We performed a large-scale analysis of 19,151 bug reports of the bug management system of Stack Exchange and studied the in-place editing, the answering and commenting, and the voting features. We find that: 1) The three features are used actively. 2) 57 percent of the edits improved the quality of bug reports. 3) Commenting provides a channel for discussing bug-related information, while answering offers a channel for explaining the causes of a bug and bug-fix information. 4) Downvotes are made due to the disagreement of the reported “bug” being a real bug and the low quality of bug reports. Based on our findings, we provide suggestions for traditional bug management systems.

A large scale study on how developers discuss code smells and anti-pattern in Stack Exchange sites

Abstract Context: In this paper, we investigate how developers discuss code smells and anti-patterns across three technical Stack Exchange sites. Understanding developers perceptions of these issues is important to inform and align future research efforts and direct tools vendors to design tailored tools that best suit developers. Method: we mined three Stack Exchange sites and used quantitative and qualitative methods to analyse more than 4000 posts that discuss code smells and anti-patterns.Results: results showed that developers often asked their peers to smell their code, thus utilising those sites as an informal, crowd-based code smell/anti-pattern detector. The majority of questions (556) asked were focused on smells like Duplicated Code, Spaghetti Code, God and Data Classes. In terms of languages, most of discussions centred around popular languages such as C # (772 posts), JavaScript (720) and Java (699), however greater support is available for Java compared to other languages (especially modern languages such as Swift and Kotlin). We also found that developers often discuss the downsides of implementing specific design patterns and ‘flag’ them as potential anti-patterns to be avoided. Some well-defined smells and anti-patterns are discussed as potentially being acceptable practice in certain scenarios. In general, developers actively seek to consider trade-offs to decide whether to use a design pattern, an anti-pattern or not.Conclusion: our results suggest that there is a need for: 1) more context and domain sensitive evaluations of code smells and anti-patterns, 2) better guidelines for making trade-offs when applying design patterns or eliminating smells/anti-patterns in industry, and 3) a unified, constantly updated, catalog of smells and anti-patterns. We conjecture that the crowd-based detection approach considers contextual factors and thus tend to be more trusted by developers than automated detection tools.

An authentic feeling? Religious experience through Q&A websites

As the “Sacred Place”—meant as the new space for religions offered by the Internet—demands for continuous investigations on the encounter between traditional narratives and social practices, the rapid growth of Question and Answering websites asks for improving social research about the Authenticity of the religious feeling as well as their responsibility in the construction of a shared knowledge. In this background, the aim of this study is to investigate the role of Q&A websites as additional interpretative resources in accordance with different religious forms of life. About 800 extracts—composed by questions and answers—from the religious pages of Stack Exchange were analyzed, in accordance with social discursive psychology, through bottom-up and top-down pathways. In relation to the different emerging questioners’ profiles, the rhetoric of “closeness” and “openness” reveal a dialectic trend of these websites in offering both supplementary and extending religious experiences.

How to Ask Better Questions? A Large-Scale Multi-Domain Dataset for Rewriting Ill-Formed Questions

We present a large-scale dataset for the task of rewriting an ill-formed natural language question to a well-formed one. Our multi-domain question rewriting (MQR) dataset is constructed from human contributed Stack Exchange question edit histories. The dataset contains 427,719 question pairs which come from 303 domains. We provide human annotations for a subset of the dataset as a quality estimate. When moving from ill-formed to well-formed questions, the question quality improves by an average of 45 points across three aspects. We train sequence-to-sequence neural models on the constructed dataset and obtain an improvement of 13.2% in BLEU-4 over baseline methods built from other data resources. We release the MQR dataset to encourage research on the problem of question rewriting.1

The price of being polite: politeness, social status, and their joint impacts on community Q&A efficiency

Sociolinguistics and computational linguistics literature have revealed negative correlations between social status and politeness in interpersonal conversations. In this article, we took a step further to uncover how social status and politeness interact with each other to jointly impact the efficiency of the Q&A process in online social Q&A communities. Using the data collected from two communities of Stack Exchange, we demonstrated that both social status and politeness had significant impacts to determine the efficiency of receiving acceptable answers. Moreover, while low-status users benefited from wording their questions more politely, high-status users were slightly “punished” for being too polite, particular in professional Q&A communities. However, social status and politeness were not significantly relevant to whether a question could be eventually answered. In general, the social Q&A process provides the conditions necessary for the manifestation of “offline” social norms. That is: individuals are still being rewarded for behaving correctly according to their social roles, no matter explicitly or implicitly. We discuss the theoretical and practical implications of this study.

Safe Food Handling Advice Provided on Question-and-Answer Web Sites Is Inconsistent

ObjectiveTo determine the accuracy of information about safe food handling posted on online question-and-answer (Q&A) Web sites and forums. DesignA retrospective study was conducted to analyze answers to questions posted on 4 Q&A platforms (Reddit, Yahoo! Answers, Quora, and Stack Exchange) employing a structured coding form. Questions were identified from November 2018 to May 2019 using an iterative search. ParticipantsUsers of online Q&A platforms. Main Outcome MeasureAnswer accuracy (incorrect, partially correct, or correct in relation to government guidelines). AnalysisA multivariable ordinal regression model was developed to identify factors associated (at the P < .01 level) with answer accuracy. ResultsQuestions (n = 160) were posted from 2006 to 2019. Of 625 answers to these questions, 41% were classified as correct, and 20% as partially correct. Answer accuracy differed among the 4 platforms. Answers to questions about adequate cooking (odds ratio [OR] = 0.29, 95% confidence interval [CI], 0.17–0.49, P < .001) and time–temperature control (OR = 0.33, 95% CI, 0.21–0.51, P < .001) constructs were less likely to be correct than answers to other questions. Top-rated answers were more likely to be correct than other answers (OR = 2.01, 95% CI, 1.40–2.90, P < .001). Conclusions and ImplicationsThe accuracy of food safety information exchanged on online Q&A platforms is inconsistent. Enhanced food safety promotion and education interventions on these platforms are warranted.

Forum Duplicate Question Detection by Domain Adaptive Semantic Matching

Community Question Answering (CQA) forums, such as Stack Overflow, Stack Exchange and Massive Open Online Course (MOOC) forums, spend a lot of manpower and time to manage duplicate questions on the forum. Mismatch of duplicate questions makes users keep asking “new” questions, and the continuous accumulation of duplicate questions may interfere with their information searching again, affecting user satisfaction. Neural Networks (NN) models for parsing semantics provide the possibility of end-to-end duplicate question detection. Whereas, due to lack of domain data and expertise, NN models for semantic parsing are rarely directly applied to CQA duplicate question detection. This paper proposes a Semantic Matching Model (SMM) integrated with the multi-task transfer learning framework for multi-domain forum duplicate question detection. By designing the word-to-sentence interaction mechanism based on the word-to-word interaction, SMM can automatically choose to ignore or pay attention to potential similar words according to the semantics at the sentence level. The experiments on the benchmark data set and MOOC forum data set state that SMM outperforms baselines, its interaction mechanism is effective and it has an advantage in cross-domain duplicate question detection.

Hierarchical Attentional Factorization Machines for Expert Recommendation in Community Question Answering

The most challenging task of Community Question Answering (CQA) is to provide high-quality answers to users’ questions. Currently, a variety of expert recommendation methods have been proposed and greatly improved the effective matching between questions and potential good answerers. However, the performance of existing methods can be adversely affected by many common factors such as data sparsity and noise problem, which cause less precise user modeling. Moreover, existing methods often model user-question interactions through simple ways, failing to capture the multiple scale interactions of question and answerers, which make it difficult to find answerers who are able to provide the best answers. In this paper, we propose an attention-based variant of Factorization Machines (FM) called Hierarchical Attentional Factorization Machines (HaFMRank) for answerer recommendation in CQA, which not only models the interactions between pairs of individual features but emphasizes the roles of crucial features and pairwise interactions. Specifically, we introduce the within-field attention layer to capture the inner structure of features belonging to the same field, while a feature-interaction attention layer is adopted to examine the importance of each pairwise interaction. A pre-training procedure is designed to generate latent FM feature embedding that encode question context and user history into the training process of HaFMRank. The performance of the proposed HaFMRank is evaluated by using real-world datasets of Stack Exchange and experimental results demonstrate that it outperforms several state-of-the-art methods in best answerer recommendation.

Developer Recommendation for Stack Exchange Software Engineering Q&A Website based on K-Means clustering and Developer Social Network Metric

Nowadays Online question answering website platforms are getting popular as it allows the users to get responses from varied experts beyond their reach. Businesses use these sites for their growth and exposure. Enormous volumes of question are posted on these sites. Finding an expert for answering large voluminous questions requires specialized techniques. This paper proposes hybrid approach for finding prominent expert for the posted questions. The proposed approach uses clustering and social network metrics. The approach clusters the database questions based on tag similarity using K Means. The new posted question is matched with the clusters and best suitable cluster is extracted. Developer Social Network is constructed for chosen cluster and Page Rank with time decay has been applied to find the prominent developer. The proposed approach attains 60% accuracy.

A Content-Based Model for Tag Recommendation in Software Information Sites

Abstract Developers use software information sites such as Stack Overflow to get and give information on various subjects. These sites allow developers to label content with tags as a short description. Tags, then, are used to describe, categorize and search the posted content. However, tags might be noisy, and postings may become poorly categorized since people tag a posting based on their knowledge of its content and other existing tags. To keep the content well organized, tag recommendation systems can help users by suggesting appropriate tags for their posted content. In this paper, we propose a tag recommendation scheme that uses the textual content of already tagged postings to recommend suitable tags for newly posted content. Our approach combines multi-label classification and textual similarity techniques to improve the performance of tag recommendation. We evaluate the performance of the proposed scheme on 11 software information sites from the Stack Exchange network. The results show a significant improvement over TagCombine, TagMulRec and FastTagRec, which are well-known tag recommendation systems. On average, the proposed model outperforms TagCombine, TagMulRec and FastTagRec by 26.2, 15.9 and 13.8% in terms of Recall@5 and by 16.9, 12.4 and 9.4% in terms of Recall@10, respectively.

Modelling and analysis of tagging networks in Stack Exchange communities

AbstractLarge Question-and-Answer (Q&amp;A) platforms support diverse knowledge curation on the Web. While researchers have studied user behaviour on such platforms in a variety of contexts, there is relatively little insight into important by-products of user behaviour that also encode knowledge. Here, we analyse and model the macroscopic structure of tags applied by users to annotate and catalogue questions, using a collection of 168 Stack Exchange websites that span a diversity of sizes and topics. We study the distribution of tag frequencies and also the structure of ‘co-tagging’ networks where nodes are tags and links connect tags that have been applied to the same question. We find striking similarity in tagging structure across Stack Exchange communities, even though each community evolves independently (albeit under similar guidelines). Our findings thus provide evidence that social tagging behaviour is largely driven by the Stack Exchange platform itself and not by the individual Stack Exchange communities. We also develop a simple generative model that creates random bipartite graphs of tags and questions. Our model accounts for the tag frequency distribution but does not explicitly account for co-tagging correlations. Even under these constraints, we demonstrate empirically and theoretically that our model can reproduce a number of the statistical properties that characterize co-tagging networks.

Quantifying Voter Biases in Online Platforms

In content-based online platforms, use of aggregate user feedback (say, the sum of votes) is commonplace as the "gold standard" for measuring content quality. Use of vote aggregates, however, is at odds with the existing empirical literature, which suggests that voters are susceptible to different biases-reputation (e.g., of the poster), social influence (e.g., votes thus far), and position (e.g., answer position). Our goal is to quantify, in an observational setting, the degree of these biases in online platforms. Specifically, what are the causal effects of different impression signals-such as the reputation of the contributing user, aggregate vote thus far, and position of content-on a participant's vote on content? We adopt an instrumental variable (IV) framework to answer this question. We identify a set of candidate instruments, carefully analyze their validity, and then use the valid instruments to reveal the effects of the impression signals on votes. Our empirical study using log data from Stack Exchange websites shows that the bias estimates from our IV approach differ from the bias estimates from the ordinary least squares (OLS) method. In particular, OLS underestimates reputation bias (1.6-2.2x for gold badges) and position bias (up to 1.9x for the initial position) and overestimates social influence bias (1.8-2.3x for initial votes). The implications of our work include: redesigning user interface to avoid voter biases; making changes to platforms' policy to mitigate voter biases; detecting other forms of biases in online platforms.

SIEVE: Helping developers sift wheat from chaff via cross-platform analysis

Software developers have benefited from various sources of knowledge such as forums, question-and-answer sites, and social media platforms to help them in various tasks. Extracting software-related knowledge from different platforms involves many challenges. In this paper, we propose an approach to improve the effectiveness of knowledge extraction tasks by performing cross-platform analysis. Our approach is based on transfer representation learning and word embedding, leveraging information extracted from a source platform which contains rich domain-related content. The information extracted is then used to solve tasks in another platform (considered as target platform) with less domain-related content. We first build a word embedding model as a representation learned from the source platform, and use the model to improve the performance of knowledge extraction tasks in the target platform. We experiment with Software Engineering Stack Exchange and Stack Overflow as source platforms, and two different target platforms, i.e., Twitter and YouTube. Our experiments show that our approach improves performance of existing work for the tasks of identifying software-related tweets and helpful YouTube comments.