The Impact of Surface Features on Choice of (in)Secure Answers by Stackoverflow Readers

Dirk Van Der Linden, Emma Williams + Show 2 more

Open Access

https://doi.org/10.1109/tse.2020.2981317

Copy DOI

Abstract

Existing research has shown that developers will use StackOverflow to answer programming questions: but what draws them to one particular answer over any other? The choice of answer they select can mean the difference between a secure application and insecure one, as the quality of supposedly secure answers can vary. Prior work has studied people posting on Stack Overflow—a two-way communication between the original poster and the Stack Overflow community. Instead, we study the situation of one-way communication, where people only read a Stack Overflow thread without being actively involved in it, sometimes long after a thread has closed. We report on a mixed-method study including a controlled between-groups experiment and qualitative analysis of participants’ rationale (N=1188), investigating whether explanation detail, answer scoring, accepted answer marks, as well as the security of the code snippet itself affect the answers participants accept. Our findings indicate that explanation detail affects what answers participants reading a thread select (p<0.01), while answer score and acceptance do not (p>0.05)—the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">inverse</i> of what research has shown for those asking and answering questions. The qualitative analysis of participants’ rationale further explains how several cognitive biases underpin these findings. Correspondence bias, in particular, plays an important role in instilling readers with a false sense of confidence in an answer through the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">way it looks</i> , regardless of whether it works, is secure, or if the community agrees with it. As a result, we argue that StackOverflow's use as a knowledge base by people not actively involved in threads—when there is only one-way-communication—may inadvertently contribute to the spread of insecure code, as the community's voting mechanisms hold little power to deter them from answers.

Highlights

COPYING and pasting code snippets from Stack Overflow is a well known, widespread, phenomenon among software developers [1]
What of the much wider group of readers who use Stack Overflow as a knowledge base: discovering questions and answers long after their threads have gone silent? Prior work has focused on the posters actively engaged in twoway communication—the question askers and answerers— we instead ask what drives Stack Overflow readers, who engage in one-way communication, to chose one answer over another in completed threads, and, to what extent can this be potentially manipulated by unscrupulous posters?
We study what features affect Stack Overflow readers when selecting an answer, and with what rationale do they do so, by exploring the following research questions: 1) Do Stack Overflow readers select answers based on security of code snippets?

Summary

Introduction

COPYING and pasting code snippets from Stack Overflow is a well known, widespread, phenomenon among software developers [1]. Developers often copy-paste snippets without realizing the impact on security [2], [3] This leads to rapidly spreading [4] less secure code [5], [6], [7] and inhibits developers’ security thinking [8]. This may be in large part by API documentation confusing developers and offering too little clear guidance on parameters and configuration [2]. Answers to such challenges often enabled vulnerabilities by advocating insecure workarounds, not contributing to an understanding of secure TLS use [3]

Methods

Results

Discussion

Conclusion