Abstract
Code review is a powerful technique to ensure high quality software and spread knowledge of best coding practices between engineers. Unfortunately, code reviewers may have biases about authors of the code they are reviewing, which can lead to inequitable experiences and outcomes. In principle, anonymous author code review can reduce the impact of such biases by withholding an author's identity from a reviewer. In this paper, to understand the engineering effects of using author anonymous code review in a practical setting, we applied the technique to 5217 code reviews performed by 300 software engineers at Google. Our results suggest that during anonymous author code review, reviewers can frequently guess authors’ identities; that focus is reduced on reviewer-author power dynamics; and that the practice poses a barrier to offline, high-bandwidth conversations. Based on our findings, we recommend that those who choose to implement anonymous author code review should reveal the time zone of the author by default, have a break-the-glass option for revealing author identity, and reveal author identity directly after the review.
Highlights
While developers believe that code changes are accepted based on change quality and fitness [1], prior research suggests that when women use profile pictures and gender-identifiable names, the acceptance of their open source code contributions drops, compared to peers with gender-neutral profiles [2]
We examined the impact of anonymous author code review on quality more broadly, so we asked all participants, “If anonymous author code review was regularly practiced at Google, I expect that. . . ” As Figure 4 shows, most (57%) treatment group participants thought that anonymous author code review would be neutral toward review quality, with most of the remainder of the treatment group participants (24%) expecting positive impact on quality
A later study at Microsoft confirmed that a reviewer provided more useful comments when they had prior experience with the code being reviewed, though there was no difference based on whether the reviewer and author were on the same team [43]. This is important for the effectiveness of anonymous author code review: reviewers provide more useful comments if they are familiar with the code, but our study suggests that reviewers who are familiar with the code may be more likely to successfully guess the author’s identity
Summary
While developers believe that code changes are accepted based on change quality and fitness [1], prior research suggests that when women use profile pictures and gender-identifiable names, the acceptance of their open source code contributions drops, compared to peers with gender-neutral profiles [2]. Implicit bias in professional decision making is increasingly handled through anonymization, where irrelevant personal details are purposefully hidden from the decision maker. With a similar motivation, having engineers review code changes without being explicitly informed of who made those changes – anonymous author code review1 – can in principle reduce the effect of bias in organizations. Kim and colleagues identified anonymous author code review as a step towards reducing structural Reviewers add comments about the change and ask for further changes, if necessary
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
