Code review effectiveness: an empirical study on selected factors influence

Abstract

Code review is a widely adopted means of improving software quality. There are numerous techniques of reviewing including tool-assisted and over-the-shoulder. Both require significant effort to perform. However, the impact the over-the-shoulder has on review effectiveness remains unexplored. The authors goal was to compare the techniques and the effect of review size to analyse how that affects the effectiveness. They performed three experiments within one company. Changes provided by developers were reviewed twice by different reviewers, each time using a different technique. Afterwards, participants rated in a questionnaire the influence of each technique to the knowledge transfer. They observed that the number of accepted comments reported with the tool-assisted technique is approximately twice as big. Also, they noticed a relationship between comments density and the review size: the correlations vary from − 0.42 to − 0.33. The knowledge transfer was typically evaluated as it supports knowledge sharing to a limited extent for the tool-assisted technique but as it is good at supporting knowledge sharing for the over-the-shoulder technique. The results did not give a simple answer whether one technique outperforms the other. The tool-assisted technique results in more comments. However, the teams evaluated the over-the-shoulder technique as better supporting knowledge transfer.