Revisiting the accuracy problem in network analysis using a unique dataset

Abstract

A series of papers published by Bernard and colleagues in the late 1970s and early 1980s, dubbed the “accuracy studies,” called into the question the validity of self-reported perceived communication in the study of networks, showing that such reports explain only about 20 % of the variance in directly observed communication. Questions remain about how well the kinds of organizations studied reflect typical formal organizations, the studies’ short observation periods, and manual observation methods. This study revisits the accuracy studies using a unique dataset comprising 144 weeks of network surveys and machine classification of 7,000 h of audio recordings to measure observable communication in a software engineering unit employing 54 people. Results show that correlations between perceived and observed communication over the weeks studied have a lower average than that reported in the accuracy studies but vary considerably from week to week. It also replicates results of earlier research showing that participants tend to overreport communication when they perceive a strong structural relationship to the alters they are rating. This study solidifies our knowledge about network self-reports using a stronger data foundation than prior research employed. Its results, along with the previous research, suggest that perceived communication is not so much a flawed measure of observable communication as it is a related, yet distinct phenomenon. This highlights the need for developments in theory and modeling that articulate the relationship between perceived and observed communication, and Network Reticulation Theory is suggested as a viable approach.