The eye of the beholder: how do public health researchers interpret regression coefficients? A qualitative study.

Abstract

Calls for improved statistical literacy and transparency in population health research are widespread, but empirical accounts describing how researchers understand statistical methods are lacking. To address this gap, this study aimed to explore variation in researchers' interpretations and understanding of regression coefficients, and the extent to which these statistics are viewed as straightforward statements about health. Thematic analysis of qualitative data from 45 one-to-one interviews with academics from eight countries, representing 12 disciplines. Three concepts from the sociology of scientific knowledge and science studies aided analysis: Duhem's Paradox, the Agonistic Field, and Mechanical Objectivity. Some interviewees viewed regression as a process of discovering 'real' relationships, while others indicated that regression models are not direct representations, and others blended these perspectives. Regression coefficients were generally not viewed as being mechanically objective, instead interpretation was described as iterative, nuanced, and sometimes depending on prior understandings. Researchers reported considering numerous factors when interpreting and evaluating regression results, including: knowledge from outside the model, whether results are expected or unexpected, 'common-sense', technical limitations, study design, the influence of the researcher, the research question, data quality and data availability. Interviewees repeatedly highlighted the role of the analyst, reinforcing that it is researchers who answer questions and assign meaning, not models. Regression coefficients were generally not viewed as complete or authoritative statements about health. This contrasts with teaching materials wherein statistical results are presented as straightforward representations, subject to rule-based interpretations. In practice, it appears that regression coefficients are not understood as mechanically objective. Attempts to influence conduct and presentation of regression models in the population health sciences should be attuned to the myriad factors which inform their interpretation.