When “Normal” Becomes Normative: A Case Study of Researchers’ Quotation Errors When Referring to a Focus Group Sample Size Study

Abstract

In 2011, we published a review exploring how researchers report and justify their focus group sample sizes. We concluded that sample sizes vary widely and that most researchers give no explanation for their sample size. The aim of our 2011 study was to describe practice rather than develop guidance. However, after our study was published, we noticed that new researchers were using our information about typical sample sizes as justification for their own sample size. In other words, practice that we had presented as typical or “normal” but generally lacking in justification was being used as normative. The current study aims to explore the misrepresentation of descriptive information as normative. Specifically, we map this type of quotation error in references to our 2011 study. Using Google Scholar, we identified all articles referencing our study. We then extracted quotations where the researchers had referred to our study and categorized these as follows: (a) quotations where the researchers had used the descriptive information from our study to justify their sample size and (b) quotations where the researchers had referred to our study for other purposes or where the purpose was unclear. We assessed 205 articles that had referred to our 2011 study. We identified the type of quotation error we were interested in, namely the misrepresentation of descriptive information as normative, in 50.7% of the included articles. Our study shows very high rates of one type of quotation error: the misrepresentation of descriptive information about focus group sample size as normative. Researchers referring to other researchers’ work carry most of the responsibility for ensuring that they do this appropriately. However, the authors of the research being referred to also need to consider how they can make their results clearer. We offer suggestions as to how this might be achieved.