An analysis of methodologic quality in survey research reported in the Journal of Clinical Oncology.

Abstract

6616 Background: Survey research (SR) has been increasing as a percentage of published manuscripts in medical journals. SR plays an important role in studies of quality of life and patient preferences in treatment. Appropriate quality of methodology in SR is critical both to assure reliability and validity of survey results as well as to derive sound generalizations for larger populations from the subsets surveyed. Surveys that have deficient methodological criteria may suffer from significant flaws. A complete description and discussion of quality survey methodology, analysis, and results is essential for a thorough understanding and evaluation of published SR. Methods: Between January 2006 and December 2010, 227 articles in JCO were identified to have either “survey” or “questionnaire” in either the title or abstract. The most recent 52 consecutive articles fulfilling criteria from that time period were reviewed for reporting of survey methodology. A modification of Bennett et al.’s checklist for reporting SR was used for this analysis (Bennett C., et al. Reporting guidelines for survey research: An analysis of published guidance and reporting practices. PLoS Med 8(8): e1001069, 2011.). 35 metrics were used to analyze each survey. These metrics were grouped in the following categories: 1) title and abstract, 2) introduction, 3) methods (research tool, sample selection, survey administration and analysis), 4) results, 5) discussion, and 6) ethical quality indicators. Results: Of the 52 survey articles reviewed, the top quartile reported greater than 72% of the analyzed metrics. Half of the articles contained 63% or more of the desired metrics, and half of the articles had between 42% and 62% of the desired metrics. Some metrics were usually reported, such as ethics board review (reported in 85% of articles), but others were not consistently reported, such as calculation or justification of sample size (neither were reported in 71% of articles). Conclusions: A substantial number of articles reviewed reporting on survey research in JCO did not report critical components of survey methodology. More rigorous quality recommendations should be offered to guide authors in the report of survey research results.