How to assess a survey in surgery

Abstract

The use of surveys is expanding in all domains of society. Frequently surgeons are presented with hard-copy or email surveys asking for information about their knowledge, beliefs, attitudes and practice patterns. The purpose of these questionnaires may be to obtain an accurate picture of what is going on in their surgical practices, and results are used by local, regional or national organizations to effect changes in surgical practice. Questionnaires can collect descriptive (reporting actual data) or explanatory (drawing inferences between constructs or concepts) data and can explore several constructs at a time.1,2 There are 2 basic types of surveys: cross-sectional and longitudinal surveys. Some cross-sectional surveys gather descriptive information on a population at a single time (e.g., survey of orthopedic trauma surgeons to explore the influence of physician and practice characteristics on referral for physical therapy in patients with traumatic lower-extremity injuries3). A different cross-sectional survey questionnaire might be designed to determine the relation between 2 factors on a representative sample at a particular time. For example, a population-based cross-sectional survey was conducted to explore geographic and sociodemographic factors associated with variation in the accessibility of total hip and knee replacement surgery in England.4 The authors found evidence of unequal access based on age, sex, rurality and race. Longitudinal surveys are conducted to determine changes in a population over a period of time.5 An example is a prospective longitudinal survey on quality of life among 558 women with breast cancer who underwent surgical treatment and were compared according to whether or not they received chemotherapy.6 The authors reported that the quality of life of both groups improved significantly in the year after primary treatment ended, but adjuvant chemotherapy was associated with more severe physical symptoms. Note that prevalence rather than incidence is normally determined in a cross-sectional survey. On the other hand, the temporal sequence of a cause and effect relation can be assessed using longitudinal surveys. The aim of a survey is to obtain reliable and unbiased data from a representative sample.7 Surveys can have a major impact if surgical organizations act on the results. If the surveys have sound methodology, most likely their inferences are correct and will be helpful. However, if proper methodology was not considered and inferences are adopted, surveys can have undesired consequences. High response rates are needed to ensure validity and reduce nonresponse bias.1,2 Response rates to mail and email surveys are particularly low among surgeons, some as low as 9%.8–11 Response rates as high as 80% have also been reported.12,13 The purpose of this article is to help surgeons critically appraise survey results reported in the surgical literature.