Survey Research: A Primer for Hand Surgery

Abstract

The majority of members of the American Society of Hand Therapy have been asked to participate in at least one survey regarding hand therapy practice. Surveys are commonly used to assess current best practice techniques or most common treatments for specific injuries. Yet, we may not be aware of the importance and influence survey results can have in establishing health care policy and national practice guidelines. Surveys are becoming a more common research tool; the Journal of Hand Surgery publishes on average three articles involving surveys in every issue. This current article examines the use of surveys, and provides guidelines for incorporating surveys into research projects. Survey design and administration is more complex than it may appear. Successful and properly conducted surveys are designed with rigorous and complex standards. Many factors are considered in survey design and administration including topic identification, hypothesis generation, survey design, sampling, survey administration, data analysis, and ethical considerations. More specialized studies may require additional stages and/or follow-up interviews. The authors use a previously published national survey on resident's decisions to pursue a career in hand surgery to illustrate the key concepts of survey design. The authors elaborate on the various factors involved in survey design. Topic identification is the first stage, in which the researcher identifies the main question to be answered by the survey. The authors maintain that surveys should ask important and relevant questions based on studies of existing data. Surveys deemed unimportant will not draw good responses from the intended population. Surveys are used to determine a relationship based on a hypothesis. The development of a hypothesis should follow standard clinical research practice: literature review, focus group discussion, and/or leadership opinions. Survey design is the most critical step of survey research. The researcher ultimately wants to collect data; therefore, he must assure that the survey is both reliable and valid so that the data collected will be relevant. A survey's reliability refers to the ability to reproduce similar data. Reliability in a survey is enhanced by asking appropriately worded and specific questions, making sure questions are phrased both negatively and positively, using easily understandable phrases, and incorporating very specific answer scales. The validity of a survey refers to the accuracy of the collected data. There are many factors that negatively influence validity. The authors suggest several methods to improve validity of the responses: write questions at a fifth grade reading level, pilot test the survey, assure anonymity, and use appropriate measurement scales. They caution that retrospective studies should be designed carefully as participants may not accurately remember important information. Low response rate may also decrease the validity of a survey. Several ideas are presented to maximize response rates including sending a cover letter prior to the survey, sending a second survey several weeks after the survey, and incentive programs. Selection of the target population is the next step. Inclusion criteria help define the target population by specifying who will be included in the survey and who will not be included. Sampling is useful in collecting data on a large population group. If the sample is a fair representative of the target population, then the data collected and the survey's findings can be generalized to the larger population. The authors further elaborate on constructing a representative sample of the target population. The Internet has opened up a new method of survey administration. Internet-based surveys are now common along with phone surveys, mail surveys, and face-to-face interviews. Internet-based surveys appear to have the multiple advantages over the other methods, including faster response time, anonymity of respondents, decreased costs, and ability to store data from multiple locations. An experienced statistician as in all research projects should perform statistical analysis of the collected data. Ideally, the statistician is involved early on in the survey design and helps to format the data collection process. Ethical considerations are always warranted when dealing with human research and informed consent for all participants must be obtained. Surveys can be misleading if inappropriate questions are asked and if survey results are erroneously compared with other survey results. Surveys have limitations as well that may have nothing to do with the survey design. They may be administered at the wrong time of year when the respondents cannot take time to answer. The same population may be the focus of several surveys at the same time, in which case none of the surveys gets the attention that it requires. The authors caution that pilot testing of the questions and large sample sizes can help to reduce some of the limitations of survey research. This article stresses the importance of solid survey construction and design. A checklist of all mandatory steps and stages is provided. Practical information and multiple suggestions offer structure to the survey process. A number of surveys have recently appeared in the Journal of Hand Therapy. As clinicians strive for increased evidence-based practice, more surveys questioning our treatment techniques and assessment tools are sure to appear. The task of creating a research study may seem daunting to the novice researcher. Hand therapists will find this an excellent reference for guiding a prospective researcher through a survey research project. The majority of members of the American Society of Hand Therapy have been asked to participate in at least one survey regarding hand therapy practice. Surveys are commonly used to assess current best practice techniques or most common treatments for specific injuries. Yet, we may not be aware of the importance and influence survey results can have in establishing health care policy and national practice guidelines. Surveys are becoming a more common research tool; the Journal of Hand Surgery publishes on average three articles involving surveys in every issue. This current article examines the use of surveys, and provides guidelines for incorporating surveys into research projects. Survey design and administration is more complex than it may appear. Successful and properly conducted surveys are designed with rigorous and complex standards. Many factors are considered in survey design and administration including topic identification, hypothesis generation, survey design, sampling, survey administration, data analysis, and ethical considerations. More specialized studies may require additional stages and/or follow-up interviews. The authors use a previously published national survey on resident's decisions to pursue a career in hand surgery to illustrate the key concepts of survey design. The authors elaborate on the various factors involved in survey design. Topic identification is the first stage, in which the researcher identifies the main question to be answered by the survey. The authors maintain that surveys should ask important and relevant questions based on studies of existing data. Surveys deemed unimportant will not draw good responses from the intended population. Surveys are used to determine a relationship based on a hypothesis. The development of a hypothesis should follow standard clinical research practice: literature review, focus group discussion, and/or leadership opinions. Survey design is the most critical step of survey research. The researcher ultimately wants to collect data; therefore, he must assure that the survey is both reliable and valid so that the data collected will be relevant. A survey's reliability refers to the ability to reproduce similar data. Reliability in a survey is enhanced by asking appropriately worded and specific questions, making sure questions are phrased both negatively and positively, using easily understandable phrases, and incorporating very specific answer scales. The validity of a survey refers to the accuracy of the collected data. There are many factors that negatively influence validity. The authors suggest several methods to improve validity of the responses: write questions at a fifth grade reading level, pilot test the survey, assure anonymity, and use appropriate measurement scales. They caution that retrospective studies should be designed carefully as participants may not accurately remember important information. Low response rate may also decrease the validity of a survey. Several ideas are presented to maximize response rates including sending a cover letter prior to the survey, sending a second survey several weeks after the survey, and incentive programs. Selection of the target population is the next step. Inclusion criteria help define the target population by specifying who will be included in the survey and who will not be included. Sampling is useful in collecting data on a large population group. If the sample is a fair representative of the target population, then the data collected and the survey's findings can be generalized to the larger population. The authors further elaborate on constructing a representative sample of the target population. The Internet has opened up a new method of survey administration. Internet-based surveys are now common along with phone surveys, mail surveys, and face-to-face interviews. Internet-based surveys appear to have the multiple advantages over the other methods, including faster response time, anonymity of respondents, decreased costs, and ability to store data from multiple locations. An experienced statistician as in all research projects should perform statistical analysis of the collected data. Ideally, the statistician is involved early on in the survey design and helps to format the data collection process. Ethical considerations are always warranted when dealing with human research and informed consent for all participants must be obtained. Surveys can be misleading if inappropriate questions are asked and if survey results are erroneously compared with other survey results. Surveys have limitations as well that may have nothing to do with the survey design. They may be administered at the wrong time of year when the respondents cannot take time to answer. The same population may be the focus of several surveys at the same time, in which case none of the surveys gets the attention that it requires. The authors caution that pilot testing of the questions and large sample sizes can help to reduce some of the limitations of survey research. This article stresses the importance of solid survey construction and design. A checklist of all mandatory steps and stages is provided. Practical information and multiple suggestions offer structure to the survey process. A number of surveys have recently appeared in the Journal of Hand Therapy. As clinicians strive for increased evidence-based practice, more surveys questioning our treatment techniques and assessment tools are sure to appear. The task of creating a research study may seem daunting to the novice researcher. Hand therapists will find this an excellent reference for guiding a prospective researcher through a survey research project.