Geographic Information System mapping as a tool to assess nonresponse bias in survey research

Abstract

Background Surveys are a useful tool for assessing professional practice patterns, although declining response rates have caused concern over external validity. This is particularly relevant to Web-based surveys, where response rates traditionally have been lower than with paper mail surveys. In a 2005 survey of North Carolina community pharmacy managers using a Web-based data collection instrument, we achieved an overall response rate of 23%. Objective To explore nonresponse bias using accepted methods and to test whether Geographic Information System mapping is a useful tool for assessing response bias. Methods Cross-sectional survey of 1593 community pharmacy managers in North Carolina using a Web-based tool. Nonresponse bias was assessed quantitatively by comparing early responders with late responders (ie, wave analysis) and by comparing respondents with nonrespondents with regard to known pharmacy, pharmacist, and population characteristics. Significant variables from these analyses were then mapped using ArcGIS 9.1. Results Pharmacy type was identified as a predictor of response, with independent pharmacies less likely to respond than chain pharmacies (odds ratio 0.75; 95% confidence interval 0.59-0.95). This conclusion was consistent in the wave analysis and the analysis of known population characteristics. Other county-level variables such as the number of physicians per capita, income, and the percentage of residents eligible for Medicaid showed trends but were not statistically significant ( P < .1). Geographic Information System mapping was able to descriptively illustrate nonresponse bias for pharmacy type but trends were more difficult to detect for statistically insignificant trends. Conclusion The best way to avoid nonresponse bias is to improve response rates. When this is not possible, Geographic Information System mapping has some utility for assessing nonresponse bias, and for aggregating known population characteristics based on location. It is most useful in conjunction with other accepted techniques such as wave analysis and analysis of known population characteristics.