Evidence for validity of five secondary data sources for enumerating retail food outlets in seven American Indian Communities in North Carolina

Sheila E Fleischhacker,Dolly Soto,Amanda Henley,Gowri Ramachandran,Kelly R Evenson,Ziya Gizlice,Daniel A Rodriguez

doi:10.1186/1479-5868-9-137

Abstract

BackgroundMost studies on the local food environment have used secondary sources to describe the food environment, such as government food registries or commercial listings (e.g., Reference USA). Most of the studies exploring evidence for validity of secondary retail food data have used on-site verification and have not conducted analysis by data source (e.g., sensitivity of Reference USA) or by food outlet type (e.g., sensitivity of Reference USA for convenience stores). Few studies have explored the food environment in American Indian communities. To advance the science on measuring the food environment, we conducted direct, on-site observations of a wide range of food outlets in multiple American Indian communities, without a list guiding the field observations, and then compared our findings to several types of secondary data.MethodsFood outlets located within seven State Designated Tribal Statistical Areas in North Carolina (NC) were gathered from online Yellow Pages, Reference USA, Dun & Bradstreet, local health departments, and the NC Department of Agriculture and Consumer Services. All TIGER/Line 2009 roads (>1,500 miles) were driven in six of the more rural tribal areas and, for the largest tribe, all roads in two of its cities were driven. Sensitivity, positive predictive value, concordance, and kappa statistics were calculated to compare secondary data sources to primary data.Results699 food outlets were identified during primary data collection. Match rate for primary data and secondary data differed by type of food outlet observed, with the highest match rates found for grocery stores (97%), general merchandise stores (96%), and restaurants (91%). Reference USA exhibited almost perfect sensitivity (0.89). Local health department data had substantial sensitivity (0.66) and was almost perfect when focusing only on restaurants (0.91). Positive predictive value was substantial for Reference USA (0.67) and moderate for local health department data (0.49). Evidence for validity was comparatively lower for Dun & Bradstreet, online Yellow Pages, and the NC Department of Agriculture.ConclusionsSecondary data sources both over- and under-represented the food environment; they were particularly problematic for identifying convenience stores and specialty markets. More attention is needed to improve the validity of existing data sources, especially for rural local food environments.

Highlights

Most studies on the local food environment have used secondary sources to describe the food environment, such as government food registries or commercial listings (e.g., Reference USA)
The average distance for 202 outlets between the latitudes and longitudes taken during primary data collection and compared to the inter-rater reliability team was 32 meters, with a range from 0 to 1418 meters
All of these outlets were in three urban settings and tended to be small convenience stores or grill venues

Summary

Introduction

Most studies on the local food environment have used secondary sources to describe the food environment, such as government food registries or commercial listings (e.g., Reference USA). Precise Global Positioning Systems (GPS) data were not collected in most of the studies [14,15,16,17] and only three used on-site observations of food outlets using GPS without a preconceived notion or list to guide the observations (i.e., “ground-truthing”) [13,22,23]. This ground-truthing approach is considered the gold standard for measuring the food environment since observers are not biased by a list or map of secondary data sources [22,24]

Methods

Results

Discussion

Conclusion