Comparing Food Security Before and During the COVID-19 Pandemic: Considerations When Choosing Measures

Abstract

Food insecurity is a public health issue in the United States. Over the past 25 years, national prevalence of food insecurity—encompassing both low and very low food security—has been consistently over 10%, and very low food security alone has persisted at over 3%.1US Department of AgricultureEconomic Research Service. Key statistics & graphics.https://www.ers.usda.gov/topics/food-nutrition-assistance/food-security-in-the-us/key-statistics-graphics.aspxGoogle Scholar The onset of the COVID-19 pandemic in March 2020 and ensuing economic crisis and disruption of food systems2Kinsey E.W. Kinsey D. Rundle A.G. COVID-19 and food insecurity: An uneven patchwork of responses.J Urban Health. 2020; 97: 332-335Crossref PubMed Scopus (50) Google Scholar have only served to exacerbate its prevalence.3Schanzenbach D.W. Pitts A. How much has food insecurity risen? Evidence from the Census Household Pulse Survey. Institute for Policy Research Rapid Research Report, Published 2020https://www.ipr.northwestern.edu/documents/reports/ipr-rapid-research-reports-pulse-hh-data-10-june-2020.pdfGoogle Scholar In response, the research community has made laudable efforts to track the pandemic’s influence on the prevalence of food insecurity across the country. However, for the sake of expediency, many food-security measures used were inconsistent with validated, prepandemic measures.4US Census BureauHousehold Pulse Survey: Measuring social and economic impacts during the coronavirus pandemic.https://www.census.gov/programs-surveys/household-pulse-survey.htmlGoogle Scholar, 5Bauer L. The COVID-19 crisis has already left too many children hungry in America.Published 2020https://www.hamiltonproject.org/blog/the_covid_19_crisis_has_already_left_too_many_children_hungry_in_americaGoogle Scholar, 6USC Dornsife. USC Center for Economic and Social ResearchUnderstanding coronavirus in America.https://uasdata.usc.edu/index.phpGoogle Scholar, 6Nord M. Effects of the decline in the real value of SNAP benefits from 2009 to 2011. US Department of Agriculture, Published 2013https://www.ers.usda.gov/webdocs/publications/45101/39326_err151.pdf?v=50Google Scholar This incompatibility will likely lead to estimation errors, which in turn will produce inaccurate conclusions resulting in potentially detrimental policy and resource-allocation decisions. This commentary aims to reinforce the necessity of consistent and careful use of validated measures when making comparisons between pre- and postpandemic data to arrive at accurate conclusions and recommendations. Without a doubt, struggles to obtain and afford food during the pandemic have increased dramatically.2Kinsey E.W. Kinsey D. Rundle A.G. COVID-19 and food insecurity: An uneven patchwork of responses.J Urban Health. 2020; 97: 332-335Crossref PubMed Scopus (50) Google Scholar To make proper decisions about solutions, policy makers at every level need to be accurately informed about food-insecurity prevalence in the population. Health care and service providers and community leaders designing rapid response programs to get food resources to patients, clients, neighbors, and other community members also need accurate food-insecurity prevalence estimates. State and federal agencies and public assistance program managers respond to food-insecurity prevalence by increasing staff and infrastructure to manage an increased demand in program participation. Measuring the success of their efforts depends, at least in part, on food-security prevalence estimates from accurate data, enabling comparisons before, during, and—whenever it may be—after the pandemic. Inaccurate data will consequently result in misleading conclusions about who was hardest hit, who needs ongoing help, or whether programs are actually making a difference and may even lead to harmful decisions about program funding or resource allocation and deployment. In the worst case, such a scenario could compound the suffering already sustained in many communities, if, for example, policy makers prematurely decide to roll back or terminate COVID-related programs and benefits before those at the lower end of the income spectrum and most at risk of food insecurity have had a chance to recover.6USC Dornsife. USC Center for Economic and Social ResearchUnderstanding coronavirus in America.https://uasdata.usc.edu/index.phpGoogle Scholar The Household Food Security Survey Module (HFSSM) measures food security and insecurity at the household level. Households composed only of adults respond to 10 questions and those with children under age 18 years respond to an additional 8 questions.7Hamilton W.L. Cook J.T. Thompson W.W. et al.Household food security in the United States in 1995: Summary report of the Food Security Measurement Project. A report provided to USDA/FCS under Contract No. 53-3198-5-028.. Abt Associates Inc, Alexadria, VAPublished September 1997https://fns-prod.azureedge.net/sites/default/files/SUMRPT.PDFDate accessed: February 20, 2021Google Scholar,8US Department of Agriculture Economic Research ServiceUS household food security survey module.https://www.ers.usda.gov/topics/food-nutrition-assistance/food-security-in-the-us/survey-tools/#householdDate: Published September 2012Date accessed: January 25, 2021Google Scholar Households are classified as having high, marginal, low, or very low food security over the 12 months prior to the survey according to the number of affirmative answers. The HFSSM was developed and validated with specific Item Response Theory psychometrics that confirmed its validity and reliability. This household-level scale, with a recall period of 12 months, is considered the most reliable scale and has been used since its development as the reference measure in the United States and as the basis for other countries to develop and validate their own national surveys. A shorter 6-item adult version,9US Department of AgricultureEconomic research service. Survey tools.https://www.ers.usda.gov/topics/food-nutrition-assistance/food-security-in-the-us/survey-toolsDate accessed: January 28, 2021Google Scholar a 9-item version for children ages 12+ years,10Connell C.L. Nord M. Lofton K.L. Yadrick K. Food Security of older children can be assessed using a standardized survey instrument.J Nutr. 2004; 134: 2566-2572Crossref PubMed Scopus (114) Google Scholar and a 2-item screen11Hager E.R. Quigg A.M. Black M.M. et al.Development and validity of a 2-item screen to identify families at risk for food insecurity.Pediatrics. 2010; 126: e26-e32Crossref PubMed Scopus (631) Google Scholar were validated against the HFSSM for use in community and clinical settings where the full module is impractical. In addition, a 30-day reference period was validated for these subscales.12United States Department of Agriculture, Economic Research ServiceSurvey tools.https://www.ers.usda.gov/topics/food-nutrition-assistance/food-security-in-the-us/survey-toolsDate accessed: January 29, 2021Google Scholar,13Makelarski J.A. Abramsohn E. Benjamin J.H. Du S. Lindau S.T. Diagnostic accuracy of two food insecurity screeners recommended for use in health care settings.Am J Public Health. 2017; 107: 1812-1817Crossref PubMed Scopus (49) Google Scholar To develop and validate a shorter scale or screen, researchers can use several distinct methodologies to assess its validity, reliability, accuracy, and effectiveness compared with the reference measure. Optimally, the validated measure carries over the metrics with which the reference measure was developed, such as the level of response and recall periods. On March 13, 2020, the US government declared a national emergency concerning COVID-19.14The White HouseProclamation on declaring a national emergency concerning the novel coronavirus disease (COVID-19) outbreak. Issued on March 13, 2020.https://www.whitehouse.gov/presidential-actions/proclamation-declaring-national-emergency-concerning-novel-coronavirus-disease-covid-19-outbreak/#:∼:text=The%20Secretary%20of%20Health%20and,in%20response%20to%20COVID%2D19Date accessed: February 18, 2021Google Scholar To document rapid changes in food insecurity since the COVID-19 pandemic began, researchers initiated both governmental and nongovernmental studies using a variety of scales, questions, and time frames different from the original validated instruments. Some examples are (1) the US Census Bureau Household Pulse Survey (HHPS), which includes the food-sufficiency question from the HFSSM, using a recall period of 7 days4US Census BureauHousehold Pulse Survey: Measuring social and economic impacts during the coronavirus pandemic.https://www.census.gov/programs-surveys/household-pulse-survey.htmlGoogle Scholar; (2) the Hamilton Project in association with the Middle Class Initiative Survey of Mothers with Young Children, which includes 2 questions from the HFSSM, using a time frame from the beginning of the pandemic15NORC at the University of ChicagoCOVID impact survey.https://www.norc.org/Research/Projects/Pages/covid-impact-survey.aspxDate accessed: February 5, 2021Google Scholar; and (3) the University of Southern California Center for Economic and Social Research’s Understanding Coronavirus in America tracking survey, which includes some questions from the HFSSM in modified form and a 7-day recall period.5Bauer L. The COVID-19 crisis has already left too many children hungry in America.Published 2020https://www.hamiltonproject.org/blog/the_covid_19_crisis_has_already_left_too_many_children_hungry_in_americaGoogle Scholar Every effort to measure food insecurity during the pandemic is a positive and laudable initiative. When developing a new survey, the decision to use one food-security measure or another is conditioned on the objective of the given research question, among other factors. In general, new food-security surveys are most helpful if resulting prevalence estimates can be compared with prepandemic food-insecurity prevalence. To do so, researchers should either compare results from the same set of validated questions or make sure adaptations are made appropriately to enable meaningful comparisons. To avoid misinterpretation of data trends, it is essential either to present food-insecurity prevalence during the pandemic only or to use the same set of validated questions to produce food-insecurity trends over time periods that bridge the start of the pandemic. In addition to the HFSSM, the Current Population Survey also includes the single-item household food-sufficiency measure in its December supplement. Despite collecting data on food insufficiency, the US Department of Agriculture does not conduct analysis of data from this question on a regular basis as it does with the HFSSM results, though it recommends—when minimizing respondent burden is a high priority—use of the food-sufficiency question together with income to screen out potential respondents highly unlikely to be food insecure.6USC Dornsife. USC Center for Economic and Social ResearchUnderstanding coronavirus in America.https://uasdata.usc.edu/index.phpGoogle Scholar The food-sufficiency question was included in the HHPS to collect data on food adequacy during the pandemic—before March 13—and in the 7 days prior to date of survey panels from April through October 2020. ∗The Pulse Survey is a Census Bureau experimental data survey administered approximately weekly from April 23 to July 16, 2020, and approximately biweekly from August 19 to October 14, 2020. Survey data are tabulated separately for households with children. Despite being a validated question, prevalence estimates based on this measure cannot be compared directly with prepandemic HFSSM estimates of food-security prevalence for several reasons. First, food insufficiency and food insecurity measure different dimensions of food inadequacy. HFSSM food-security questions address 4 dimensions (anxiety or worry about the adequacy of food supply, quality of food intake, quantity of food intake, and social acceptability of the sources of food), whereas food sufficiency measures at most 2 dimensions (quantity and possibly quality of food intake).7Hamilton W.L. Cook J.T. Thompson W.W. et al.Household food security in the United States in 1995: Summary report of the Food Security Measurement Project. A report provided to USDA/FCS under Contract No. 53-3198-5-028.. Abt Associates Inc, Alexadria, VAPublished September 1997https://fns-prod.azureedge.net/sites/default/files/SUMRPT.PDFDate accessed: February 20, 2021Google Scholar Second, although food sufficiency has 4 levels of response, which could potentially allow a category-by-category comparison with the 4-level HFSSM measure of food security (food secure, marginally food secure, low food security, and very low food security), the conceptual foundations of the measures are quite different. In addition, it is important to note that when dichotomizing households on the food-sufficiency question responses, it is necessary to use 2 categories: “food sufficiency,” when collapsing the 2 less severe answers (enough of the kinds of food we want to eat, and enough but not always the kinds of food we want to eat), and “food insufficiency,” when aggregating the 2 more severe answers (sometimes not enough to eat, and often not enough to eat). Third, the HFSSM was validated at the household level, as was the food-sufficiency question. The dimension of individual-level food security was not included in the measurement framework for either the HFSSM or the food-sufficiency question (unless there is only 1 adult or 1 child in a household). For this reason, it is more acceptable to refer to individuals who live in food-secure or food-insecure households, instead of food-secure or food-insecure individuals, although this difference may seem subtle and is often not observed. This is even more important for children when not using the child-level scale. Thus, prevalence estimates utilizing the food-sufficiency question should use household-level weights, and not individual-level response weights. Fourth, the time reference for the food-sufficiency question used in the December Current Population Survey is 12 months, as opposed to a 7-day recall period, or an indefinite “before March 13, 2020” period used in the HHPS. The HFSSM was validated with 12-month and 30-day recall periods, and these are the only recall periods that would allow comparability between pre- and during-pandemic prevalences. Using the Current Population Survey data, it is possible to observe that prevalence of food insecurity with the 30-day recall period is far below annual rates. In this sense, the prevalence observed with a recall period less than 12 months has a high probability of providing lower food-insecurity prevalence.7Hamilton W.L. Cook J.T. Thompson W.W. et al.Household food security in the United States in 1995: Summary report of the Food Security Measurement Project. A report provided to USDA/FCS under Contract No. 53-3198-5-028.. Abt Associates Inc, Alexadria, VAPublished September 1997https://fns-prod.azureedge.net/sites/default/files/SUMRPT.PDFDate accessed: February 20, 2021Google Scholar Thus, other measures utilizing 30-day, 7-day, or nonspecific prepandemic dates are not directly comparable with the HFSSM prepandemic prevalence estimates based on a 12-month recall period. The rapid response of the research community to ensure that food adequacy is monitored and addressed is commendable. However, if accurate comparisons are to be made between data from prior to the COVID-19 pandemic and data during the pandemic using the food-sufficiency question, or any other short scale or screen, researchers should compare results derived with the same measure during both periods or make adequate efforts to reconcile data differences and achieve consistency across periods. Comparisons using different measures will likely lead to estimation errors arising from variation in level of response (eg, individual versus household level) and/or recall period (eg, 12 months or 30 days versus 7 days, or not determined), or other inconsistencies, and have the harmful potential to lead to inaccurate conclusions and detrimental policy and resource-allocation decisions.