P32 Five Tips for Large-Scale Assessment of School Wellness Policies Using the WellSAT Scoring System

Abstract

Background The AZ Health Zone administers Arizona's SNAP-Ed program. Our multi-year, statewide evaluations follow national evaluation standards, including accuracy and utility. The Rudd Center for Food Policy & Obesity developed the WellSAT to measure the quality of written SWPs, in consultation with an advisory board. The WellSAT has been updated twice to reflect changes to national regulations from the Healthy, Hunger-Free Kids Act. Objective To identify key characteristics of an accurate, useful system for large-scale SWP assessment. Study Design, Settings, Participants From October 2015—September 2018, the AZ Health Zone State Evaluation Team used a WellSAT scoring process to assess the quality of written school wellness policies (SWPs) from over 120 SNAP-Ed-qualified districts across Arizona's 15 counties. Local SNAP-Ed agencies submitted SWPs for WellSAT scoring and received back results and recommendations to use with districts. Measurable Outcome/Analysis We conducted a meta-evaluation to improve the WellSAT scoring process. Data from SWP meetings, evaluation meetings, debrief sessions, and interviews with Local SNAP-Ed Agencies were systematically analyzed to identify best practices for large-scale SWP assessment using the WellSAT. Results We identified five key characteristics of our scoring system. Use two consistent scorers. We enhanced accuracy by comparing results from two trained, independent scorers to catch errors and resolve discrepancies. Train scorers using state-specific guidelines. We further improved accuracy by developing a state-specific scoring template. Aim for rapid, responsive turnaround. We bolstered the utility of findings by generating results within four to eight weeks, or before district SWP meetings. Offer guidance on interpreting scores. We also enhanced utility by helping users understand how their scores compared to state averages and national trends. Create easy-to-use recommendations. We improved utility by providing users with editable, score-based recommendations on how to revise SWP language. Conclusion The accuracy and utility of large-scale SWP assessment can be enhanced using the five practices described above.