A Mixed Method Program Evaluation of Annual Inspections Conducted in Childcare Programs in Washington State

Abstract

This mixed method study used a program evaluation to assess the reliability and social validity of the focused childcare monitoring checklist used in Washington State, as well as its social validity in maintaining quality programming in licensed childcare centers. The focused monitoring checklist and interview responses were used to answer two specific research questions: (1) How do stakeholders describe the value, usefulness, and effects of state administrated focused monitoring?; and (2) What is the inter-rater reliability of the focused monitoring tool used to assess the foundational health and safety issues that must be met by state licensed early childhood programs? The study found that licensors and providers found the focused monitoring tool as more efficient and informative than the current differential monitoring system. The use of a checklist focusing on real time compliance increased the value placed on the relevance of the inspection with respect to meeting licensor and provider needs. The results also showed that even with a controlled tool, performance of onsite inspections can vary greatly along a continuum of reliability and objectivity due to licensor rater drift and individual perceptions of licensing procedures. Licensing agencies should consider further evaluation of the monitoring process and the reliability of the checklist tool as the process is implemented statewide, concentrating on the training content and training methods provided to licensors.
 https://doi.org/10.21423/jrs-v08stevens