Development of an Evaluation Scale to Measure Participant Perceptions of After-School Enrichment Programs

Abstract

The purpose of this study was to develop the Scale for After-School Enrichment Programs (SASEP) to assess the quality of after-school programs by participants. This study was completed over a 3-year period and through 3 phases: (a) formulation of the preliminary scale, (b) an exploratory factor analysis (EFA), and (c) a confirmatory factor analysis (CFA). During Year 1, the preliminary SASEP with 27 items with 3-point Likert response scales was formulated through a review of literature, open-ended surveys of program participants in 2 urban after-school programs (N = 754), and test of content validity. During the following 2 years, program participants (N = 4,091) responded to the preliminary scale at the end of each annual program. The data set was randomly split into two halves: one for EFA and the other for CFA. Using the criteria of an eigenvalue equal to or greater than 1.0 and a factor loading equal to or greater than .30 without double loading in the EFA with alpha extraction and promax rotation, 3 factors emerged with all 27 items retained: Personal and Social Benefits (11 items), School and Community Benefits (9 items), and Overall Program Benefits (7 items). These factors were consistent with the general program objectives. Based on the weighted least squares (WLS) estimation method in CFA, the 3-factor model was confirmed as providing good fit to the data. Discussions were undertaken in the context of the scale development and its application potential in after-school program evaluations.