Measuring the unmeasured in educational programs: filling in the blanks through evaluation

Abstract

The indicators of performance put forward as measures of achievement at a state and territory level in Australia reflect to some extent the priorities of those jurisdictions. These are revealed in the annual reports of departments, usually under headings of targets and corresponding outcomes. It may seem reasonable to assume that these performance measures line up with stated objectives, and with what matters on the ground. But do they? This paper argues that while the aims of education are broad, the measures of education are narrow. Philosophically, a good education is one that has social, developmental, intellectual and economic aims. However, even though some of those broad aims are reflected in national, state and territory foundational documents and reports, they are not reflected in Australian measurement or reporting frameworks, which seem to suggest that it is neither practical nor cost-effective to collect such data. The purpose of this paper is to demonstrate that it is possible to measure educational success in ways that support the broader goals of education and schooling in Australia. To this end, the paper draws on two evaluation case studies in the field of education (run within schools), to highlight ways that the unmeasured aspects of educational activity can be measured. One case highlights the significance of social capital in a school-family partnership program, and the other demonstrates the psycho-social benefits of an alternative education program for ‘at risk’ children. These cases illustrate what can be measured and they provide useful data to fill in the blanks of what is not measured at a system level. However, the cases also raise bigger questions about what should be measured and reported as indicators of what matters to educational stakeholders.