Bar raising or Navel-Gazing?:the Effectiveness of Self-Study Programmes in eading to Improvements in Institutional Performance

Abstract

Higher Education Institutes worldwide are investing significant resources in self study programmes to improve institutional performance, to enhance quality and to meet external stakeholder demands. The institutional impacts of both internally and externally mandated self study programmes is an area where little empirical research exists. A key question is whether self study programmes are effective (or otherwise) in leading to improvements in institutional performance and the reasons why. Covering an eight year period, from 1997–2006, this paper reports on the use of systematic evaluation methodology (Rossi et al., 2003) to evaluate the effectiveness of three self study programmes in leading to improvements in institutional performance. The impact of the programmes was assessed in two ways: (i) the degree to which the programme met its stated goals and objectives (including peer review panel recommendations) and (ii) other impacts (intended or otherwise). The methods of inquiry used were primarily qualitative and the main data sources were proceedings of the main decision making fora in the Institute, Institute publications and interviews with n=17 key informants. Introduction and context National quality assurance agencies, almost unheard of 20 years ago, are now in place in almost all OECD countries (OECD, 2003). Stensaker notes that while there are a growing number of studies on quality assurance, there is a lack of research on the impact of quality assurance at institutional level (Stensaker, 2007). Stensaker cites methodological issues surrounding assessing the impact of quality assurance processes as a major challenge (Stensaker, 2007). Harvey and Newton note that establishing definitive causal links and isolating their effects from other factors is a difficult task (Harvey and Newton, 2004). In the broader public service arena Pollitt and Boukaert note the dearth of empirical studies which tackle the issue of evidence of improvements in effectiveness as a result of management reforms (Pollitt and Bouckaert 2004). Birnbaum states that there are ‘few published examples in the academic sector of attempts to assess the institutional consequences of a management fad through data that provide evidence either of organizational outcomes or of the satisfaction of users’ (Birnbaum 2000). The fourteen Irish Institutes of Technology are part of a binary system in Ireland and account for slightly more than half of all enrolments to third level. Although an identical model to the IOTs does not exist elsewhere they exhibit some similarities with the Finnish Polytechnics, Dutch HBOs, French IUTs, German FHS and the Institutes of Technology in New Zealand. The Institutes operate under the auspices of the Higher Education and Training Awards Council of Ireland (HETAC) for quality assurance