1.4.5 A Systems Approach to Open‐Inquiry Evaluation of Engineering Programmes

Abstract

AbstractThe problem of how to evaluate engineering programmes is considered. Engineering organisations are under pressure to improve the performance of their programmes, as failures in technological projects receive increasing criticism, while the demand remains for increasingly complex projects to be undertaken. Many engineering organisations have realised that the success of such programmes depends more on the management and related programme issues and less on the engineering details. Evaluation of engineering programmes is demanded, yet evaluation techniques from ‘soft’ (non‐engineering) domains with little understanding of engineering values and culture may be quickly dismissed as ‘not sufficiently quantitative’ and ‘without a systems (engineering) basis‘.Programme evaluation in the human services field is well established, and the types of programme evaluations are outlined. However, engineering programmes present a complex problem space for the evaluator, and require an evaluation method capable of accounting for the values and culture of engineering stakeholders.This paper outlines the systems approach for solving this problem of engineering programme evaluation, on the assumption that the engineering programmes are sufficiently complex that specialist evaluators from ‘soft’ domains conduct the evaluation. An evaluation model is proposed, based upon the open inquiry evaluation method used in human services programme evaluation, superimposed on the proven process used in the engineering discipline of Test and Evaluation (T&E) to validate ‘hard’ products of engineering programmes. With this model, it is suggested that an evaluation technique is easily understood and appreciated by the critical reference group in technology‐intensive domains.