Research in engineering design: the role of mathematical theory and empirical evidence

Abstract

The editors of this journal have offered an opportunity toreply to Dr. Hazelrigg’s letter in depth. Indeed, with itsnumerous points of critique of the paper ‘‘The Pugh Con-trolled Convergence method’’ (Frey et al. 2009) stated sostrongly, the letter demands a detailed rebuttal. We providea response to the specific points discussed in the letter aswell as the broader issues raised. Writing on these topicshas been an opportunity to explore some issues of interestto us, including the role of mathematical theory andempirical science in design research. To pursue this fully,additional authors participated to add more varied expertiseon social sciences, preference measurement, and industrypractices. We hope that our response will do more thandefend the paper; we hope that it will also suggest someconstructive paths forward in design research.1 The main point: interaction between analysis andsynthesisTo show that a wrong problem is being solved –wrong in the sense that it is not the empirically givenone – is the first ground for rejecting a theory: amatter of irrelevance. A second basis for rejectionwould be to show that improper, inadequate, oroversimplifying assumptions have been made (Mor-genstern 1972).Research in engineering design, like all science, benefitsfrom active critique based on both theory and empiricaldata. Hazelrigg’s letter in effect asserts a veto power of hispreferred mathematical theory over empirical evidence.For example, he writes ‘‘the reader of this or any other suchpaper should never merely assume that it is correct, butverify its validity through personal derivation’’. We agreethat readers should never assume any particular publicationis correct but disagree that personal derivation is anappropriate procedure in this context. If a paper presentsdata inconsistent with the hypotheses of a reader, a math-ematical derivation will not give an adequate justificationto ignore the data. A more appropriate procedure is tocheck the data for accuracy at their source, by replicationof an experiment or by seeking data from other relevantrecords. If the data hold up to review, the deductiveframework of the reader may need to be revised; forexample, by changing its premises or by broadening theframework to incorporate more considerations. In theevaluation of Pugh Controlled Convergence, Hazelrigg’spreferred mathematical framework suggests it will fail, butthe evidence from practice indicates it does not. We submitthat Hazelrigg’s mathematical framework makes improper,inadequate, or oversimplifying assumptions.The primary point of the paper ‘‘The Pugh ControlledConvergence method: model-based evaluation and impli-cations for design theory’’ (Frey et al. 2009) is that deci-sion-making (analysis) and alternative generation(synthesis) have significant interactions that should bemodeled if one is to evaluate design methodologies. Thepaper discusses a documented case study (Khan and Smith