Identifying and Controlling Biases in Expert-Opinion Research: Guidelines for Variations of Delphi, Nominal Group Technique, and Focus Groups

Abstract

In construction engineering and management (CEM) research, conducting field studies often is infeasible because of resource constraints, limited access to sites, practicality, confounding factors, and ethical limitations. Thus, researchers rely on the collection and analysis of expert opinions as an alternative method. Delphi, the nominal group technique, and focus groups often are used to solicit opinions through different processes and controls. Controls implemented during data collection and analysis techniques such as anonymity, multiple rounds, and controlled feedback are used to decrease cognitive and social biases that threaten the validity and reliability of the results. Although there are standard processes for these methods, researchers commonly make modifications to balance research constraints with the study objectives, and it often is unclear how specific modifications promote or degrade the validity and reliability of the results. This paper comprehensively reviewed the existing literature on expert opinion–based studies and argues that seemingly innocuous changes to the traditional methodological frameworks can introduce cognitive biases. A novel conceptual decision-making framework is presented to assist the research community with the development of rigorous experimental designs and transparent interpretation of results when various permutations of key controls are included or omitted in the execution of expert-opinion studies.