Quantitative and qualitative analysis of the expert and non-expert opinion in fire risk in buildings

Abstract

Expert judgment procedure is a method very often used in the area of risk assessments of complex systems or processes to fill in quantitative data. Although it has been proved to be a very reliable source of information when no other data are available, the choice of experts is always questioned. When the available data are limited, the seed questions cover only partially the domains of expertise, which may cause problems. Expertise is assessed not covering the full object of study but only those topics for which seed questions can be formulated. The commonly used quantitative analysis of an expert judgment exercise is combined with a qualitative analysis. The latter adds more insights to the relation between the assessor's field and statistical knowledge and their performance in an expert judgment. In addition the qualitative analysis identifies different types of seed questions. Three groups of assessors with different levels of statistical and domain knowledge are studied. The quantitative analysis shows no differences between field experts and non-experts and no differences between having advanced statistical knowledge or not. The qualitative analysis supports these findings. In addition it is found that especially technical questions are answered with larger intervals. Precaution is required when using seed questions for which the real value can be calculated, which was the case for one of the seed questions.