Observing the observer: non-intrusive verbalisations using the Concurrent Observer Narrative Technique

Abstract

This paper presents the application of, and a discussion about, a new method of information acquisition, the Concurrent Observer Narrative Technique (CONT), that requires an expert to narrate the work of experienced colleagues. The method is explored through a case study of instructors in a training simulation environment. The method involves multiple expert users of the system equally matched in experience; one or more of which perform the task at hand, namely running the training session, whilst another, not engaged in the training activity, provides verbal reports whilst observing task progression. All are situated in the work environment concurrently. The method is discussed in terms of the richness and detail of the data obtained from the reports and in terms of some advantages and disadvantages over other verbal reporting methods, specifically collegial verbalisation (Jansson et al. in Cogn Tech Work 8:41–49, 2006) and Elicitation By Critiquing (Miller et al. in Cogn Tech Work 8: 90–102, 2006). It is argued that the CONT provides a relatively easy, inexpensive, and non-intrusive method that can supply a wealth of information that does not require further interpretation and that the method is of particular applicability in domains where the subject matter expert is unable to provide concurrent verbalisations themselves and where the subject matter is too complex to be understood by a lay person.