Comparison of Expertise for Performing the Video Editing Task from Different CTA Interview Methods

Abstract

Unlike traditional task analysis which is usually conducted with a single subject-matter expert (SME), cognitive task analysis (CTA) utilizes multiple SMEs to elicit experts’ domain knowledge of a target task to verify whether all necessary knowledge and skills were included. Interview is one of the most commonly used techniques for CTA. As previous studies have suggested, CTA interviewers should be familiar not only with the domain to be analyzed, but also know well with the process of CTA. However, the constraints of time and cost cause lack of sufficient preparation for interviewers to gain enough background knowledge of the target domain and to be skilled in CTA. The aim of this study was to provide empirical evidence on how the knowledge elicited from experts is different when interviewers are possessed with different background knowledge of the domain and CTA, and factual evidence on how interview structures make different results of the elicitation of expertise. In this study, by the forms of unstructured or semi-structured, CTA interviews were conducted by a CTA analyst and/or a SME with three experts. Whole processes of the interviews were transcribed, coded based on a CPP framework, and analyzed. The results showed that the semi-structured interview conducted by both SME and CTA analyst elicited more knowledge from experts than other interview methods.