The Use of Qualitative Data Analysis for the Evaluation of Design Ethnography Training Among Undergraduate Engineering Students

Abstract

Abstract While qualitative data analysis (QDA) is an established method in education research, QDA is less common in engineering research and may be a challenge for engineering faculty not formally trained in qualitative methods to apply it in engineering education. The following describes the collaborative effort between an engineering design instructor and an anthropologist who used QDA to evaluate the implementation of design ethnography training in a third-year biomedical engineering design course. In their partnership, the study investigators examined student perspectives regarding design ethnography training and how such training in an engineering curriculum may prepare students for careers in biomedical design. Data for the study consisted of reflective essays (N = 42) that the students completed following two primary exercises dedicated to design ethnography skills training. Investigators input typed and anonymized text files of the student essays into ATLAS.ti X7, a qualitative data analysis software program, for qualitative content analysis. QDA was conducted using the constant comparison method to inductively identify pertinent themes. Throughout the QDA process, the investigators routinely met to discuss, merge and interpret themes as needed. Upon the finalization of themes, researchers re-reviewed the data using the finalized codebook (a list of themes and their definitions) for coding reliability. This regular contact was invaluable for the engineering instructor, providing instruction on the process necessary for proper application of QDA. The unique partnership between investigators offered the engineering design instructor the opportunity to evaluate engineering student perceptions of a new curriculum implementation in an in-depth manner not commonly attempted in engineering education. Results from the QDA showed that the incorporation of design ethnography skills training into an engineering design curriculum increased student awareness of the value of ethnography in understanding user environments while offering engineering students the opportunity to develop better observation skills. This study was successful not only in demonstrating efficacy of design ethnography training among undergraduate engineering students, but it also serves as an example of how QDA may be applied by engineering instructors for the evaluation of student experience and work in engineering education.