Performing an Inductive Thematic Analysis of Semi-Structured Interviews With a Large Language Model: An Exploration and Provocation on the Limits of the Approach

Abstract

Large Language Models (LLMs) have emerged as powerful generative Artificial Intelligence solutions. This paper presents results and reflections of an experiment done with the LLM GPT 3.5-Turbo to perform an inductive Thematic Analysis (TA). Previous research has worked on conducting deductive analysis. Thematic Analysis is a qualitative method for analysis commonly used in social sciences and it is based on interpretations by the human analyst(s) and the identification of explicit and latent meanings in qualitative data. The paper presents the motivations for attempting this analysis; it reflects on how the six phases to a TA proposed by Braun and Clarke can partially be reproduced with the LLM and it reflects on what are the model’s outputs. The paper uses two datasets of open access semi-structured interviews, previously analysed by other researchers. The first dataset contains interviews with videogame players, and the second is a dataset of interviews with lecturers teaching data science in a University. This paper used the analyses previously conducted on these datasets to compare with the results produced by the LLM. The results show that the model can infer most of the main themes from previous research. This shows that using LLMs to perform an inductive TA is viable and offers a good degree of validity. The discussion offers some recommendations for working with LLMs in qualitative analysis.