Biterm topic modelling of responses to open-ended questions: A study with US consumers about vertical farming

Abstract

The aim of the present work was to explore the potential of Biterm Topic Modelling (BTM) for analysing responses to open-ended survey questions in the pursuit of consumer insights. The selection of BTM was based on its suitability to the analysis of short texts compared to other approaches such as Latent Dirichlet Allocation. The empirical context was vertical farming (VF), which participants read a short text about and then answered a question about interest in buying fruits and vegetables from VF. The data came from US consumers (n = 1803) who provided answers with a median length of 9 words. Results were promising about the potential of BTM to uncover latent topics from responses to open-ended questions. The models predicted coherent and consistent topics relating to VF, including buying VF produce or not and year-round fruit and vegetable supply. Two variants of the text about VF were used (n ∼ 900 per variant), and BTM models trained on responses to the variant which emphasised the possibility of using genetic engineering in VF consistently predicted topics related to genetic engineering. Models trained on responses to the variant which emphasized urban production and reduction of carbon emissions linked to product transportation, consistently revealed topics associated to production, environment, and local farming. The major limitation was not taking account of sentiment in consumers’ responses, an issue that BTM shares with other topic models. This is a significant hindrance in the efforts to rely on machine learning for consumer insights.