Abstract
ABSTRACTOpen-ended responses are widely used in market research studies. Processing of such responses requires labour-intensive human coding. This paper focuses on unsupervised topic models and tests their ability to automate the analysis of open-ended responses. Since state-of-the-art topic models struggle with the shortness of open-ended responses, the paper considers three novel short text topic models: Latent Feature Latent Dirichlet Allocation, Biterm Topic Model and Word Network Topic Model. The models are fitted and evaluated on a set of real-world open-ended responses provided by a market research company. Multiple components such as topic coherence and document classification are quantitatively and qualitatively evaluated to appraise whether topic models can replace human coding. The results suggest that topic models are a viable alternative for open-ended response coding. However, their usefulness is limited when a correct one-to-one mapping of responses and topics or the exact topic distribution is needed.
Highlights
Surveys are a pivotal research instrument to gain insight into a study subject
Using a set of real-world OE responses from a market research company, this study explores the potential of three short text topic models for OE responses and compares them to Latent Dirichlet Allocation (LDA) as a benchmark: Latent Feature LDA (LFLDA) (Nguyen et al, 2015), Biterm Topic Model (BTM) (Yan et al, 2013) and Word Network Topic Model (WNTM) (Zuo et al, 2016)
Given the scarcity of prior work dedicated to topic modelling from OE responses, we suggest that the application of a text standard preprocessing pipeline is suitable for this paper
Summary
Surveys are a pivotal research instrument to gain insight into a study subject. In market research, for example, surveys facilitate eliciting the opinions, attitudes, and preferences of consumers and provide critical insights for product development and business process management. Open-ended (OE) questions are a crucial component of surveys They are used to clarify ambiguities and identify opinions that researchers have not thought of before (Lazarsfeld, 1935; Roberts et al, 2014; Schuman, 1966). OE questions provide an opportunity to elicit a subject even if a research lacks sufficient knowledge about the topic to define a closed question (Converse, Jean McDonnell, & Presser, 1986). Another advantage of OE questions compared to closed questions is the ability to detect spontaneous thoughts and explore attitudes. Common use cases of OE questions in market research include measuring the awareness and recall of brands, attitudes towards a product, or activity as well as likes and dislikes among consumers (Brace, 2018)
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
