Getting to Why: Semi-Supervised Topic Modeling of Customer Purchase Histories

Abstract

The design and marketing of new products is fundamentally about understanding a customer's underlying needs. In this paper we describe research to learn needs by analyzing a customer's past purchases. We leverage what companies already (believe that they) know about how customers solve problems in the form of guides, instructions, and/or recipes. Using a semi-supervised form of LDA topic modeling, we align customer purchases with lists of equipment and materials that define established solutions for known problems. From known needs and solutions, we seek to discover new ways that customers are solving (un)known problems in the unlabeled data. We evaluate the approach on purchase-data from 18,000 customers of a multi-billion dollar specialty retailer.