Content and Context: Identifying the Impact of Qualitative Information on Consumer Choice

Abstract

Managers and researchers alike suspect that the vast amounts of qualitative information found in blogs, product reviews, real estate listings, news stories, analyst reports and experts’ advice influence consumer behavior. But, do these kinds of qualitative information impact or rather reflect consumer choices? We argue that message content and consumer choice are endogenous, and that non-random selection and the conflation of awareness and persuasion complicate causal estimation of the impact of message content on economic decisions and outcomes. Using data on the transcribed content of 2,397 stock recommendations provided by Jim Cramer on his CNBC show Mad Money from 2005 to 2008, combined with data on Internet search volume, the content of prior news, and prior stock price and trading volume data, we show that selection bias in the stocks Cramer chooses to recommend and prior product awareness on the part of his audience create measurable upward bias in estimates of the impact of Cramer’s advice on stock prices. Using Latent Dirichlet Allocation (LDA) to characterize the topical content of Cramer’s speech and the content of prior news, we show that he is less persuasive when he supports his recommendations with arguments that have themselves been recently mentioned in the news. We argue that the classic sales skill of “knowing what a customer needs to hear” can significantly enhance the influence of qualitative information precisely because what the consumer already knows affects how they evaluate messages. The tools and techniques we develop can be put to practical use in a variety of settings where marketers can present subjects with different messages depending on what they already know.