Stochastic Optimal Control Models for Online Stores: Indefinite Horizon

Abstract

In this work we propose and analyze a model which addresses the pulsing behavior of sellers in an online auction (online store). This pulsing behavior is observed when sellers switch between advertising and processing states. We assert that a seller switches her state in order to maximize her profit, and further that this switch can be identified through the seller's reputation. We find that for each seller there is an optimal reputation, i.e. the reputation at which the seller should switch her state in order to maximize her total profit. Relying on techniques of stochastic optimal control, we design a stochastic behavioral model for an online seller. This model incorporates the dynamics of resource allocation and reputation. We optimize the design model by using a stochastic advertising model put forth by Sethi (Optimal Control Applications and Methods, vol. 4, no. 2, 1983, pp. 179-184) and used effectively in Raman's more recent work (Automatica, vol. 42, no. 8, 2006, pp. 1357-1362). This model of reputation is combined with the effect of online reputation on sales price empirically verified by by Mink and Seifert (Group Decision and Negotiation International Conference, June 2006, pp. 253-255), and we derive the resulting Hamilton-Jacobi-Bellman (HJB) differential equation. The solution of this HJB equation relates optimal wealth level to a seller's reputation. We formulate both a full model, as well as a reduced model with fewer parameters, both of which have the same qualitative description of optimal seller behavior. Coincidentally, this reduced model has a closed form analytical solution which we also construct. Although the techniques we use are standard, we believe that the optimal strategy presented both analytically and numerically has implications on the way reputational information can be used to predict the behavior of an individual seller in an online market.