User-level incremental conversion ranking without A/B testing

Abstract

The traditional approach to build incremental conversion prediction model has to rely on A/B test results. However, under certain intense business environment, A/B testing can be limited by technical support, platform, or budget, and become not practically available. In this paper, we propose an algorithm to rank users by incremental conversions resulted from advertising effects, which is based on user’s conversion history and the output from a conversion prediction model. By appropriately defining who an active user is, this algorithm is proven to work well with real data. In case where an A/B test is not available and incremental conversion-based user targeting is desired, this algorithm offers a practical solution.