Abstract
Bucket testing is a common practice in the internet industry, where new features and services are tested by exposing them to a randomly selected small subset of users. However, in this simple version of bucket testing, since a very small fraction of the total users are selected through uniform independent sampling of the population, the samples chosen, at times, do not adequately serve as a reasonable statistical proxy for the total population. This may lead to erroneous interpretation of the bucket testing results, particularly for online sites having large audiences with varying demographics and preferences. In this work, we present a novel algorithmic framework that addresses this challenge and provides an efficient and more accurate interpretation of the bucket testing results by analyzing the big audience data and factoring in the nature of the overall population in terms of the different user attributes. We demonstrate the effectiveness of our algorithm through the data obtained from real experiments conducted on Yahoo's bucket testing platform.
Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
