Efficient crowdsourced best objects finding via superiority probability based ordering for decision support systems

Abstract

Best objects finding is a fundamental operation in decision support systems and applications. When numerical values of objects cannot be obtained from existing computer systems or in a machine learning manner, crowdsourcing proves a viable approach via harnessing human intelligence for data gathering. Most of existing studies ask crowds to submit pairwise preferences where a large number of crowdsourced questions are produced, thereby incurring huge monetary cost and long latency. To address this issue, we propose a framework for efficient best objects computation by leveraging crowdsourcing to provide object values. The framework employs three query operators ( i.e., top-k, knn, and skyline queries) to compute best objects, and minimizes the number of crowdsourced objects by eagerly pruning non-result objects via superiority probability based ordering. We first propose the concept of superiority probability, which describes the probability that an object is better than or equal to another object from the perspective of statistics. We then explore properties for objects pruning, and propose sequential and parallel ordering techniques for objects crowdsourcing based on the concept of superiority probability. Extensive experimental results show that the proposed framework achieves the promising efficiency in reducing the number of crowdsourced objects and latency.