A worker clustering-based approach of label aggregation under the belief function theory

Abstract

Crowdsourcing platforms have been attracting a wide attention in the field of artificial intelligence in recent years, providing a cheap and reachable human-powered resource to gather massive labeled data. These data are used to effectively build supervised learning models for academic research puposes. However, despite the attractiveness of these systems, the major concern has always been the quality of the collected labels. In fact, a wide range of workers contributes in labeling data leading to be in possession of potentially noisy and imperfect labels. Therefore in this paper, we propose a new label aggregation technique that allows to determine workers qualities via a clustering process and then represent and combine their labels to estimate the final one under the belief function theory. This latter is notorious for its strength and flexibility when dealing with imperfect information. Experimental results demonstrate that our proposed method outperforms the related work baseline and improves results quality.