Instance redistribution-based label integration for crowdsourcing

Abstract

Crowdsourcing provides an inexpensive solution to employ crowd workers labeling instances, and hence each instance is labeled with a multiple noisy label set instead of its true label. Label integration aims to infer a true label from its multiple noisy label set. However, existing fine-tuned methods increase the model complexity while enhancing the performance of label integration. Therefore, it is a challenge to search for a simple but efficient label integration method. Meeting this challenge, this paper proposes instance redistribution-based label integration (IRLI), a novel method for label integration. Firstly, from the label view, we estimate the probability of an instance belonging to each class based on its multiple noisy label set and obtain a multiple noisy label distribution vector. Secondly, from the attribute view, we reestimate the probability of an instance belonging to each class based on its attribute set and obtain an instance redistribution vector. Finally, we combine the multiple noisy label distribution vector and the instance redistribution vector of each instance to infer its integrated label. Comprehensive experimental results on both simulated and real-world crowdsourced datasets validate the superiority of IRLI.