Dual-View Noise Correction for Crowdsourcing

Abstract

In crowdsourcing scenarios, each instance obtains a multiple noisy label set from different crowd workers on the Internet and then gets its integrated label via label integration. Although label integration algorithms are often effective, a certain level of noise still remains in the integrated labels. To reduce the impact of noise on label quality, many noise correction algorithms have been proposed in recent years. However, most of them are hard to fully utilize the joint information of the original attribute view and multiple noisy label view. Motivated by multi-view learning, in this paper, we propose a dual-view noise correction (DVNC) algorithm. Benefiting from the complementary and consensus principle, DVNC can fully utilize the joint information of two views and thus enhance the effect of noise correction. In DVNC, we at first construct the original attribute view and multiple noisy label view for each instance and respectively train a classifier on each view. Then, we use the trained two classifiers to filter each noise instance and thus obtain a clean set and noise set. Finally, we train a classifier on the clean set to correct each instance in the noise set via reclassifying it as the class with the maximum posterior probability. The experimental results on simulated and real-world datasets indicate that DVNC significantly outperforms all the other noise correction algorithms used for comparison.