Federated Truth Inference over Distributed Crowdsourcing Platforms

Abstract

This work examines the truth inference problem in a distributed crowdsourcing scenario. Labeling tasks are outsourced to workers associated with different platforms, and truth inference is to be performed without sharing the workers’ individual responses with other platforms or the central coordinator. The reliability of the labels may vary over different workers and tasks, and is characterized by a Gaussian mixture model. A federated truth inference (FTI) algorithm is proposed based on a distributed implementation of the block expectation-maximization (EM) algorithm. The messages sent by each local platform to the central coordinator contain aggregates of workers’ responses, instead of individual labels. The convergence of the FTI algorithm can be verified theoretically. A communication-efficient variant of the FTI scheme is also proposed by allowing the distributed platforms to perform multiple local EM computations before updating the global estimates at the central coordinator in each iteration. The effectiveness of our proposed schemes is demonstrated using both synthetic and real-world datasets.