Dynamic, Multidimensional, and Skillset-Specific Reputation Systems for Online Work

Abstract

Current reputation systems in online (labor) markets are overly positive and unidimensional. This article presents a new reputation framework that combines human input with machine learning to provide dynamic, multidimensional, and skill-set-specific quality assessments. The framework significantly outperforms current reputation systems. By providing more representative reputation scores, the framework helps workers to differentiate, employers to make informed decisions, and the market to improve its recommendation algorithms and understand the supply distributions across different dimensions. The framework generalizes in other contexts where reputation systems are overly positive and unidimensional. The framework highlights how combining human input with advanced machine learning techniques can augment intelligence by creating the necessary conditions for humans to make informed decisions. Such systems have the potential to increase efficiency and outcome quality precisely because they intelligently differentiate workers. The deployment of the proposed intelligence augmentation framework in different types of online platforms could have implications for workers, employers, businesses, and the future of work.