ProficiencyRank: Automatically ranking expertise in online collaborative social networks

Abstract

Users of online collaborative social networks (e.g., StackExchange, Yahoo! Answers, and Yask, among others) are distributed across a continuous range of expertise, ranging from beginners to experts. However, only a minority of users contribute to their communities by posting questions and answers, while the majority (lurkers) only contribute by voting up or down on others’ posts. Since current methods for ranking expertise are based mainly on user submitted posts, their results are limited to that minority. We introduce ProficiencyRank, a wrapping method on top of PageRank that leverages voting information to determine user expertise for both contributors and lurkers. We validated our method to measure the proficiency of English Language Learners in the Yask social network. The new data set is naturally annotated with self-assessments of proficiency given by the users themselves, providing ground truth for all users. Experimental validation showed that ProficiencyRank produces a meaningful ranking of users that overcame various baselines based on intrinsic and extrinsic information. We conclude that this technology can benefit these communities by expanding the coverage of user expertise/reputation measurement and providing a re-ranking method for the expert finding task.