Feature-enriched author ranking in incomplete networks

Abstract

Evaluating scientists based on their scientific production is a controversial topic. Nevertheless, bibliometrics and algorithmic approaches can assist traditional peer review in numerous tasks, such as attributing research grants, deciding scientific committees, or choosing faculty promotions. Traditional bibliometrics rank individual entities (e.g., researchers, journals, faculties) without looking at the whole data (i.e., the whole network). Network algorithms, such as PageRank, have been used to measure node importance in a network, and have been applied to author ranking. However, traditional PageRank only uses network topology and ignores relevant features of scientific collaborations. Multiple extensions of PageRank have been proposed, more suited for author ranking. These methods enrich the network with information about the author’s productivity or the venue and year of the publication/citation. Most state-of-the-art (STOA) feature-enriched methods either ignore or do not combine effectively this information. Furthermore, STOA algorithms typically disregard that the full network is not known for most real-world cases.Here we describe OTARIOS, an author ranking method recently developed by us, which combines multiple publication/citation criteria (i.e., features) to evaluate authors. OTARIOS divides the original network into two subnetworks, insiders and outsiders, which is an adequate representation of citation networks with missing information. We evaluate OTARIOS on a set of five real networks, each with publications in distinct areas of Computer Science, and compare it against STOA methods. When matching OTARIOS’ produced ranking with a ground-truth ranking (comprised of best paper award nominations), we observe that OTARIOS is >30% more accurate than traditional PageRank (i.e., topology based method) and >20% more accurate than STOA (i.e., competing feature enriched methods). We obtain the best results when OTARIOS considers (i) the author’s publication volume and publication recency, (ii) how recently the author’s work is being cited by outsiders, and (iii) how recently the author’s work is being cited by insiders and how individual he is. Our results showcase (a) the importance of efficiently combining relevant features and (b) how to adequately perform author ranking in incomplete networks.