An Iterative Method for Calculating Robust Rating Scores

Abstract

In this paper we introduce an iterative voting algorithm and then use it to obtain a rating method which is very robust against collusion attacks as well as random and biased raters. Unlike the previous iterative methods, our method is not based on comparing submitted evaluations to an approximation of the final rating scores, and it entirely decouples credibility assessment of the cast evaluations from the ranking itself. The convergence of our algorithm relies on the existence of a fixed point of a continuous mapping which is also a stationary point of a constrained optimization objective. We have implemented and tested our rating method using both simulated data as well as real world data. In particular, we have applied our method to movie evaluations obtained from MovieLens and compared our results with IMDb and Rotten Tomatoes movie rating sites. Not only are the ratings provided by our system very close to IMDb rating scores, but when we differ from the IMDb ratings, the direction of such differences is essentially always towards the ratings provided by the critics in Rotten Tomatoes. Our tests demonstrate high efficiency of our method, especially for very large online rating systems, for which trust management is both of the highest importance and one of the most challenging problems.