‘Apparent’ and actual hotel scores under Booking.com new reviewing system

Abstract

In September 2019, Booking.com changed its reviewing system based on the simple average of six items on a 2.5–10 scale by an unrestricted valuation on a 1–10 scale. This change has resulted in the drop of observed average scores. However, it is unclear which part of the shrinkage is due to the scale adjustment and which to priorly neglected aspects that consumers consider when valuing their satisfaction with the hotel stay. Using a dataset of more than 429,000 individual reviews for hotels in Madrid, Barcelona, Rome, Milan and Lisbon before and after the change, this paper disentangles apparent from actual changes in scores produced by the new scoring system. Using linear regressions and Propensity Score Matching, we show that, once the scale effect is left out, the new system has led to an increase by around 0.1 points in the actual valuation. Our results are potentially explained by the existence of unpacking effects.