Modeling Change and Variation in U.S. News & World Report College Rankings: What would it really take to be in the Top 20?

Abstract

University administrators may invest significant time and resources with the goal of improving their U.S. News & World Report ranking, but the real impact of these investments is not well known since, as other universities make similar changes, rankings become a moving target. This research removes the mystique of the U.S. News ranking process by producing a ranking model that faithfully recreates U.S. News outcomes and quantifies the inherent “noise” in the rankings for all nationally ranked universities. The model developed can be a valuable tool to institutional researchers and university leaders by providing detailed insight into the U.S. News ranking process. It allows the impact of changes to U.S. News subfactors to be studied when variation between universities and within subfactors is present. Numerous simulations were run using this model to understand the effect of each subfactor individually and to determine the amount of change that would be required for a university to improve its rank or move into the top 20. Results show that for a university ranked in the mid-30 s it would take a significant amount of additional resources, directed in a very focused way, to become a top-ranked national university, and that rank changes of up to ± 4 points should be considered “noise”. These results can serve as a basis for frank discussions within a university about the likelihood of significant changes in rank and provide valuable insight when formulating strategic goals.