Cluster Analysis and Rankings of Canadian Universities: Misadventures with Rank-based Data and Implications for the Welfare of Students

Abstract

We present a data-based perspective concerning the Maclean’s magazine (November 17, 2003) rankings of Canadian universities, including two cluster analyses and other nonparametric analyses. These data are similar to those in recent university ranking exercises conducted by other magazines, such as U.S. News. In many cases, the cluster procedure showed that universities actually resemble and relate to each other in a manner different from their formal classification and final rank ordering by Maclean’s. Several pitfalls in ranking procedures, related to unreliable relationships among specific indices underlying the final ranks, are outlined. Comparisons are made also with the most recent student satisfaction rankings for 47 Canadian universities, published in November, 2003, by the Toronto Globe and Mail. The latter rankings do not reliably reflect the general results of the Maclean’s data. In their present format, and although they have become increasingly publicized and promoted, it remains difficult for the Maclean’s data to be consistently or empirically useful to students. Ranking exercises have unintended, though increasingly predictable, consequences, which likely bear heavily upon the intellectual and personal well being of students.