Political polarization is one of the most discussed challenges facing contemporary democracies and is often associated with a broader epistemic crisis. While inspiring a large literature in political science, polarization’s epistemic problems also have significance for normative democratic theory, and this study develops a new approach aimed at understanding them. In contrast to prominent accounts from political psychology—group polarization theory and cultural cognition theory—which argue that polarization leads individuals to form unreliable political beliefs, this study focuses on system-level diversity. It argues that polarization’s epistemic harms are best located in its tendency to reduce the diversity of perspectives utilized in a democratic system and in how this weakens the system’s ability to identify and address problems of public concern. Understanding such harms is also argued to require a greater consideration of the political dynamics of polarization and issues of elite discourse, alongside political psychology.