Small changes in a protein’s core packing produce changes in function, and even small changes in function bias species fitness and survival. Therefore individually deleterious mutations should be evolutionarily coupled with compensating mutations that recover fitness. Co-evolving pairs of mutations should be littered across evolutionary history. Despite longstanding intuition, the results of co-evolution analyses have largely disappointed expectations. Regardless of the statistics applied, only a small majority of the most strongly co-evolving residues are typically found to be in contact, and much of the “meaning” of observed co-evolution has been opaque. In a medium-sized protein of 300 amino acids, there are almost 20 million potentially-important interdependencies. It is impossible to understand this data in textual format without extreme summarization or truncation. And, due to summarization and truncation, it is impossible to identify most patterns in the data. We developed a visualization approach that eschews the common “look at a long list of statistics” approach and instead enables the user to literally look at all of the co-evolution statistics simultaneously. Users of our tool reported visually obvious “clouds” of co-evolution statistics forming distinct patterns in the data, and analysis demonstrated that these clouds had structural relevance. To determine whether this phenomenon generalized, we repeated this experiment in three proteins we had not previously studied. The results provide evidence about how structural constrains have impacted co-evolution, why previous “examine the most frequently co-evolving residues” approaches have had limited success, and additionally shed light on the biophysical importance of different types of co-evolution.