Immobilized metal ion affinity chromatography (IMAC) is useful in purification of histidine-tagged or histidine-rich proteins and peptides from a variety of hosts. However, phenolic compounds including polyphenols interfere with IMAC due to their high affinities for the transition metals immobilized on the column resins, which hampers the purification of proteins from plant-based host systems. In contrast to extensive knowledge of the mechanism of the interactions between phenolic compounds and transition metal ions in solution, an understanding of the interactions on the columns, where transition metal ions are immobilized on the resins, remains elusive. This study systematically investigated the affinity of phenolic compounds for transition metal ions by varying the number and position of phenolic hydroxyl groups (OH groups) and using different transition metals—Fe(II), Cu(II) and Ni(II)—on various IMACs, in which the columns were fabricated by equilibrating the cation-exchange column with transition metal solutions. It was found that the more OH groups the aromatic compounds have, the higher the affinity for transition metal ions; in particular, methyl gallate and pyrogallol were permanently bound to the IMAC column, which reflected coordinate bond formation with the transition metal ions. Importantly, the phenolic compounds showed no obvious affinity for the Ni(II)-IMAC column, in contrast to the Fe(II)- and Cu(II)-IMAC columns, whereas imidazole and histidine-tagged proteins showed evident binding to the Ni(II)-IMAC column. Ni(II)-IMAC should thus be especially effective in isolating histidine-tagged and histidine-rich species from phenolic compound-containing systems. These results indicate that the affinity between phenolic compounds and transition metal ions on the column is consistent with the results in solution. They also provide a comprehensive view for devising strategies to improve IMAC purification of target proteins and peptides from samples containing phenolic compounds.