Records from polar and alpine ice reflect past changes in background and industrial toxic heavy metal emissions. While Northern Hemisphere records have been used to evaluate environmental effects and linkages to historical events such as foreign conquests, plagues, economic downturns, and technological developments during the past three millennia, little is known about the magnitude and environmental effects of such emissions in the Southern Hemisphere or their historical linkages, especially prior to late 19th century industrialization. Here we used detailed measurements of the toxic heavy metals lead, cadmium, and thallium, as well as non-toxic bismuth, cerium, and sulfur in an array of five East Antarctic ice cores to investigate hemispheric-scale pollution during the Common Era. While thallium showed no anthropogenic increases, the other three metals increased by orders of magnitude in recent centuries after accounting for crustal and volcanic components. These first detailed records indicate that East Antarctic lead pollution started in the 13th century coincident with Late Intermediate Period metallurgy in the Andes and was pervasive during the Spanish Colonial period in parallel with large-scale exploitation of Andean silver and other ore deposits. Lead isotopic variations suggest that 19th-century increases in lead, cadmium, and bismuth resulted from Australian lead and Bolivian tin mining emissions, with 20th century pollution largely the result of the latter. As in the Northern Hemisphere, variations in heavy metal pollution coincided with plagues, cultural and technological developments, as well as global economic and political events including the Great Depression and the World Wars. Estimated atmospheric heavy metal emissions from Spanish Colonial-era mining and smelting during the late 16th and early 17th century were comparable to estimated European emissions during the 1st-century apex of the Roman Empire, with atmospheric model simulations suggesting hemispheric-scale toxic heavy metal pollution during the past five centuries as a result.