The content of toxic heavy metals (THMs), including lead (Pb), arsenic (As), cadmium (Cd), and mercury (Hg), was determined in a total of 10,245 samples for 279 types of Chinese herbal medicine (CHM) using a validated inductively coupled plasma mass spectrometry method. The exceeding rate (ER) for the four THMs were calculated based on diverse permissible limits (PLs) established by different organizations and national pharmacopeias. Cluster analysis was used to classify the degree risk of THMs contamination according to the calculated ER. Results revealed that Cibotii rhizome, Selaginellae herba, Morindae officinalis radix, Asprellae ilicis radix, and Toxicodendri resina exhibited high-degree risk of Pb contamination. Eckloniae/Laminariae thallus, Spirodelae herba, and Naturalis indigo possessed high-degree risk of As contamination. Tetrapanacis medulla, Centipedae herba, Cyathulae radix, Linderae radix, Meretricis/Cyclinae concha, and Tabanus displayed high-degree risk of Cd contamination. Toxicodendri resina has high-degree risk of Hg contamination. In addition, six types of CHM, including Asprellae ilicis radix, Toxicodendri resina, Eckloniae/Laminariae thallus, Fossilia Ossis Mastodi, Haematitum, and Hedyotidis diffusae herba, may have non-carcinogenic health risk after consumption of raw materials because the calculated hazard quotient and hazard index were over 1.0. In summary, these data provide useful information about THMs contamination in CHM.