The tea plant [Camellia sinensis (L.) O. Kuntze] is an industrial crop in China. The Anhui Province has a long history of tea cultivation and has a large resource of tea germplasm with abundant genetic diversity. To reduce the cost of conservation and utilization of germplasm resources, a core collection needs to be constructed. To this end, 573 representative tea accessions were collected from six major tea-producing areas in Anhui Province. Based on 60 pairs of simple sequence repeat (SSR) markers, phylogenetic relationships, population structure and principal coordinate analysis (PCoA) were conducted. Phylogenetic analysis indicated that the 573 tea individuals clustered into five groups were related to geographical location and were consistent with the results of the PCoA. Finally, we constructed a core collection consisting of 115 tea individuals, accounting for 20% of the whole collection. The 115 core collections were considered to have a 90.9% retention rate for the observed number of alleles (Na), and Shannon’s information index (I) of the core and whole collections were highly consistent. Of these, 39 individuals were preserved in the Huangshan area, accounting for 33.9% of the core collection, while only 10 individuals were reserved in the Jinzhai County, accounting for 8.9% of the core set. PCoA of the accessions in the tea plant core collection exhibited a pattern nearly identical to that of the accessions in the entire collection, further supporting the broad representation of the core germplasm in Anhui Province. The results demonstrated that the core collection could represent the genetic diversity of the original collection. Our present work is valuable for the high-efficiency conservation and utilization of tea plant germplasms in Anhui Province.