Tea-oil tree (Camellia oleifera Abel) is an important edible oil woody plant with a planting area over 3,800,000 hectares in southern China. Anthracnose is a serious disease of tea-oil tree in southern China, causing severe economic losses and posing a huge threat to the Ca. oleifera industry. Based on recent developments in the classification of Colletotrichum species, the objective of this study was to identify Colletotrichum species associated with tea-oil tree and examine their pathogenicity on leaves and fruits of Ca. oleifera. In total, 232 isolates were obtained from Ca. oleifera leaves and fruits with anthracnose symptoms. These isolates were further characterized based on morphology and multilocus phylogenetic analyses using partial DNA sequences at the ribosomal internal transcribed spacer regions and β-tubulin, actin, calmodulin, glyceraldehyde-3-phosphate dehydrogenase, and chitin synthase-encoding genes. The fungal isolates belong to five species: C. camelliae, C. fructicola, C. siamense, C. aenigma, and C. gloeosporioides. C. camelliae was the most predominant and widely distributed species on fruits of Ca. oleifera (91.4%), followed by C. fructicola (6.3%). However, C. fructicola was common and widely distributed species on leaves (75.9%), followed by C. camelliae (17.2%). There was no evidence of geographical specialization of the different species. Pathogenicity assays showed that all tested isolates, including 20 of C. camelliae, 11 of C. fructicola, four of C. siamense, two of C. aenigma, and one of C. gloeosporioides, were pathogenic to leaves and fruits of Ca. oleifera. Among the five species, C. camelliae species showed strong pathogenicity on both leaves and fruits of Ca. oleifera, and C. fructicola, C. siamense, C. aenigma, and C. gloeosporioides all showed weak pathogenicity on both leaves and fruits. No relationship was found between origin of isolates and their virulence. This is the first description of C. camelliae, C. fructicola, C. siamense, and C. gloeosporioides from the fruits of Ca. oleifera in China.