Apple Glomerella leaf spot (GLS) is an emerging fungal disease caused by Colletotrichum fructicola and other Colletotrichum species. These species are polyphyletic and it is currently unknown how these pathogens convergently evolved to infect apple. We generated chromosome-level genome assemblies of a GLS-adapted isolate and a non-adapted isolate in C. fructicola using long-read sequencing. Additionally, we resequenced 17 C. fructicola and C. aenigma isolates varying in GLS pathogenicity using short-read sequencing. Genome comparisons revealed a conserved bipartite genome architecture involving minichromosomes (accessory chromosomes) shared by C.fructicola and other closely related species within the C. gloeosporioides species complex. Moreover, two repeat-rich genomic regions (1.61 Mb in total) were specifically conserved among GLS-pathogenic isolates in C. fructicola and C. aenigma. Single-gene deletion of 10 accessory genes within the GLS-specific regions of C. fructicola identified three that were essential for GLS pathogenicity. These genes encoded a putative non-ribosomal peptide synthetase, a flavin-binding monooxygenase and a small protein with unknown function. These results highlight the crucial role accessory genes play in the evolution of Colletotrichum pathogenicity and imply the significance of an unidentified secondary metabolite in GLS pathogenesis.