Aflatoxins are the most dangerous mycotoxins for food safety. They are mainly produced by Aspergillus flavus, A. parasiticus, and A. minisclerotigenes. The latter, an understudied species, was the main culprit for outbreaks of fatal aflatoxicosis in Kenya in the past. To determine specific genetic characteristics of these Aspergillus species, their genomes are comparatively analyzed. Differences reflecting the typical habitat are reported, such as an increased number of carbohydrate-active enzymes, including enzymes for lignin degradation, in the genomes of A. minisclerotigenes and A. parasiticus. Further, variations within the aflatoxin gene clusters are described, which are related to different chemotypes of aflatoxin biosynthesis. These include a substitution within the aflL gene of the A. parasiticus isolate, which leads to the translation of a stop codon, thereby switching off the production of the group 1 aflatoxins B1 and G1. In addition, we demonstrate that the inability of the A. minisclerotigenes isolates to produce group G aflatoxins is associated with a 2.2 kb deletion within the aflF and aflU genes. These findings reveal a relatively high genetic homology among the three Aspergillus species investigated. However, they also demonstrate consequential genetic differences that have an important impact on risk-assessment and food safety.