Taxonomy of the brown algal genus Dictyota has a long and troubled history. Our inability to distinguish morphological plasticity from fixed diagnostic traits that separate the various species has severely confounded species delineation. From continental Europe, more than 60 species and intraspecific taxa have been described over the last two centuries. Using a molecular approach, we addressed the diversity of the genus in European waters and made necessary taxonomic changes. A densely sampled DNA data set demonstrated the presence of six evolutionarily significant units (ESUs): Dictyota dichotoma (Huds.) J. V. Lamour., D. fasciola (Roth) J. V. Lamour., D. implexa J. V. Lamour., D. mediterranea (Schiffn.) G. Furnari, D. spiralis Mont., and the newly described D. cyanoloma sp. nov., which was previously reported as D. ciliolata from the Mediterranean Sea. Species distributions, based on DNA‐confirmed occurrence records, indicate that all species are geographically confined to the NE Atlantic Ocean with the exception of D. dichotoma and D. implexa, which also occur in South Africa and Bermuda, respectively. To investigate potential hybridization between D. dichotoma and D. implexa, which were previously shown to be sexually compatible in culture, we compiled and analyzed sets of mitochondrial, plastid, and nuclear markers to detect putative hybrids or introgression in natural populations. Failure to detect natural hybrids indicates that effective pre‐ and postzygotic isolation mechanisms are at play in natural populations and supports the by‐product hypothesis of reproductive isolation.