Despite their ancient past and high diversity, African populations are the least represented in human population genetic studies. In this study, uniparental markers (mtDNA and Y chromosome) were used to investigate the impact of sociocultural factors on the genetic diversity and inter-ethnolinguistic gene flow in the three major Nigerian groups: Hausa (n = 89), Yoruba (n = 135) and Igbo (n = 134). The results show a distinct history from the maternal and paternal perspectives. The three Nigerian groups present a similar substrate for mtDNA, but not for the Y chromosome. The two Niger–Congo groups, Yoruba and Igbo, are paternally genetically correlated with populations from the same ethnolinguistic affiliation. Meanwhile, the Hausa is paternally closer to other Afro-Asiatic populations and presented a high diversity of lineages from across Africa. When expanding the analyses to other African populations, it is observed that language did not act as a major barrier to female-mediated gene flow and that the differentiation of paternal lineages is better correlated with linguistic than geographic distances. The results obtained demonstrate the impact of patrilocality, a common and well-established practice in populations from Central-West Africa, in the preservation of the patrilineage gene pool and in the affirmation of identity between groups.