Seven human-specific Alu markers were studied in 574 unrelated individuals from 10 endogamous groups and 2 hill tribes of Tamil Nadu and Kerala states. DNA was isolated, amplified by PCR-SSP, and subjected to agarose gel electrophoresis, and genotypes were assigned for various Alu loci. Average heterozygosity among caste populations was in the range of 0.292–0.468. Among tribes, the average heterozygosity was higher for Paliyan (0.3759) than for Kani (0.2915). Frequency differences were prominent in all loci studied except Alu CD4. For Alu CD4, the frequency was 0.0363 in Yadavas, a traditional pastoral and herd maintaining population, and 0.2439 in Narikuravars, a nomadic gypsy population. The overall genetic difference (Gst) of 12 populations (castes and tribes) studied was 3.6%, which corresponds to the Gst values of 3.6% recorded earlier for Western Asian populations. Thus, our study confirms the genetic similarities between West Asian populations and South Indian castes and tribes and supported the large scale coastal migrations from Africa into India through West Asia. However, the average genetic difference (Gst) of Kani and Paliyan tribes with other South Indian tribes studied earlier was 8.3%. The average Gst of combined South and North Indian Tribes (CSNIT) was 9.5%. Neighbor joining tree constructed showed close proximity of Kani and Paliyan tribal groups to the other two South Indian tribes, Toda and Irula of Nilgiri hills studied earlier. Further, the analysis revealed the affinities among populations and confirmed the presence of North and South India specific lineages. Our findings have documented the highly diverse (micro differentiated) nature of South Indian tribes, predominantly due to isolation, than the endogamous population groups of South India. Thus, our study firmly established the genetic relationship of South Indian castes and tribes and supported the proposed large scale ancestral migrations from Africa, particularly into South India through West Asian corridor.