Y-chromosome short tandem repeat (Y-STR) and Y-chromosome single nucleotide polymorphism (Y-SNP) are genetic markers on the male Y chromosome for individual identification, forensic applications, and paternal genetic history analysis. In this study we successfully genotyped 38 Y-STR loci and 24 Y-SNP loci of Pudong Han (n = 689) and Chongming Han (n = 530) in Shanghai. The haplotype diversity of the Y filer platinum genotyping system was the highest in the Han population in the Pudong area of Shanghai (0.99996) and Chongming Island (0.99997). The proportion of unique haplotypes was 97.10% (Pudong) and 98.49% (Chongming), respectively. The multidimensional scaling analysis and phylogenetic analysis were performed according to the genetic distance Rst, which was calculated based on the Y-STR gene frequency data. Moreover, we made a comparison on the frequency distribution analysis and principal component analysis of haplogroups in both populations. As a result, Shanghai Pudong Han, Chongming Island Han, and Jiangsu Han were determined to have a strong genetic affinity. The haplogroup distribution characteristics of the Pudong Han and Chongming Han populations were similar to those of the southern Han population. The results of haplotype network analysis showed that Jiangsu Wujiang Han and Jiangsu Changshu Han had more paternal genetic contributions to the formation of Shanghai Pudong Han and Chongming Island Han. Through the joint analysis of SNPs and STRs, this study deeply analyzed the paternal genetic structure of the Pudong Han and Chongming Han populations. The addition of Y-SNP haplogroups to forensic applications can provide information for pedigree investigation.