Rare species consisting of small populations are subject to random genetic drift, which reduces genetic diversity. Thus, determining the relationship between population size and genetic diversity would provide key information for planning a conservation strategy for rare species. We used six microsatellite markers to investigate seven extant populations of the rare conifer Pseudotsuga japonica, which is endemic to the Kii Peninsula and Shikoku Island regions that are geographically separated by the Kii Channel in southwest Japan. The population differentiation of P. japonica was relatively high (FST = 0.101) for a coniferous species, suggesting limited gene flow among populations. As expected, significant regional differentiation (AMOVA; p < 0.05) indicated genetic divergence across the Kii Channel. A strong positive correlation between census population size and the number of rare alleles (r = 0.862, p < 0.05) was found, but correlations with major indices of genetic diversity were not significant (allelic richness: r = 0.649, p = 0.104, expected heterozygosity: r = 0.361, p = 0.426). The observed order of magnitude of correlation with three genetic diversity indices corresponded with the theoretically expected order of each index’ sensitivity (i.e., the rate of decline per generation) to the bottleneck event. Thus, features that exhibit a faster response, i.e., the number of rare alleles, would have been subject to deleterious effects of the recent decline in population size, which is presumably caused by the development of extensive artificial plantations of other tree species over the last several decades. Finally, we propose a conservation plan for P. japonica based on our findings.