Recent molecular surveys have advanced our understanding of the forces shaping the large-scale ecological distribution of microbes in Earth's extreme habitats, such as hot springs and acid mine drainage. However, few investigations have attempted dense spatial analyses of specific sites to resolve the local diversity of these extraordinary organisms and how communities are shaped by the harsh environmental conditions found there. We have applied a 16S rRNA gene-targeted 454 pyrosequencing approach to explore the phylogenetic differentiation among 90 microbial communities from a massive copper tailing impoundment generating acidic drainage and coupled these variations in community composition with geochemical parameters to reveal ecological interactions in this extreme environment. Our data showed that the overall microbial diversity estimates and relative abundances of most of the dominant lineages were significantly correlated with pH, with the simplest assemblages occurring under extremely acidic conditions and more diverse assemblages associated with neutral pHs. The consistent shifts in community composition along the pH gradient indicated that different taxa were involved in the different acidification stages of the mine tailings. Moreover, the effect of pH in shaping phylogenetic structure within specific lineages was also clearly evident, although the phylogenetic differentiations within the Alphaproteobacteria, Deltaproteobacteria, and Firmicutes were attributed to variations in ferric and ferrous iron concentrations. Application of the microbial assemblage prediction model further supported pH as the major factor driving community structure and demonstrated that several of the major lineages are readily predictable. Together, these results suggest that pH is primarily responsible for structuring whole communities in the extreme and heterogeneous mine tailings, although the diverse microbial taxa may respond differently to various environmental conditions.