Wildfire is globally an important ecological disturbance affecting biochemical cycles and vegetation composition, but also puts people and their homes at risk. Suppressing wildfires has detrimental ecological effects and can promote larger and more intense wildfires when fuels accumulate, which increases the threat to buildings in the wildland-urban interface (WUI). Yet, when wildfires occur, typically only a small proportion of the buildings within the fire perimeter are lost, and the question is what determines which buildings burn. Our goal was to examine which factors are related to building loss when a wildfire occurs throughout the United States. We were particularly interested in the relative roles of vegetation, topography, and the spatial arrangement of buildings, and how their respective roles vary among ecoregions. We analyzed all fires that occurred within the conterminous United States from 2000 to 2010 and digitized which buildings were lost and which survived according to Google Earth historical imagery. We modeled the occurrence as well as the percentage of buildings lost within clusters using logistic and linear regression. Overall, variables related to topography and the spatial arrangement of buildings were more frequently present in the best 20 regression models than vegetation-related variables. In other words, specific locations in the landscape have a higher fire risk, and certain development patterns can exacerbate that risk. Fire policies and prevention efforts focused on vegetation management are important, but insufficient to solve current wildfire problems. Furthermore, the factors associated with building loss varied considerably among ecoregions suggesting that fire policy applied uniformly across the United States will not work equally well in all regions and that efforts to adapt communities to wildfires must be regionally tailored.