Core forests are an important component of forest landscapes and wildlife habitat. Although the core forests were damaged during the development of shale gas sites, it remain unclear how much damage the shale gas development has caused to this ecologically vulnerable region. We analyzed high-resolution remote sensing images of a shale gas development area in 2012, 2014, and 2017 in the karst region in southwestern China. The results showed that the core forest area decreased by approximately 4.0% from 2012 to 2017. Of this decrease, approximately 32.3% was related to the shale gas development activities, while 67.7% was related to other human activities, i.e., agricultural lands and residential developments. Approximately 5.6% of the decrease in the core forest was for new pipelines, with 0.5 ha occurred in 2012–2014 and 248.6 ha occurred in 2014–2017. Of the shale gas development activities, the pipeline constructions were most detrimental to the core forest. The patchiness of the core forest increased by 8.2% from 2012 to 2017 by the expansions of dry fields, towns, and settlements. The core forest Effective Mesh Size (MESH) decreased by 86.3%, primarily caused by the shale gas development pipelines. In conclusion, human activities that were not directly related to shale gas development were the main driver of the core forest decreases. The pipelines caused most losses of the core forest among the shale gas activities and the impacts deteriorated as the shale gas development proceeds. Therefore, we propose that new shale gas pads should be placed adjacent to existing shale gas pipelines and new shale gas pipelines should be constructed in parallel with existing roads to reduce the damages on core forest.