Mountainous ethnic tourism lands are important social-ecological system types. With tourism as the main disturbance factor, the theory of social-ecological system resilience provides a new way to realize the sustainable development of ethno-tourism in mountainous areas. This study divides the social-ecological system into social, economic, and ecological subsystems. It constructs an evaluation index system to assess the resilience of ethnic tourism destinations in mountainous areas, considering vulnerability and adaptability. We investigate 64 counties in the Wuling Mountain area and use set-pair analysis to assess the resilience index of the social-ecological system from 2000 to 2020 and reveal the temporal and spatial characteristics. Obstacle degree models and a genetic algorithm-back propagation neural network are utilized to determine the influencing factors and predict future development trends. The following results were obtained: (1) Temporally, the resilience index shows a steady upward trend, reaching a moderate level. The resilience of the social subsystem fluctuates and rises; the economic subsystem exhibits slow, fast, and slow growth rates with occasional abrupt changes; and the ecological subsystem demonstrates a stable, slightly increasing trend. (2) Spatially, the resilience index is high at the edges and low in the central area, exhibiting a concave distribution. Most counties have moderate or higher resilience. The social and ecological subsystems have low resilience in the south and high resilience in the north. The resilience of the economic subsystem is high at the edges and low in the central area. (3) On the distribution of major obstacle factors, the first two are similar at the county level, and the last three are significantly different. The similarity of the barrier factors is related to the degree of regional proximity of the county, and overall, the similarity is decreasing from north to south and from west to east in the distribution pattern within the area. and to a certain extent, it is affected by terrain and geomorphology. (4) The spatial distribution of the resilience index is similar in 2025 and 2030. The index decreases slightly and then increases annually, with a lower growth rate in the south than in the north. Lower values occur in the northern and southwestern parts, whereas higher values are observed around high-value areas. The region as a whole will develop in a coordinated and integrated manner in the future.