This study employed unsupervised machine learning clustering algorithms to systematically analyze the spatial layout characteristics of residential buildings in villages along the Miao Frontier Wall and Miao Frontier Corridor in Western Hunan. The results indicated significant differences between the two regions in terms of the number of building clusters, distribution patterns, and compactness. A comparative analysis of the K-means and DBSCAN algorithms revealed that K-means is more effective in uncovering the internal spatial layout characteristics of settlements. Further analysis showed that villages along the Miao Frontier Wall exhibited greater diversity and complexity, whereas those along the Miao Frontier Corridor demonstrated higher clustering efficiency and denser internal building distribution. These differences can be attributed to variations in historical functions, geographical environments, planning concepts, and social structures. This research uncovers the spatial layout patterns of traditional settlements and proposes a machine learning-based approach to cultural heritage preservation, providing a theoretical foundation for future heritage conservation and spatial optimization, thereby promoting the sustainable development and protection of traditional cultural heritage.