Building wall texture is an important manifestation of national architectural culture and plays an important role in the protection and inheritance of national architecture. The walls of traditional villages in Jiarong Tibetan area have unique textures. Xisuo Village, as a representative traditional village in Jiarong Tibetan area, represents the geographical heritage of national cultural memory and brick and stone technology. The wall texture of traditional buildings in the village is unique and representative. Therefore, this paper conducted field research and sampling on the texture of jiarong Tibetan stone masonry building in Xisuo Village, and conducted quantitative analysis with the combination of digital processing technology. Meanwhile, we also selected the wall texture of stone buildings in other areas for comparative analysis. Finally, through the quantitative analysis of the four factors of stone size, shape, horizontal and vertical arrangement of the Jiarong Tibetan stone building walls, the variation law of the wall texture is summarized, and the irregular index of the wall texture is calculated. The results show that the indexes of factors affecting the texture of the building wall in Xisuo Village are Stone shape index (0.4–0.7), stone size index (0.25–0.6), stone horizontal arrangement index (0.05–0.35) and stone horizontal arrangement index (0.25–0.8), the overall irregularity index of the wall is (0.4–0.5). Compared with the building walls in other areas, it was found that the closer the stones of the building wall to the regular rectangle, the smaller the value of the stone shape factor. The stone size index value would decrease along with the increase of the content of large sized stones on the wall, and the regularity degree of the wall was related to the content and alignment of smaller stones. At the same time, the more irregular the stone shapes were and the changes in masonry angle, the more tortuous the horizontal joints would become; the larger the size of the stones, the more direct vertical seams formed a more favorable wall texture. Geological environment, construction methods, different Tibetan branches and administrative divisions will have certain influence on the texture of the building wall. The characteristic of the image formation of this texture was unique which was different from other regions. This study provides a quantitative research method for the protection of traditional village buildings, objectively defines the irregular index range of the texture of the building walls in Xisuo Village, and analyzes the changing rules and influencing factors of the texture characteristics of stone building walls. It also provides techniques and methods for the preservation and inheritance of traditional architectural texture culture.