The development of wool has a complex regulatory mechanism both influenced by genetic and environmental factors. MicroRNAs (miRNA) were involved in various biological processes of animals, and may play an important role in the regulation of wool development. In this study, we comprehensively analyzed and identified the histological parameters of hair follicles, as well as the miRNAs, target genes, pathways, and Gene Ontology terms related to wool fineness regulation and wool growth and development using HE staining and RNA-Seqs methods. Both coarse (group C, mean fiber diameter (MFD) = 22.26 ± 0.69 μm, n = 6) and fine (group F, MFD = 16.91 ± 0.29 μm, n = 6) of Gansu alpine fine-wool sheep with different wool fineness were used in this study. The results showed that the primary follicle diameter and secondary wool fiber diameter in group C were significantly higher than those in group F (P < 0.05). And the number of primary and secondary hair follicles in group C was significantly lower than that in group F (P < 0.05). Furthermore, a total of 67 DE miRNAs and 290 potential DE miRNAs target genes were screened in the skin tissues of sheep from groups F and C, and some potential target genes related to wool fineness regulation were screened, such as CDH2, KRT82, FOXN1, LOC101106296, KRT20, MCOLN3, KRT71, and TERT. These genes were closely related to Glutathione metabolism, epidermal cell differentiation, keratinization, and regulation of hair cycle. Moreover, the regulatory network of miRNAs-mRNAs suggested that miRNAs (miR-129-x, novel m0079-3p, miR-2484-z, novel m0025-5P, etc.) may play a key role in the wool development and wool fineness regulation of Gansu alpine fine-wool sheep. In summary, this study expands the existing miRNAs database and provides new information for studying the regulation of wool development in Gansu alpine fine wool sheep.