AbstractDetection of defects is an essential quality control method in fabric production. Unsupervised deep learning‐based reconstruction algorithms have recently been deeply concerned owing to scarce fabric defect samples, high annotation cost, and deficient prior knowledge. Most unsupervised reconstruction models are prone to overfitting and poor generalisation performance, resulting in blurred images, residual defects, and uneven textures in the reconstruction results. On this account, an unsupervised fabric surface defect detection method using the Progressive Mask Repair Model (PMRM) has been developed. Specifically, PMRM with transformer architecture gathers detailed feature information. In order to pay closer attention to the textural properties of fabrics, the model incorporates structural similarity as a constraint in the training stage. In the detection stage, we designate the non‐defective area of the fabric image as the background and the defective area as the foreground. Next, a progressive mask is applied to repair the background of the defective area, which avoids defect false detection resulting from the poor reconstruction effect of the traditional reconstruction model in the non‐defective area. Finally, image processing methods such as image difference, frequency‐tuned salient detection, and threshold binarisation are used to segment the defects. Relative to the other six unsupervised defect detection methods, the proposed scheme increases the F1 score and intersection over union (IoU) by at least 9.34% and 8.49%, respectively. According to the earlier results, PMRM is effective and exhibits superiority.