Fabric defect detection is closely related to quality control. Therefore, it is an important topic in the modern textile field. In this article, a new fabric defect detection method using ACS-based thresholding and GA-based optimal Gabor filter is proposed. The multi-level gray-scale image thresholding is improved by using Adaptive cuckoo search (ACS) to calculate the best threshold. In order to suppress texture interference, the optimal Gabor filter calculated by genetic algorithm (GA) is used to achieve the optimal feature extraction of fabric defects. A specific image acquisition system is set up to obtain clear defective fabric images and corresponding defect-free images which are used to build the database. The performance of the method is evaluated by extensive experiments on various types of fabric defects. Experimental results show that the proposed method, effective and robust, is superior than the other six methods.