A new texture-feature description operator, called the multidirectional binary patterns (MDBP) operator, is proposed in this paper. The operator can extract the detailed distribution of textures in local regions by comparing the differences in the gray levels between neighboring pixels. Moreover, the texture expression ability is enhanced by focusing on the texture features in the linear neighborhood of the image in multiple directions. The MDBP operator was modified by introducing a “uniform” pattern to reduce the grayscale values in the image. Combining the “uniform” MDBP operator and the gray-level co-occurrence matrix, an unpatterned fabric-defect detection scheme is proposed, including texture-feature extraction and detection stages. In the first stage, the multidirectional texture-feature matrix of a nondefective fabric image is extracted, and then the detection threshold is determined based on the similarity between the feature matrices. In the second stage, the defect is detected with the detection threshold. The proposed method is adapted to various grayscale textile images with different characteristics and is robust to a wide variety of image-processing operations. In addition, it is invariant to grayscale changes, performs well when representing textures and detecting defects and has lower computational complexity than other methods.