Incorporating convolutions into transformers for textile fiber identification from fabric images

Abstract

Simple, fast and effective fiber identification can help consumers purchase their desired apparel and help the industry conduct large-scale textile testing. This paper presents a transformer architecture incorporating convolutions to recognize fibers in textile surface images, which meets the above requirements. Firstly, a convolution operation is performed on textile images to pick up overlapping patches as tokens and the linear projections in transformer encoders are replaced by depth-wise separable convolutions to extract the fiber representations. Secondly, the multi-head cross-attention module enables each label embedding to be compared with features at each spatial location to locate and pool the corresponding fiber characteristics. Finally, a simplified asymmetric loss is introduced to further purify the extracted fiber features. Experiments demonstrate that the proposed approach provides a significant improvement in fiber identification accuracy over both state-of-the-art multi-label classification frameworks and fiber identification architectures.