To solve the problem of low detection accuracy due to the loss of detailed information when extracting pavement crack features in traditional U-shaped networks, a pavement crack detection method based on multiscale attention and hesitant fuzzy set (HFS) is proposed. First, the encoding-decoding structure is used to construct a pavement crack segmentation network, ResNeXt50 is used to extract features in the encoding stage, and a multiscale feature fusion module (MFF) is designed to obtain multiscale context information. Second, in the decoding stage, a high-efficiency dual attention module (EDA) is used to enhance the ability of capturing details of the cracks while suppressing background noise. Finally, the membership degree of the crack is calculated based on the advantages of the HFS in multiattribute decision-making to obtain the similarity of the crack, and the binary image after segmentation is judged by the hesitation fuzzy measure. The experiment was conducted on the public road crack dataset Crack500. In terms of segmentation performance, the evaluation indexes Intersection over Union (IoU), Precision, and Dice coefficients of the proposed network reached 55.56%, 74.26%, and 67.43%, respectively; in terms of classification performance, for transversal and longitudinal cracks, the classification accuracy was 84% ± 0.5%, while the block and the alligator were both 78% ± 0.5%. The experimental results prove that the crack details detected by the proposed method are more abundant, and the image detection effect of complex topological structures and small cracks are better.