In the detection of escape ladders in the context of smart construction sites, due to the relatively small target size of the escape ladder compared to the entire input image frame, significant environmental interference, and high missed detection and false detection rates, an improved YOLOv5s escape ladder real-time detection algorithm is proposed by combining the attention mechanism network. The model uses CSPLocknet53 as the backbone network for feature extraction, introduces the attention module CA, and integrates spatial and channel information, while increasing a small amount of computation, performance has been significantly improved. Optimize the network structure of YOLOv5s algorithm, strengthen shallow feature weights to enhance small target detection effectiveness, add attention mechanisms to increase the weight of small targets and their surrounding features, and use Mosaic methods for data augmentation to improve detection accuracy and recall. After multiple repeated experiments, these experimental results have proven that the optimized YOLOv5s algorithm for real-time detection of escape ladders has an average detection accuracy (accuracy, recall) of (81.8, 82.6). Compared with the traditional YOLOv5s algorithm, the accuracy and recall have been improved by 1.4% and 1.2%, respectively. The optimized YOLOv5s algorithm can effectively improve the detection accuracy of real-time detection of escape ladders, and improve the detection and resolution performance of small escape ladder targets.