When performed from an Unmanned Aerial Vehicle (UAV) perspective, infrared target detection often suffers from low accuracy, excessive model parameters, and slow processing speed. To address these challenges, this paper proposes a lightweight infrared target detection algorithm, named YOLO-TSL, which is based on an improved version of YOLOv8n. Building on an in-depth analysis of UAV infrared image characteristics, this paper introduces Triplet Attention to the model backbone to enhance target detection accuracy and effectively suppress background interference. Moreover, YOLO-TSL incorporates the Slim-Neck architecture, which features GSConv and GSbottleneck in the neck structure and employs a one-time aggregation method to design the cross-level partial network known as the VoV-GSCSP module. This architecture significantly reduces the model’s computational complexity while maintaining high detection accuracy. Furthermore, this paper introduces an innovative inner-MPDIoU loss function that optimizes IoU loss computation by enhancing bounding box similarity and adaptive auxiliary bounding box scale adjustments, based on both inner-IoU and MDPIoU. Following experimental validation, YOLO-TSL demonstrate significant improvements over YOLOV8n, including a 3.9% increase in mAP50 and a 3% increase in recall. Additionally, YOLO-TSL has 15.7% fewer parameters, requires 11% less computation, and offers 26% faster inference. Comparative experiments on the FLIR dataset reveal that the algorithm not only outperforms YOLOv8n by 1.4% in mAP50, but also offers significant advantages over other algorithms in terms of parameter reduction and computational efficiency, thus demonstrating YOLO-TSL’s superiority in accuracy, efficiency, and speed.