Traffic helmet object detection is playing an increasing important role in the smart traffic fields. However, object size variation and small-shaped helmet detection has still been a challenging problem by reason of their poor visual appearance in the image. In this work, we present an efficient traffic helmet detector through feature enhancement and lightweight design based on YOLOv8n called THDet. Specifically, we employ the coordinate attention into C2f blocks combined with softmax activate function to achieve feature channel aggregation and strong non-linear expression of the backbone for further effective feature extraction; Next, Focal_CIoU loss function embedded with Focal Loss method is utilized for the more precise measure of various objects bounding box regression and balance of positive and negative examples during training; Then, a new lightweight detection head style is designed only with two proper position heads (P3 & P4) to perform final classification and localization, through this scheme saving the 33.7% parameters than baseline method. Finally, Attention Refined Features Module (ARFM) is built to calibrate the multi-scale fused features by introducing 3-D weights generated from SimAttention to boost the final detection accuracy. Extensive experiments have demonstrated that our proposed method realizes noticeable performance in terms of detection accuracy and inference speed compared with baseline YOLOv8n and many end-to-end detectors of similar model size. Concretely, THDet achieves 0.447 at the overall evaluation metric of mAP0.5−0.95, accomplishing 3.2% detection accuracy improvement than YOLOv8n. Besides, THDet only holds 2.2M parameters with 295 FPS inference speed, reducing 33.4% parameters compared with YOLOv8n. The experimental results validate the effectiveness of our proposed method, showcasing that THDet outperforms the mainstream real-time detection algorithms in the terms of accuracy, inference speed and lightweight model design for traffic helmet object detection.