In recent years, artificial intelligence technology has been widely used to assist radiologists in diagnosing and analyzing medical images. The use of artificial intelligence technology can well assist doctors in the localization of lesions. However, the mainstream target detection models at this stage are difficult to be practically applied in medical systems because of factors such as the use of large backbone networks and high input resolution, which leads to low model accuracy and high consumption of computational resources. In this paper, we propose a fast detection speed and high accuracy of the lung lesion detection network IAB- RefineDet. By improving the channel and spatial attention mechanisms and introducing the improved attention module into RefineDet, the lesion detection accuracy is dramatically improved without significantly increasing the number of parameters. We conduct extensive experiments on VinDr-CXR, the world's largest publicly available chest radiograph detection dataset, and comparative experiments with existing mainstream target detection models. The experimental results show that IAB-RefineDet achieves a mAP of 16.23%, and the lesion detection performance is significantly better than mainstream deep learning models.