Coronary heart disease is a disease that seriously endangers human health and life which is caused by plaque formation due to coronary artery atherosclerosis or spasm. The detection of coronary plaques through medical imaging is a non-destructive and fast diagnostic method, which holds significant medical and clinical value for the diagnosis of coronary heart disease. In this paper, We proposed an automatic detecting method for coronary artery vulnerable plaque based on CTA image sequence. We collected CTA image sequences from 132 patients and obtained the coronary artery tree by performing three-dimensional (3D) segmentation. The coronary artery centerline was extracted using the skeleton thinning method. To address the issue of insufficient features in a single CPR image, we constructed Curved Planar Reconstructed (CPR) images from multiple directions perpendicular to the coronary artery, following the centerline. In total, we established 8797 images in the training dataset and 1553 images in the test dataset. We proposed an attentive detection method that fused a new Cascade RCNN ResNet50-FPN neural network model with a convolutional attention module (CBAM). This method achieved automatic recognition and detection of plaques on the two-dimensional multi-angle fused CPR images. The proposed ResNet50-FPN network performed feature extraction, and CBAM was added after the first and last convolutional layers to enhance the network’s attention to plaque features. Our experimental results demonstrate that deep learning neural networks can provide a feasible approach to automatically detecting coronary artery plaques. Our method achieved a plaque detection accuracy of 94.6%, outperforming the RCNN method proposed by Zriek.