Accurate delineation of cardiac structures in CTA images remains challenging, due to the similar appearance of different components, the highly variable shape and intensity among individuals, and especially the vicinity of heart volumes and backgrounds (including fat tissues, great vessels, and livers). Therefore, we proposed an accurate heart volume segmentation method with landmark-based registration (LMR) and 3D fully convolutional network (3D-FCN). First, we defined uniformity constrained landmarks in the training images and then trained a regression forest model (RFM) to detect these landmarks in the testing image. Second, the registration between the landmarks in each training image and the test image was performed by a shallow neural network, which guided the label propagation from atlases to the test image. After the label fusion with majority voting, we finally constructed a 3D-FCN to further refine the boundary voxels with low voting values. In 22 cardiac CTA images, we compared our method with multi-atlas segmentation, active shape model, DeepMedic, LMR + DeepMedic, and the separately implemented LMR and 3D-FCN. The results demonstrated the superiority of our method for the segmentation of heart volumes, with the average accuracy, dice coefficient, and mean Hausdorff distance as 96.25%, 93.98%, and 2.12 voxels, respectively. Furthermore, the training efficiency of the constructed 3D-FCN was higher than that of the DeepMedic through the comparison of training time and convergence of loss. The proposed heart segmentation could provide an accurate region of interest (ROI) for not only the four heart chambers but also the major trunks of vessels and coronary arteries.