Myocardial perfusion imaging (MPI) with PET plays a vital role in the management of coronary artery disease. High sensitivity systems can contribute to maximizing the potential value of PET MPI; therefore, we have proposed two novel detector arrangements, an elliptical geometry and a D-shape geometry, that are more sensitive and more compact than a conventional large-bore cylindrical geometry. Here we investigate two items: the benefits of the proposed geometries for cardiac imaging; and the effects of scatter components on cardiac PET image quality. Using the Geant4 toolkit, we modeled four time-of-flight (TOF) PET systems: an 80 cm diameter cylinder, a 40 cm diameter cylinder, a compact ellipse, and a compact D-shape. Spatial resolution and sensitivity were measured using point sources. Noise equivalent count rate and image quality were examined using an anthropomorphic digital chest phantom. The proposed geometries showed higher sensitivity and better count rate characteristics with a fewer number of detectors than the conventional large-bore cylindrical geometry. In addition, we found that the increased intensity of the scatter components was a big factor affecting the contrast in defect regions for such a compact geometry. It is important to address the issue of the increased intensity of the scatter components to develop a high-performance compact cardiac TOF PET system.