A systematic ab initio calculations based on a complete Configuration Interaction (CI) including the pseudo-potential model where the core valence correlation was presented by the Core Polarization Potential (CPP), were employed to determine the geometric structures of CaXen (n = 1–4) neutral clusters. For the diatomic (CaXe) and tri-atomic (CaXe2) systems, the potential energy curves for the ground and several excited states are determined and the different spectroscopic constants are illustrated. Interested structures were seen in the PECs which their effects in the vibrational levels' spacing and the electric dipole moments variations were shown. The agreement with the experimental and the theoretical constants for n = 1 was excellent. Accordingly, for the higher clusters the predicted geometries should be reliable. The bent geometry of CaXe2 ground state with (C2v) symmetry was performed and hence was concluded to be the most adaptable with a bond angle of 54.84° and bond distance dXe-Xe of 8.08 a0. As well as, the bent isomer for the excited states was also studied to illustrate the most compact geometry for each state. For the ground state of CaXe3, the most stable geometry was trigonal-pyramidal with (C3v) symmetry with dXe-Xe = 8.13 a0. And concerned n = 4, four configurations were treated; the tetrahedral structure, the square pyramid structure, the trigonal bipyramid structure with (C3v) symmetry and the trigonal bipyramid structure with (C2v) symmetry. The comparison between their minimal energies showed that the most compact geometry for CaXe4 is the trigonal bipyramid with (C3v) symmetry with dXe-Xe = 8.17 a0. Consequently, this work illustrates that the mechanism of cluster forming is consistent with the calcium location on the xenon clusters' surface model.