We report a new phase called clinohexagonal prism (CHP) that accounts for all the ground states of dipolar hard spheres prepared at any density. This phase merely consists of an oblique prismatic lattice with a hexagonal base. Our calculations show that at intermediate densities, a special close packed body-centered orthorhombic phase coincides with the CHP phase in the ground state for a wide density window. In the high packing regime, i.e., in the vicinity of the density of the hexagonal close packed phase, it is a limiting case of the CHP phase with vanishing obliquity that emerges. These findings provide a unified and clarified view of the solid-solid transitions occurring at zero temperature in dipolar systems and should be relevant in other related molecular or soft matter systems governed by anisotropic (and possibly isotropic) soft potentials.