The heat transfer performance of coaxial ground heat exchangers (GHEs) can be strongly influenced by their structure parameters and improved by the optimal design. To improve the heat transfer efficiency, this study proposed a coaxial GHE with large pipe diameters and thick inner pipe insulation for shallow ground source heat pump systems, which had heat transfer rates of more than 120 W/m in 12 h during both heat injection and extraction experiments. The validated three-dimensional CFD model was used to numerically investigate the influence of various structural parameters on the heat transfer efficiency considering the thermal short-circuiting effect. The thermal short-circuiting effect could be alleviated with thicker inner pipe insulation, while the decreasing annular space occupied by the insulation would reduce the borehole heat transfer rates, thus, an inner pipe with a 35 mm-thick insulation obtained the optimal heat transfer rate. The inner pipe diameter increase contributed to a more severe thermal short-circuiting and poor heat transfer performance. Increasing the outer pipe diameter improved the heat transfer efficiency and an optimal outer pipe diameter of 155 mm existed when considering the heat transfer performance and its efficiency. Therefore, increasing the inner pipe insulation thickness and outer pipe diameters can be regarded as an efficient method to improve the thermal performance of coaxial GHEs.