Ground-coupled heat pumps (GCHP) are highly impactful and promising systems for extracting geothermal energy. Nevertheless, the effectiveness of GCHP can be further improved by reshaping and optimizing the configuration of the coaxial ground heat exchanger (CGHE), which represents the core component of the GCHP and has a significant influence on the heat extraction performance of the system. This paper introduces a detailed comparative numerical study on a novel CGHE with an oval cross-section for enhancing the performance of the GCHP. The proposed design's simplicity stems from altering the outer tube to an oval shape, compared to the commercially used circular CGHE. The conjugate fluid flow and heat transfer are simulated utilizing a three-dimensional numerical model. Heat flux, GHE thermal resistance, and pressure drop within the tube are computed and analyzed. A comparison between circle and oval cross-sections is carried out. The adopted 3-D model shows a fit verification with their experimental pairs via a field experiment. Parametric study and sensitivity analysis are performed, and the effect of groundwater flow on heat transfer is also examined. An optimization approach is developed to explore the optimal inner tube position and incorporate multi-inner tubes that attain the maximum heat transfer efficiency of the oval-CGHE. The results reveal that oval-CGHE significantly surpasses the conventional circle-CGHE, improving the maximum and average heat transfer by 21.03% and 10.24%, corresponding to a reduction in thermal resistance by 28.83% and 9.06%, respectively. Additionally, optimizing the inner tube position and incorporating multi-inner tubes further enhances the performance of oval-CGHE by maximum and average heat transfer rates of 33.01% and 22.91% for inner tube position optimization and 39.08% and 28.84% for multi-inner tube optimization, respectively.