Ground Source Heat Pumps (GSHPs) are used for space heating and cooling, where the Ground Heat Exchangers (GHEs) are used to extract or reject heat from/to the ground. GHEs come in various configurations, vertical or horizontal. Compared to Air Source Heat Pumps (ASHPs), GSHPs, albeit having a higher installation cost, achieve a better coefficient of performance (COP) and, hence, electricity savings. This reduction in consumed energy is translated to a reduction in fossil fuels and environmental “harmful” gas emissions. As the environmental impact does not lie in a single aspect, it would be useful not to stop the discussion in terms of COP and cost, but to identify whether a GSHP system is indeed a sufficiently overall greener solution. Hence, a more comprehensive investigation on the environmental impact of different types of GHEs as part of a GSHP compared to an ASHP system is attempted in the current study. A case study of a residential building with a fixed heating and cooling load is considered for moderate climate conditions. Using GLD software, a GSHP system is studied for various GHE configurations. The system undergoes a Life Cycle Analysis (LCA), with the yearly heating and cooling load as functional unit, for a direct environmental impact comparison between the GHE configurations and an ASHP system. The openLCA software is used for the application of two different methods, namely CML2001 and Eco-Indicator99 for seven and three impact categories respectively. It is concluded that the ASHP system exhibits the highest impact for all the main categories, while among the GSHP systems the vertical coaxial GHE configuration exhibits the highest impact and the horizontal GHEs the lowest, at times significantly below the ASHP impact. The largest reduction among all categories for the vertical and horizontal GHEs respectively is about 22% and 24% for the CML2001, and 17% and 21% for the Eco-Indicator99.