There is an urgent need to reduce carbon emissions from the building sector. This study focused on the whole life carbon of a building’s heating, ventilation, and air conditioning system. A methodology to compare the whole life carbon of different HVAC systems was proposed and used in a case study with boundary conditions in Denmark. All-air system and radiant systems were compared, as they have different working principles and the potential for differences in both their embodied and operational carbon. The radiant system was a Thermally Active Building System (TABS), and the all-air system was a packaged variable-air-volume system with reheat. The building model was based on the medium-sized office of prototype buildings developed by the U.S. Department of Energy. Life cycle stages of the building were classified based on EN15978:2011. Two models, one for dynamic building simulation and the other for measuring the mass of materials (e.g., concrete), were adopted in a novel approach. The operational carbon of the HVAC systems was calculated under very similar indoor thermal comfort conditions. The whole life carbon was 10.1 kgCO2-eq/m2/year and 9.0 kgCO2-eq/m2/year for the all-air system and TABS, respectively. Compared to the all-air system, TABS reduced annual total primary energy use by 34% and whole life carbon by 11%. If dynamic carbon intensity of the grid were to be implemented, further reduction of carbon emission is expected with TABS, owing to its flexibility in operation with the activated thermal mass.