Reducing the total CO2 emissions intensity (TCEI) in the Chinese construction industry is an important strategy to cut its CO2 emissions, which share a considerable proportion of sectoral CO2 emissions in China. However, there is lack of an in-depth analysis of the effect of production structure on the TCEI. This study comprehensively investigates the effect of production structure on the TCEI in the Chinese construction industry by using structural decomposition analysis and sensitivity analysis methods collectively. The results show that the TCEI in the Chinese construction industry declined by 0.3505 tons CO2/104 RMB between 2007 and 2012, and the effect of production structure contributed 110.66% to such decline. Over 95% of the effect was contributed by the electricity, steam and hot water production and supply industry and the refined petroleum, coke products, and nuclear fuel industry. The results also demonstrate that the three most critical technical coefficients were for the nonmetallic mineral products industry consumed by the construction industry, the metal smelting and pressing industry consumed by the construction industry, and the metal smelting and pressing industry consumed by itself in 2012. Besides the first two coefficients, the coefficient for the electricity industry consumed by itself was also critical in 2007. The production chains of the construction industry itself and the metal smelting and pressing industry included over half of the twenty most critical technical coefficients. This study can provide valuable reference for Chinese policy makers to take proper strategies to adjust the production structure in order to reduce the TCEI.