The carbon generalized system of preferences (CGSP) is an innovative incentive mechanism implemented by the Chinese government, which has also become an important part of carbon emission reduction at the living end, and it is of great significance to study whether the Pilot Policy can reduce the carbon emissions of residents. This study firstly accounts for the total carbon emissions and per capita carbon emissions of the residents of 284 cities in China, and on this basis, adopts the SCM method to quantitatively study and analyze the overall and local implementation effects of CGSP in China by taking the first batch of CGSP pilots in China as an example, and further applies the mediation effect model to test the pathways of the role of CGSP. The main findings of the study are as follows: (1) During the period of 2010–2020, the total carbon emissions from urban residents' living in China showed a yearly growth trend, from 36,623.98 ×10−2Mt in 2010–85,241.20 ×10−2Mt in 2020, an increase of 8.83%. Total carbon emissions present a structural difference of "electricity consumption > central heating > private transport > gas (oil, natural gas) consumption". (2) Overall, the implementation of the CGSP had a robust positive impact on the overall carbon emission reduction in the pilot cities, with an average annual emission reduction effect value of 36.53 ×10−2Mt. Locally, the annual net policy effect values of Dongguan, Zhongshan, Heyuan, and Guangzhou are 6169.79 ×10−2, 26,600.17 ×10−2, 17,081.34 ×10−2 and 9393.36 ×10−2Mt respectively. (3) CGSP has a good carbon emission reduction effect by suppressing the impact on residents' carbon emissions through enhancing the city's innovation capacity and promoting electricity saving and consumption reduction, while the mediating effect played by the promotion of green and low-carbon travel in the pilot policy is not significant. Finally, based on the research findings, relevant suggestions are targeted.