In rural China, the conversion between fine-grained farmland and large-scale farmland is a widespread phenomenon, changes in the size of farmland can have an impact on agricultural carbon emissions. Based on the agricultural panel data of Gansu Province for 2000–2020, taking the scale of agricultural land management as the breakthrough point and the consumption intensity of agricultural materials as the intermediary factor, this paper discusses the driving mechanism of agricultural carbon emissions by the scale of agricultural land management. The results including: (1) From the perspective of the intermediary effect, large-scale farmers pay more attention to input efficiency of chemicals such as fertilizers than small-scale farmers, which can effectively promote the development of low-carbon agriculture. (2) A “U-shaped” relationship existed between agricultural land management scale and agricultural carbon emissions. The agricultural carbon emissions were the lowest when agricultural land management scale in Gansu was 0.608 hm2/person. (3) The carbon emission intensity reached its peak when the scale of farmland management in the Hexi region was 0.143 hm2/person. The optimal scale of farmland management in the Longdong and Longnan regions was 0.143 and 0.348 hm2/person, respectively, Longzhong and Gannan regions was all showed complete intermediary effects.