Global warming has become one of the major threats to the security of human survival, security, and sustainable development. Agricultural production has been widely suspected as one of the main sources of anthropogenic carbon emissions. Analyzing the changing characteristics and influencing factors of agricultural carbon emissions is of great significance for the mitigation of global climate change and the sustainable development in agriculture. Taking China, a large agricultural country, as an example, this study used the empirical model to quantify carbon emissions from agricultural inputs from 1991 to 2019, and analyzed the driving factors using ridge regression. We found that agricultural carbon emissions in China have been on the rise in the past 30 years, but at a markedly slower pace. From 2008 to 2019, the average annual growth rate of agricultural carbon emissions was 1.47%, down significantly from 2.92% between 1991 and 2007. The carbon emissions per unit of planting area showed an overall increasing trend, which grew from 179.35 t ce/km2 to 246.26 t ce/km2, with an average annual growth rate of 1.13%. The carbon emissions per unit of agricultural output mainly showed a decreasing trend, which decreased from 0.52 kg ce/CNY to 0.06 kg ce/CNY, with an average annual rate of change of −7.42%. China’s agricultural carbon emissions were closely related to macro-policies. Fertilizer inputs, agricultural industry structure, and energy use intensity were significantly positively correlated with carbon emission intensity. The degree of urban feedback to rural areas, public investment in agriculture, and large-scale planting were significantly negatively correlated with carbon emission intensity, but the impacts of these factors had a “lag effect”. In order to reduce carbon emissions from agriculture and promote development in green agriculture, we suggest that the government should further increase the degree of urban feedback to rural and public investment in the agricultural sector. In addition, large-scale agricultural production should be encouraged to increase resource efficiency and reduce carbon emissions.