This study quantitatively examines the effects of land fragmentation and non-agricultural labor supply on the circulation of agricultural land management rights. The examination is conducted from the perspective of labor heterogeneity and family joint decision-making, using the rural fixed observation point data from the Ministry of Agriculture of the People's Republic of China. The results reveal that land fragmentation significantly affects circulation decisions of agricultural land circulation. Land fragmentation strengthens the effect of non-agricultural labor supply on agricultural land outflow, and this effect is more pronounced among females. Compared with males, the female non-agricultural labor supply has a greater effect on agricultural land circulation. When non-agricultural labor supply increases, the effect of the female non-agricultural labor supply on agricultural land circulation becomes significant, land outflows increase, and land inflows decrease. In the areas of eastern, central, and northeastern China, the female non-agricultural labor supply has a significant impact on agricultural land outflow. Furthermore, the number of land plots strengthens the effect of the non-agricultural labor supply on the outflows of agricultural land in eastern and northeastern China; this effect is more pronounced for females in northeastern China. The government and related departments should strengthen non-agricultural employment training, and design conditions and policies to promote the orderly transfer of household labor, thus achieving intense agricultural development in the process of human urbanization.