To vigorously promote the integrated development and mutual adaptation of agriculture and logistics is an important way to realize agricultural modernization and rural revitalization. Along with the policy support of agricultural industry chain and the steady rise of the demand for agricultural products market, the total amount of agricultural product logistics continues to increase, and the growth rate remains stable. With the booming development of "Internet + agriculture" and e-commerce platform, agricultural logistic market welcomes a new round of development opportunities, reaching several trillion yuan. Compared with the developed countries, our agricultural product logistics is still far behind. At present, only 15% of vegetables and fruits and 30% of fresh meat have professional logistic transportation, while most of the rest are still in the state of local and primitive. The gap of logistic environment construction and logistic elements determines the difference in final benefit of agricultural products. The purpose of this study is to find out the influence of logistic elements on regional agricultural economic growth, and whether the influence between neighboring regions presents "the same prosperity", "the same loss" or "sharing weal and woe". Based on the panel data of the statistical yearbook of 31 provinces in China from 2005 to 2020, the spatial Durbin model was constructed under the spatial weight matrix of economic distance and economic geographical distance to conduct empirical analysis, and the internal factors of logistics industry, factor spillover effect and its impact on agricultural economic growth were studied. Results showed that: (1) considering economic distance factor, the spatial coefficient of the time-fixed effects model passed the significance test in eastern China. Considering economic geographic distance factors, the individual and double fixed effect models passed the significance test in central China, and all models passed the significance test in western China. (2) From the perspective of logistic infrastructure, AVLFA, HM, TN and RM had a positive effect on the growth of agricultural economy in eastern China, but LIAV is on the contrary. AVLFA had a positive effect on agricultural economic growth, but TN was on the contrary in central China. In western China, LIAV and TN promoted agricultural economy while HM and RM held back it. From the perspective of the volume of logistics activity, both eastern and central regions did not pass the significance test, but FA was tested by the double fixed effect model and showed negative in western China. From the perspective of control variables, FU, AO and PT all promoted agricultural economy in eastern China, and FU and AO did the same in central and western China. PT was invalid in central regions and hindered agricultural economy in western China, which was different. From the perspective of spatial spillover effect decomposition, the eastern region presents "one prosperity and all prosperity, and sharing weal and woe", while the central and western regions present "one prosperity and all prosperity, and one lost and all lost". At last suggestions as formulating the overall plan for the development of regional logistics, paying attention to regional differences and promoting coordinated development of logistics and agriculture in light of local conditions, and paying attention to the spatial spillover effect of elements were put forward.