Urban and rural relations are important for the sustainable development of a nation or region. Urban and rural integration (URI), as a conceptual framework and strategic tool for managing urban–rural relations, is supposed to play a role in linking urban and rural socio-economic systems to promote the two-way flow of resources, facilitate spatial restructuring and functional transformation, enhance resource allocation efficiency, and shorten the gap between urban and rural areas. This article takes agriculture technological innovation in China as an example, and discuss whether and how URI can promote agricultural technological innovation (ATI). Based on a dataset of 288 prefecture-level cities spanning the years from 1999 to 2018, this article analyzes the mechanism and effect of URI on the development of ATI by using the negative binomial regression model and other models based on measuring the level of URI in the municipal area. The main conclusions are as follows. (1) URI positively promotes the development of ATI and exhibits significant heterogeneity across various dimensions, regions, and agricultural production areas. (2) The mechanism analysis shows that governance systems and mature markets positively moderate the impact of URI on promoting ATI, which also exhibits heterogeneous characteristics across various regions. (3) The impact of URI on the development of ATI exhibits a double threshold effect, and its marginal impact on the development of ATI increases with the deepening of URI. Moreover, the threshold effect of URI demonstrates significant heterogeneity. In central and urban areas, after surpassing the threshold value, the influence of URI on agricultural technological innovation becomes greater. Therefore, it is crucial to continuously deepen the development of URI, smooth the channels of factor flow, enhance resource sharing, break down the urban and rural barriers, and create a new engine for ATI development.