As China’s urbanization has shifted from high-speed to high-quality development, Urban Land Use Efficiency (ULUE) has become an important scale for evaluating urban connotative development. However, existing research has paid less attention to errors caused by different urban environmental factors and random disturbances in ULUE. Therefore, the purpose of this study is to eliminate the impact of environmental factors and random disturbances on ULUE measurement results by placing different cities under the same environmental conditions. First, a three-stage DEA envelopment analysis (T-DEA) model is introduced to calculate the ULUE of 16 prefecture-level cities in Anhui Province from 2001 to 2020. On this basis, the kernel density estimation model, gravity center model, and geographic detector models are used to study the spatial and temporal evolution and driving factors. The results show that (1) the ULUE increases nonlinearly with time, with an increase of 12.74%; (2) the overall peak of ULUE is on the rise, and changes from a single peak to a multi-peak, indicating that ULUE is constantly improving and that there is multi-level differentiation between different cities; (3) during the study period, the center of gravity of the ULUE value moved 22.66 km to the northwest; the overall moving distance was small, and the moving rate was slow; and (4) the influence of the interaction of double driving factors on ULUE is significantly greater than that of a single driving factor, and the factors of urban built-up area and degree of openness, as the key driving factors affecting ULUE, also have a degree of duality. In addition, to achieve efficient urban land use and to coordinate the environmental differences faced by different cities, the government must formulate systematic policies and development strategies considering the spatial characteristics of urban land use efficiency and the complexity of the driving factors.