Improving irrigation efficiency (IE) to alleviate water shortage has been deemed viable. However, the so-called irrigation efficiency paradox (IEP) can complicate this process, where agricultural water consumption remains high despite the enhanced irrigation efficiency, particularly in arid regions. This study developed a judgment method of IEP based on a water accounting framework and the IEP definition to detect the IEP occurring across spatial scales in the middle Heihe River Basin, the second-largest inland river basin in Northwest China. Furthermore, the scale effects of the IEP were analyzed. The results showed that evaluated by evapotranspiration (ET) and transpiration (T), the scale effect occurred because IEP does not necessarily occur at the field scale but at the irrigation district scale and basin scale. Evaluated by Inflows (I) and surface water diversion and groundwater extraction (Isg), the scale effect occurred because IEP may occur at the field and the irrigation district scales but not at the basin scale. Further, the IEP was also influenced by time scales and regional agricultural characteristics. The results can be referenced for water-saving measures, avoiding the potential IEP for actual water savings in arid regions worldwide.