Micro-heterogeneity in low-permeability sandstone oil reservoirs significantly influences the uniformity of water injection during development. This leads to the formation of preferred channels for water injection, causing premature water breakthroughs in oil wells. Not only does this reduce oil displacement efficiency, but it also enriches residual oil in the formation, which ultimately impacts the overall recovery rate. This study employed various methods, including thin-section casting, qualitative analysis through scanning electron microscopy, quantitative analysis of X-ray diffraction, high-pressure mercury intrusion and particle size, and experimental techniques, such as wettability and micro-displacement, to investigate the impact mechanism of micro-heterogeneity on water injection development in low-permeability oil reservoirs. A typical low-permeability sandstone oil reservoir in the Ordos Basin was used as a case study. The results reveal that the reservoir’s micro-heterogeneity is determined by the heterogeneity of the interstitial material, porosity, and particle size. Micro-heterogeneity plays a critical role in the flow characteristics and oil displacement efficiency of low-permeability oil reservoirs. The less the micro-heterogeneity, the better the water injection development outcome. This study suggests a technical policy adjustment method that is critical for guiding the development of low-permeability water injection oil reservoirs, thereby improving the effectiveness of water injection development.