High crystalline quality of AlN layers are commonly grown at temperatures above 1200 °C. However, this work revealed that introducing a rough surface of AlN layer, defined as dislocation filtering (DF) herein, can improve the crystalline quality of 1.5 μm thick AlN layers grown at 1175 °C via metal organic vapor phase epitaxy technique. With 7 slm of ammonia flow during the DF growth, the lowest threading dislocation density (TDD) of ∼2.9 × 109 cm−2 was obtained. The result was associated to the formation of 3D growths which become reasonably smaller, denser, and more uniformly distributed with increasing ammonia flow up to 7 slm. The 3D growths were suggested to promote lateral growth that can increase the possibility for TD inclination and hence, mutual annihilation. The 3D growths characteristic disappeared when the ammonia flow was 10 slm. The optimized AlN layer that was obtained with 7 slm of ammonia flow exhibited an atomically smooth surface with a few deep trenches and improved transparency. Meanwhile, the calculated in-plane tensile strain for the AlN layers grown on DF with 1–7 slm of ammonia was about 3 times higher than the AlN layer without DF. However, no evidence of microcracks on the surface was observed.