In arid coal mining areas, plants face significant soil moisture deficiency and inadequate water use strategies, hindering the ecological recovery process in the region. Through indoor soil column and growth chamber experiments, we investigated the effects of two homogeneous (S, a profile with entirely sand from 0 to 100 cm; L, a profile with entirely loess from 0 to 100 cm) and three layered soil profiles (SLS1, loess from 20 to 40 cm, with the rest being sand; SLS2, loess from 40 to 60 cm, with the rest being sand; SLS3, loess from 60 to 80 cm, with the rest being sand) on plant growth and root development. Additionally, we compared the impact of inoculating arbuscular mycorrhizal fungi (AMF, Funneliformis mosseae) in homogeneous soil (S) and layered soil profile (SLS1) on soil moisture at different depths, plant root distribution, and water use strategies. The research results indicate that the layered soil profile (SLS1) exhibits favorable infiltration and evaporation characteristics, leading to a significant increase in plant biomass. Furthermore, both the layered soil profile (SLS1) and the inoculation of AMF can reduce the moisture absorption proportion of alfalfa in the shallow soil (0–20 cm) (>10%), increase the utilization proportion of water in the intermediate-deep soil, and enhance plant water use efficiency. Implementing the layered soil profile (SLS1) and AMF treatment proves effective in addressing soil moisture deficiency, improving plant water use strategies, and enhancing water use efficiency. The combined application of these two approaches is more suitable for the growth of plants in arid coal mining areas.