A field experiment was conducted in the Loess Plateau of Northwest China to study the effects of plastic film mulching and straw mulching on soil water, soil organic carbon (SOC), total nitrogen (TN), microbial biomass carbon (MBC) and nitrogen (MBN), dissolved organic carbon (DOC) and nitrogen (DON), crop yield and water use efficiency under winter wheat (Tricicum aestivum L.)—summer maize (Zea mays L.) double-cropping system conditions using the following three cultural practices: (i) traditional plough with no mulching (CK), (ii) traditional plough with plastic film mulching (PM), and (iii) traditional plough with straw mulching (SM). Soil water contents were measured by the gravimetric method. SOC was determined using the dichromate oxidation method. TN was analyzed by the Kjeldahl method. MBC and MBN were determined using the chloroform fumigation extraction method. DOC and TDN were determined following Jones’ procedures proposed by Jones and Willett (2006). The results showed that soil water was higher under the PM treatment than under the SM treatment and mainly changed in the upper 60 cm soil layer. Compared with the CK treatment, the concentrations of SOC and TN under the SM treatment were increased by 16.9% and 7.7% at the 0–10 cm soil depth, respectively, and the PM treatment had the similar SOC and TN concentrations. Compared with the CK treatment, soil C:N ratio was increased under the SM treatment by 6.2% (P < 0.05), and that under the PM treatment was decreased by 5.2% (P < 0.05) after three years. The concentrations of MBC under the PM and SM treatments were significantly increased by 42.0% and 24.1%, respectively, and MBN under the PM treatment was significantly increased by 5.6% at 0–10 cm soil depth after the maize season. Compared with the CK treatment, DOC was significantly increased by 21.0% under the SM treatment and decreased by 13.1% under the PM treatment, and DON was significantly increased by 10.5% under the SM treatment and decreased by 4.3% under the PM treatment at the 0–10 cm soil depth after the maize season. Relative changes of labile soil organic carbon and nitrogen fractions were more sensitive than that of SOC and TN. The relative decline or increase of labile soil organic carbon and nitrogen fractions was on average almost 13.6% for the mulching practices. Compared with the CK treatment, the average maize yields under the PM and SM treatments were increased by 26.4% and 9.8%, and the average wheat yields under the PM and SM treatments were increased by 21.3% and 7.4%, respectively. The average water use efficiencies under the PM and SM treatments were 24.5%, 8.8% in winter wheat and 22.9%, 6.3% in summer maize higher than that under the CK treatment, respectively. Our results suggested that plastic film mulching could be used as an effective practice to improve low soil quality with adequate nitrogen and increase crop yield and water use efficiency in the Loess Plateau, China.