Use of perennial grasses as vegetative barriers to reduce soil erosion from farm and non-farm lands is increasing world-over. A number of perennial grasses have been identified for their soil conserving properties, but their effectiveness varies with location and method of planting. Installing vegetative barriers in combination with suitable mechanical measures, like bunds or trenches or both, on the appropriately spaced contours may enhance their conservation potential. Hence, the effect of vegetative barriers, viz., sambuta (Saccharum spp.)—a local grass, vetiver (Vetiveria zizanioides) and lemongrass (Cymbopogon citratus) planted in combination with trench-cum-bund, on runoff, soil loss, nutrient loss, soil fertility, moisture retention and crop yield in the rainfed uplands, was studied in Kokriguda watershed in southern Orissa, India through 2001–2005. However, runoff, soil and nutrient losses were studied for 2002, 2003 and 2004 only. Analysis of the experimental data revealed that on a 5% slope, the lowest average runoff (8.1%) and soil loss (4.0 Mg ha−1) were observed in the sambuta + trench-cum-bund treatment followed by vetiver + trench-cum-bund (runoff 9.8%, soil loss 5.5 Mg ha−1). Lemongrass permitted the highest runoff and soil loss. Further, the conservation effect of grass barriers was greater under bund planting than berm planting. Minimum organic C (50.02 kg ha−1), available N (2.49 kg ha−1) and available K (1.56 kg ha−1) loss was observed under sambuta with bund planting. The next best arrester of the soil nutrients was vetiver planted on bund. Significantly better conservation of nutrients under sambuta and vetiver resulted in the soil fertility build-up. Soil moisture content was also higher in the sambuta and vetiver than lemongrass treated plots. Increase in the yield of associated finger millet (Eleusine coracana (L.) Gaertn.) due to vegetative barriers ranged from 18.04% for lemongrass to 33.67% for sambuta. Further, the sambuta and vetiver treated plots produced 13.23 and 11.86% higher yield, respectively, compared to the plots having lemongrass barrier (1.17 Mg ha−1). Considering the conservation potential, and crop yield and soil fertility improvements, the sambuta barrier with trench-cum-bund is the best conservation technology for treating the cultivated land vulnerable to water erosion. Farmers also showed greater acceptance for the sambuta barrier as it is erect growing and available locally. Vetiver with-trench-cum bund can be the second best option.