Widening the range of organic nutrient resources, especially N sources, is a major challenge for improving crop productivity of smallholder farms in southern Africa. A study was conducted over three seasons to evaluate different species of indigenous legumes for their biomass productivity, N 2-fixation and residual effects on subsequent maize crops on nutrient-depleted fields belonging to smallholder farmers under contrasting rainfall zones in Zimbabwe. Under high rainfall (>800 mm yr −1), 1-year indigenous legume fallows (indifallows), comprising mostly species of the genera Crotalaria, Indigofera and Tephrosia, yielded 8.6 t ha −1 of biomass within 6 months, out-performing sunnhemp ( Crotalaria juncea L.) green manure and grass (natural) fallows by 41% and 74%, respectively. A similar trend was observed under medium (650–750 mm yr −1) rainfall in Chinyika, where the indifallow attained a biomass yield of 6.6 t ha −1 compared with 2.2 t ha −1 for natural fallows. Cumulatively, over two growing seasons, the indifallow treatment under high rainfall at Domboshawa produced biomass as high as 28 t ha −1 compared with ∼7 t ha −1 under natural fallow. The mean total N 2 fixed under indifallows ranged from 125 kg ha −1 under soils exhibiting severe nutrient depletion in Chikwaka, to 205 kg ha −1 at Domboshawa. Indifallow biomass accumulated up to 210 kg N ha −1, eleven-fold higher than the N contained in corresponding natural fallow biomass at time of incorporation. Application of P to indifallows significantly increased both biomass productivity and N 2-fixation, translating into positive yield responses by subsequent maize. Differences in maize biomass productivity between indifallow and natural fallow treatments were already apparent at 2 weeks after maize emergence, with the former yielding significantly ( P < 0.05) more maize biomass than the latter. The first maize crop following termination of 1-year indifallows yielded grain averaging 2.3 t ha −1, significantly out-yielding 1-year natural fallows by >1 t ha −1. In the second season, maize yields were consistently better under indifallows compared with natural fallows in terms of both grain and total biomass. The first maize crop following 2-year indifallows yielded ∼3 t ha −1 of grain, significantly higher than the second maize crop after 1-year indifallows and natural fallows. The study demonstrated that indigenous legumes can generate N-rich biomass in sufficient quantities to make a significant influence on maize productivity for more than a single season. Maize yield gains under indifallow systems on low fertility sandy soils exceeded the yields attained with either mineral fertilizer alone or traditional green manure crop of sunnhemp.