The Indo-Gangetic Plains (IGP) of India is dominated with rice − wheat cropping system that occupies almost 10.5 million ha area. The sustainability of rice-wheat system is under threat due to numerous water-, nutrients-, weeds- and environment-related problems, mainly, due to the cultivation of rice. Suitable crop and soil management practices with a bias to conservation agriculture (CA) that can sustain soil and environmental health as well as improve crop and water productivity, are required for the Indian IGP. Maize can be a viable alternative to rice and a potential driver for diversification of rice-wheat system. The acreage of maize is on the increase in conventional and conservation agriculture-based cereal systems of India in recent years. Therefore, a field experiment, involving a maize (Zea mays L.)-wheat cropping system was undertaken on a sandy clay loam soil for three years (2010–11 to 2012-13) in New Delhi to evaluate the impacts of CA on crop and water productivity, profitability and soil organic carbon (SOC) accumulation. There were five CA-based treatments in first year, and two treatments were introduced in second year (2011-12) onwards. The experiment was laid out in a randomized complete block design with three replications. In all the residue retention plots, wheat residue was retained in maize crop and maize residue was retained in wheat crop under zero till conditions. Results showed that the plots under permanent broad bed with residue (PBB+R) and without residue (PBB) resulted in ∼29 and ∼26% higher maize grain yield, respectively than conventional tillage (CT) (2.6tha−1), but wheat grain yields were comparable in all the treatments in first year. Maize grain yield in second year under PBB+R and zero tillage with residue (ZT+R) were 55 and 43% higher than CT plots (2.8tha−1). Three-year mean maize yields due to PBB+R and permanent narrow bed with residue (PNB+R) were 28 and 15% higher than that in CT plots (3.3tha−1). The PBB+R resulted in 11% higher two-year mean water productivity in maize than PBB (∼without residue), but both these treatments were comparable in this regard in wheat. The ZT+R plots resulted in 14% and 22% higher two-year mean water productivity, respectively in maize and wheat than ZT plots. Overall, the plots under PBB+R had 57% and 19% higher mean water productivities in maize and wheat, respectively compared with CT plots. Again, the PBB+R plots gave 12% higher two-year mean net returns compared with CT plots. With regard to net returns, the plots under permanent narrow bed with and without residue (PNB+R; PNB) were inferior to PBB, PBB+R, ZT and ZT+R plots. Retention of both-season crop residues could significantly improve SOC concentration in surface (0–5cm) soil. The PBB+R resulted in highest SOC pool at 0–30cm soil layer, which was significantly higher than that in CT. This system showed maximum carbon sequestration potential. Thus, this CA practice, which involves PBB+R is superior to other practice. This would save water through higher water-use efficiency, and lead to accumulation of more carbon in soil with higher sequestration potential, besides giving sustainable production through maize-wheat system over the years. This can be adopted across the IGP regions of India, where irrigated rice-wheat system is in practice, and in similar agro-ecologies of the tropics and sub-tropics under irrigated conditions.