Characterization of soil organic carbon (SOC) in terms of size, storage inside aggregates and chemical recalcitrance is vital to understand mechanisms of its stabilization and to devise climate smart agricultural management practices. Tillage and residue retention are known to influence carbon (C) storage within soil aggregates and its accumulation as particulate organic matter as well as organomineral complexes. However, the effect of tillage and crop residue retention on C stabilization in coarse textured soils through various mechanisms is not well understood. We studied the effect of conservation agriculture involving no tillage with surface residue mulch (NTM) in maize-wheat sequence on particulate (POC) and mineral associated organic C (MinOC), C storage within aggregates and acid non-hydrolysable C (NHC) in a sandy loam soil. Compared to conventional tillage without residue retention (CTM0), the NTM improved SOC stocks by 23% in top 15-cm soil and significantly increased coarse POC (∼92 to 284%) and fine POC (67 to 123%) with relatively little effect on MinOC. This indicated that MinOC had relatively small contribution towards SOC stabilization in coarse textured sandy loam soils with limited potential to form organomineral complexes. The results further showed that the effects of NTM were brought about by improved aggregate stability and C preservation inside macroaggregates of size >1 mm. Furthermore, greater amount of SOC (2.64 g C kg−1) and macroaggregate associated C (0.35–0.59 g C kg−1) occurred in recalcitrant forms (NHC) under NTM compared to conventional (CT) and deep tillage (DT). The NTM also impacted the belowground C input through improved root length density (RLD) and development of fibrous roots expressed as specific root length (SRL), which influenced SOC build-up and stabilization. It is concluded that compared to the existing practice of CTM0, the conservation agriculture involving NT with residue retention leads to SOC sequestration in coarse textured soils. Its large scale adoption, besides helping in climate change mitigation will lead to soil health improvement.