Soil organic carbon (SOC) is an important pool in the global carbon cycle, playing a vital role in moderating atmospheric CO2 concentrations. Soil is the largest carbon pool in terrestrial ecosystems and, as the basic unit of soil structure, soil aggregates are key to protecting soil carbon pools. However, the influence of soil aggregate particle size, SOC distribution, and the contribution of different particle sizes to SOC is still unclear, particularly under different land use types. In this study, soil samples were collected from five land use types (slope farmland (SF), forest (FL), grassland (GL), shrubland (SL), and terraced field (TF)) in a typical small watershed in the Loess Plateau, China. We analyzed the soil aggregate particle size, composition, and stability after dry and wet sieving, SOC content of different particle sizes in aggregates, and the effects of different aggregate particle sizes and land use on the distribution of SOC. The results showed that, the surface soil (0 ∼ 20 cm) water-stable aggregates were relatively stable, particularly under FL, the mean weight diameter (MWD) value was 2.16 mm. Deep soil (40 ∼ 60 cm) non-water stable aggregates were more stable, and GL was optimal, the MWD value was 3.94 mm. SOC and total nitrogen (TN) were significantly correlated with water-stable aggregate stability indicators (p < 0.01). The SOC content in the small aggregates (0.25 ∼ 2 mm) was the highest and lowest in the microaggregates (<0.25 mm) under different land use. The carbon/nitrogen (C/N) ratio of microaggregates was significantly higher in SF (the C/N ratio was 23.17) and GL (the C/N ratio was 31.04) than in other land uses (p < 0.01) in the 20–40 cm soil layer. In surface soil, small aggregates contributed>50% to SOC in SF and TF. In deep soil, small aggregates under all five land uses made the largest contribution rate to SOC at 57%. These findings indicate that carbon sequestration in the study area can be improved by a combination of appropriate soil management with ecological construction to increase the content of small aggregates, strengthen the fixation and protection of SOC, and reduce carbon emissions in the soil.