Natural vegetation restoration and tree plantation are the two most important measures for ecosystem restoration on the Loess Plateau of China. However, few studies have compared the effects of the two contrasting measures on soil organic and inorganic carbon (SOC and SIC) sequestration or have further used SOC and SIC isotopes to analyze the inherent sequestration mechanism. This study examined a pair of neighboring small watersheds with similar topographical and geological backgrounds. Since 1954, natural vegetation restoration has been conducted in one of these watersheds, and tree plantation has been conducted in the other. The two watersheds have now formed completely different landscapes (naturally restored grassland and artificial forestland). Differences in soil bulk density, SOC and SIC content and storage, and SOC and SIC δ13C values were investigated in the two ecosystems in the upper 1 m of the soil. We found that SOC storage was higher in the grassland than in the forestland, with a difference of 14.90 Mg ha− 1. The vertical changes in the δ13CSOC value demonstrated that the two ecosystems have different mechanisms of soil surface organic carbon accumulation. The SIC storage in the grassland was lower than that in the forestland, with a difference of 38.99 Mg ha− 1. The δ13CSIC values indicated that the grassland generates more secondary carbonate than the forestland and that SIC was most likely transported to the rivers from the grassland as dissolved inorganic carbon (DIC). The biogeochemical characteristics of the grassland were favorable for the formation of bicarbonate. Thus, more DIC derived from the dissolution of root and microbial respired CO2 into soil water could have been transported to the rivers through flood runoff. It is necessary to study further the transportation of DIC from the grassland because this process can produce a large potential carbon sink.