The process of vegetation restoration is often accompanied by significant changes in aboveground plant diversity. To explore the driving mechanism of litter nutrient-soil nutrient-enzyme activity stoichiometry on aboveground vegetation change is of great importance for maintaining regional biodiversity conservation and ecological stability. Taking typical abandoned farmland of different restoration years (1, 8, 16, 31, and 50 a) in the Qinling Mountains as the research object, the variation characteristics of plant community diversity during vegetation restoration were analyzed through field investigation. Litter nutrients, soil nutrients, and the activities of five extracellular enzymes, including β-1,4-glucosidase (BG), cellobiohydrolase (CBH), β-1,4-N-acetylglucosaminidase (NAG), leucine aminopeptidase (LAP), and acid phosphatase (AP), were determined. The characteristics of litter nutrients, soil nutrients, and enzyme stoichiometric ratios during vegetation restoration and the driving mechanism of plant diversity changes were discussed. The results showed that the plant community diversity index firstly decreased and then increased with the increase in vegetation restoration years, and the minimum was reached at 16 years after restoration. The results of principal component analysis showed that there were significant differences between total plant community diversity index and litter-soil-enzyme stoichiometric characteristics in different years of vegetation restoration. The plant community diversity index had a strong positive correlation with litter C∶P ratio and litter N∶P ratio but had a negative correlation with soil enzyme C∶P ratio (EEA C∶P). The results of redundancy analysis showed that soil EEA C∶P had the highest explanation rate of plant diversity changes during vegetation restoration (25.93%), followed by soil TP (5.94%), which was the key factor regulating plant diversity changes. In conclusion, plant species and quantity increased significantly in abandoned farmland in the middle part of the Qinling Mountains at the late stage of vegetation restoration. Changes in the soil environment affected microbial metabolic activities and thus changed enzyme activities. Litter-soil-soil extracellular enzymes affected the community environment and plant diversity through feedback and regulation. EEA C∶P and TP were the main driving factors of aboveground plant diversity change during vegetation restoration.