Arid and semi-arid ecosystems dominated by shrubby species are an important component in the global carbon cycle but are largely under-represented in studies of the effect of climate change on carbon flux. This study synthesizes data from long-term eddy covariance measurements and experiments to assess how changes in ecosystem composition, driven by precipitation patterns, affect inter-annual variability of carbon flux and their components in a halophyte desert community dominated by deep-rooted shrubs (phreatophytes, which depend on groundwater as their primary water source). Our results demonstrated that the carbon balance of this community responded strongly to precipitation variations. Both pre-growing season precipitation and growing season precipitation frequency significantly affected inter-annual variations in ecosystem carbon flux. Heavy pre-growing season precipitation (November–April, mostly as snow) increased annual net ecosystem carbon exchange, by facilitating the growth and carbon assimilation of shallow-rooted annual plants, which used spring and summer precipitation to increase community productivity. Sufficient pre-growing season precipitation led to more germination and growth of shallow-rooted annual plants. When followed by high-frequency growing season precipitation, community productivity of this desert ecosystem was lifted to the level of grassland or forest ecosystems. The long-term observations and experimental results confirmed that precipitation patterns and the herbaceous component were dominant drivers of the carbon dynamics in this phreatophyte-dominated desert ecosystem. This study illustrates the importance of inter-annual variations in climate and ecosystem composition for the carbon flux in arid and semi-arid ecosystems. It also highlights the important effect of changing frequency and seasonal pattern of precipitation on the regional and global carbon cycle in the coming decades.