Carbon dioxide (CO2) is involved in plant growth as well as plant responses to abiotic stresses; however, it remains unclear whether CO2 is involved in the response of rice (Oryza sativa) to aluminum (Al) toxicity. In the current study, we discovered that elevated CO2 (600 μL·L−1) significantly alleviated Al-induced inhibition of root elongation that occurred in ambient CO2 (400 μL·L−1). This protective effect was accompanied by a reduced Al accumulation in root apex. Al significantly induced citrate efflux and the expression of OsALS1, but elevated CO2 had no further effect. By contrast, elevated CO2 significantly decreased Al-induced accumulation of hemicellulose, as well as its Al retention. As a result, the amount of Al fixed in the cell wall was reduced, indicating an alleviation of Al-induced damage to cell wall function. Furthermore, elevated CO2 decreased the Al-induced root nitric oxide (NO) accumulation, and the addition of the NO scavenger c-PTIO (2-(4-carboxyphenyl)-4,4,5,5-tetramethylimidazoline-1-oxyl-3-oxide) abolished this alleviation effect, indicating that NO maybe involved in the CO2-alleviated Al toxicity. Taken together, these results demonstrate that the alleviation of Al toxicity in rice by elevated CO2 is mediated by decreasing hemicellulose content and the Al fixation in the cell wall, possibly via the NO pathway.