Grass xylan consists of a linear chain of β-1,4-linked xylosyl residues that often form domains substituted only with either arabinofuranose (Araf) or glucuronic acid (GlcA)/methylglucuronic acid (MeGlcA) residues, and it lacks the unique reducing end tetrasaccharide sequence found in dicot xylan. The mechanism of how grass xylan backbone elongation is initiated and how its distinctive substitution pattern is determined remains elusive. Here, we performed biochemical characterization of rice xylan biosynthetic enzymes, including xylan synthases, glucuronyltransferases and methyltransferases. Activity assays of rice xylan synthases demonstrated that they required short xylooligomers as acceptors for their activities. While rice xylan glucuronyltransferases effectively glucuronidated unsubstituted xylohexaose acceptors, they transferred little GlcA residues onto (Araf)-substituted xylohexaoses and rice xylan 3-O-arabinosyltransferase could not arabinosylate GlcA-substituted xylohexaoses, indicating that their intrinsic biochemical properties may contribute to the distinctive substitution patterns of rice xylan. In addition, we found that rice xylan methyltransferase exhibited a low substrate binding affinity, which may explain the partial GlcA methylation in rice xylan. Furthermore, immunolocalization of xylan in xylem cells of both rice and Arabidopsis showed that it was deposited together with cellulose in secondary walls without forming xylan-rich nanodomains. Together, our findings provide new insights into the biochemical mechanisms underlying xylan backbone elongation and substitutions in grass species.