Xylan is tightly associated with cellulose and lignin in secondary plant cell walls, contributing to its rigidity and structural integrity in vascular plants. However, the molecular features and the nanoscale forces that control the interactions among cellulose microfibrils, hemicelluloses, and lignin are still not well understood. Here, we combine comprehensive mass spectrometric glycan sequencing and molecular dynamics simulations to elucidate the substitution pattern in softwood xylans and to investigate the effect of distinct intramolecular motifs on xylan conformation and on the interaction with cellulose surfaces in Norway spruce (Picea abies). We confirm the presence of motifs with evenly spaced glycosyl decorations on the xylan backbone, together with minor motifs with consecutive glucuronation. These domains are differently enriched in xylan fractions extracted by alkali and subcritical water, which indicates their preferential positioning in the secondary plant cell wall ultrastructure. The flexibility of the 3-fold screw conformation of xylan in solution is enhanced by the presence of arabinofuranosyl decorations. Additionally, molecular dynamic simulations suggest that the glycosyl substitutions in xylan are not only sterically tolerated by the cellulose surfaces but that they increase the affinity for cellulose and favor the stabilization of the 2-fold screw conformation. This effect is more significant for the hydrophobic surface compared with the hydrophilic ones, which demonstrates the importance of nonpolar driving forces on the structural integrity of secondary plant cell walls. These novel molecular insights contribute to an improved understanding of the supramolecular architecture of plant secondary cell walls and have fundamental implications for overcoming lignocellulose recalcitrance and for the design of advanced wood-based materials.