It has been claimed that Dendrobium officinale polysaccharides (PSs) can degrade into oligosaccharide and then transform into short-chain fatty acids in the intestine after oral administration, and play an anti-colitis-associated cancer (CAC) effect by inhibiting intestinal inflammation. However, the material basis and core chemical structure underlying the anti-colon cancer properties of PSs have not yet been elucidated. In this study, PSs were degraded into enzymatic oligosaccharides (OSs) using β-mannanase. The results of in vivo experiments revealed that PSs and OSs administered by gastric lavage had similar antitumor effects in CAC mice. OS-1 (Oligosaccharide compounds 1) and OS-2 (Oligosaccharide compounds 2) were further purified and characterized from OSs, and it was found that OS-1, OS-2, OSs, and PSs had similar and consistent anti-inflammatory activities in vitro. Chemical structure comparison and evaluation revealed that the chemical structure of β-D-Manp-(1 → 4)-β-D-Glcp corresponding to OS-1 was the least common PS structure with anti-colitic activity. Therefore, our findings suggest that OSs are the material basis for PSs to exert anti-CAC activity and that the chemical structure of β-D-Manp-(1 → 4)-β-D-Glcp corresponding to OS-1 is the core chemical structure of PSs against CAC.