As an important component of N-linked glycoproteins, the core pentasaccharide is highly crucial to the potential application prospect of glycoprotein. However, the gas phase conformation study is a challenging one due to the size and complexity of the molecule, together with the necessity to rely on quantum chemistry modeling for relevant energetics and structures. In this paper, the structures of the trisaccharides and core pentasaccharides in N-linked glycans in the gas phase were constructed by a three-step tree-based (TSTB) sampling. Since single point energies of all the conformers are calculated at the temperature of zero, it is necessary to evaluate the stability at a high temperature. We calculate the Gibbs free energies using the standard thermochemistry model (T = 298.15 K). For trimannose, the energetic ordering at 298.15 K can be strongly changed compared to 0 K. Moreover, two structures of trimannose with high energies at 0 K are considered to provide a much better match of IR vibration signatures with the low Gibbs free energies. On this basis, the core pentasaccharide was constructed in three ways. The building configurations of core pentasaccharide were optimized to obtain reasonable low-energy stable conformers. Fortunately, the lowest-energy structure of core pentasaccharide is eventually the minimum at 0 K and 298.15 K. Furthermore, spectrum analysis of core pentasaccharide was carried out. Although poorly resolved, its contour from the experiment was in qualitative correspondence with the computed IR spectrum associated with its minimum free energy structure. A large number of strongly and weakly hydrogen-bonded hydroxyl and acetylamino groups contribute to a highly congested set of overlapping bands. Compared with traditional conformation generators, the TSTB sampling is employed to efficiently and comprehensively obtain preferred conformers of larger saccharides with lower energy.