In this work, we present a nanometer resolution structural characterization of epitaxial graphene (EG) layers grown on 4H-SiC (0001) 8° off-axis, by annealing in inert gas ambient (Ar) in a wide temperature range (Tgr from 1600 to 2000°C). For all the considered growth temperatures, few layers of graphene (FLG) conformally covering the 100 to 200-nm wide terraces of the SiC surface have been observed by high-resolution cross-sectional transmission electron microscopy (HR-XTEM). Tapping mode atomic force microscopy (t-AFM) showed the formation of wrinkles with approx. 1 to 2 nm height and 10 to 20 nm width in the FLG film, as a result of the release of the compressive strain, which builds up in FLG during the sample cooling due to the thermal expansion coefficients mismatch between graphene and SiC. While for EG grown on on-axis 4H-SiC an isotropic mesh-like network of wrinkles interconnected into nodes is commonly reported, in the present case of a vicinal SiC surface, wrinkles are preferentially oriented in the direction perpendicular to the step edges of the SiC terraces. For each Tgr, the number of graphene layers was determined on very small sample areas by HR-XTEM and, with high statistics and on several sample positions, by measuring the depth of selectively etched trenches in FLG by t-AFM. Both the density of wrinkles and the number of graphene layers are found to increase almost linearly as a function of the growth temperature in the considered temperature range.