The thermal responsivity of monolayer epitaxial graphene grown on the Si-face surface of semi-insulating SiC substrate is investigated as a function of temperature below 300 K. The measurements showed that adsorption/desorption of atmospheric adsorbates can randomly modify the electrical characteristics of graphene which is indeed undesirable for consistent temperature sensing operations. Therefore, in order to avoid the interaction between graphene layer and adsorbates, the grown graphene layer is encapsulated with a thin SiO2 film deposited by Pulsed Electron Deposition technique. Temperature dependent resistance measurement of encapsulated graphene exhibited a clear thermistor type behavior with negative temperature coefficient resistance character. Both the sensitivity and transient thermal responsivity of the SiO2/graphene/SiC sample were found to be enhanced greatly especially for the temperatures lower than 225 K. The experimentally obtained results suggest that SiO2 encapsulated epitaxial graphene on SiC can be used readily as an energy efficient and stable temperature sensing element in cryogenic applications.