Surface chemistry plays an essential role in gas-sensing applications, enabling significant improvements in the sensing performance. This study investigates the influence of the graphene oxide (GO) synthesis route on the sensitivity to NO2 gas at room temperature (25 °C) and 100 °C in a dry and humid atmosphere. GO powders were synthesized using both the classical Hummers' method (HGO), and an improved version of the Hummers' method (IGO) using a mixture of phosphoric and sulfuric acids. The subsequent reduction (resulting in rHGO and rIGO, respectively) aimed to enhance the electrical properties and procure nanomaterials sensitive to surface adsorbates. In contrast to the HGO, both IGO and rIGO samples exhibited a transient sensor response and an outstanding recovery performance, which was attributed to the existence of phosphate groups in the latter samples. Notably, the rIGO sample achieved a 23-fold increase in response to NO2 compared to rHGO. Additionally, the limit of detection (LOD) was calculated to be 0.98 ppb at 100 °C. Computational studies considering models of GO and of GO with phosphorus-containing species demonstrate that the presence of the latter, either at the surface or below the surface, leads to a four-fold increase in the NO2 adsorption energies, hence accounting for the significant enhancement in the sensing performance observed experimentally. This underscores the importance of tailoring the structure and chemical properties of GO/rGO materials for optimal performance in gas sensing applications.