Physical aging in colloidal dispersions manifests as a reduction in kinetic freedom of the colloids. In aqueous dispersions of charged clay colloids, the role of interparticle electrostatic interactions in determining the aging dynamics has been evaluated extensively. Despite water being the dispersion medium, the influence of water structure on the physicochemical properties of aging clay dispersions has, however, not been considered before. In this work, we use LAPONITE®, a model hectorite clay mineral that acquires surface charges when dispersed in water, to study the relative contributions of dispersion medium structure and interparticle electrostatic interactions on the physicochemical properties of aging hectorite clay dispersions. The structure of the dispersion medium is modified either by incorporating dissociating/non-dissociating kosmotropic (structure-inducing) or chaotropic (structure-disrupting) molecules or by changing dispersion temperature. Photon correlation spectroscopy, rheological measurements and particle-scale imaging are employed to evaluate the physicochemical properties of the dispersions. Our experiments involving incorporation of external additives demonstrate a strong influence of dispersion medium structure on the dispersion properties when the interparticle electrostatic interactions are weak. We introduce a new temperature dependent measurement protocol, wherein the temperature of the medium is fixed before adding the clay particles, to manipulate the hydrogen bonds in the aqueous medium in the absence of external additives. Accelerated aging, observed upon raising the temperature regardless of the experimental thermal histories, is attributed to increased interparticle electrostatic interactions as in the room temperature experiments with ionic additives. Our study identifies that in the presence of weak interparticle electrostatic interactions, changes in the physicochemical properties of charged clay dispersions can be driven by manipulating hydrogen bond populations in aqueous medium.