At temperatures below 1 K, the capacitance of a glass sample changes due to the application of a dc field in accordance with Burin's dipole gap theory [J. Low Temp. Phys. 100, 309 (1995)]]. However, we now report that below 20 mK, during the first sweep cycle of the dc electric field, the capacitance is smaller by about 10(-5) compared to any subsequent sweep. Despite this overall shift, the field dependence follows the dipole gap predictions. In a subsequent sweep to higher dc fields the dielectric constant drops by about 10(-5) as soon as the applied field is higher than any field previously applied. A picture involving the dynamics of resonant pairs provides a qualitative description of this behavior.