We find strong path dependence in the evolution of the Plio-Pleistocene glaciations using CLIMBER-2 Earth System Model simulations from the mid-Pliocene to modern preindustrial (3 My-0 My BP) driven by a gradual decrease in volcanic carbon dioxide outgassing and regolith removal from basal ice interaction. Path dependence and hysteresis are investigated by alternatively driving the model forward and backward in time. Initiating the model with preindustrial conditions and driving the model backward using time-reversed forcings, the increase in volcanic outgassing back-in-time (BIT) does not generate the high CO2 levels and relatively ice-free conditions of the late Pliocene seen in forward-in-time (FIT) simulations of the same model. This behavior appears to originate from nonlinearities and initial state dependence in the carbon cycle. A transition from low-amplitude sinusoidal obliquity (~41 ky) and precession (~23 ky) driven glacial/interglacial cycles to high-amplitude ~100 ky likely eccentricity-related sawtooth cycles seen between -1.25 My and -0.75 My BP (the Mid-Pleistocene transition or "MPT") in FIT simulations disappears in BIT integrations depending on the details of how the regolith removal process is treated. A transition toward depleted regolith and lowered atmospheric CO2 levels are both required to reproduce the MPT.