Enhanced diffusion is an emergent property of many experimental microswimmer systems that usually arises from a combination of ballistic motion with random reorientations. A subset of these systems, autophoretic droplet swimmers that move as a result of Marangoni stresses, have additionally been shown to respond to local, self-produced chemical gradients that can mediate self-avoidance or self-attraction. Via this mechanism, we present a mathematical model constructed to encode experimentally observed self-avoidant memory and numerically study the effect of this particular memory on the enhanced diffusion of such swimming droplets. To disentangle the enhanced diffusion due to the random reorientations from the enhanced diffusion due to the self-avoidant memory, we compare to the widely used active Brownian model. Paradoxically, we find that the enhanced diffusion is substantially suppressed by the self-avoidant memory relative to that predicted by only an equivalent reorientation persistence timescale in the active Brownian model. We attribute this to transient self-caging that we propose is novel for self-avoidant systems. Additionally, we further explore the model parameter space by computing emergent parameters that capture the velocity and reorientation persistence, thus finding a finite parameter domain in which enhanced diffusion is observable.