Diffusion and relaxation of defects in bulk systems is a complex process that can only be accessed directly through simulations. We characterize the mechanisms of low-temperature aging in self-implanted crystalline silicon, a model system used extensively to characterize both amorphization and return to equilibrium processes, over 11 orders of magnitudes in time, from 10 ps to 1 s, using a combination of molecular dynamics and kinetic activation-relaxation technique simulations. These simulations allow us to reassess the atomistic mechanisms responsible for structural relaxations and for the overall logarithmic relaxation, a process observed in a large number of disordered systems and observed here over the whole simulation range. This allows us to identify three microscopic regimes, annihilation, aggregation, and reconstruction, in the evolution of defects and to propose atomistic justification for an analytical model of logarithmic relaxation. Furthermore, we show that growing activation barriers and configurational space exploration are kinetically limiting the system to a logarithmic relaxation. Overall, our long-time simulations do not support the amorphous cluster model but point rather to a relaxation driven by elastic interactions between defect complexes of all sizes.