The charge and lattice dynamics at the neutral-ionic phase transition in the organic mixed-stack compound TTF-CA were measured by time-resolved Fourier-transform infrared spectroscopy and modeled by Monte Carlo simulations on a one-dimensional chain. After photoexcitation in the ionic state, we observe transient changes in reflectivity, which follow a stretched-exponential behavior on a microsecond time scale. This implies that the dimerization between the TTF and CA molecules vanishes, creating metastable domains. The time, temperature and intensity dependences can be modeled by a diffusion-annihilation process of neutral-ionic domain walls. Their random walk explains the relaxation process and its characteristic decay with time.