Understanding and determination of the ambipolar diffusivity of a semiconductor is fundamental for predicting device behavior and optimizing its performance. Ultrafast pump-probe measurements allow for the determination of energy carrier dynamics with sub-picosecond resolution. Due to the inherent diffusive nature of carriers, measurements with a high spatial resolution are needed, in addition to the traditional pump-probe system, to determine not only the carrier dynamics but also the spatial extent caused by carrier diffusion as well. In this work, a spatiotemporal measurement system with ultrafast temporal and nanometric spatial resolution, together with a comprehensive transport model, is used to determine the ambipolar diffusivity and carrier-phonon energy coupling time in both undoped and doped silicon. The results show that as the carrier density increases, the measured ambipolar diffusivity decreases with minimal variation in the carrier-phonon energy coupling time. In general, this work demonstrates an optical-based method for determining ambipolar diffusivity in a semiconductor material.