Reflections at rough surfaces change the temporal structure of the reflected signal. This paper shows how to incorporate this temporal behavior in geometric room acoustics modeling. Specifically, a beam tracer is used for calculating the image sources and reflection paths. The roughness of the surfaces is taken into account in post-processing. A single reflection is assumed to distribute the energy according to an exponential function in time based on Biot's rough surface modeling theory. Multiple reflections are modeled with convolutions of exponential functions which are approximated as gamma functions.