Most models of disparity selectivity consider only the spatial properties of binocular cells. However, the temporal response is an integral component of real neurons' activities, and time-varying stimuli are often used in the experiments of disparity tuning. To understand the temporal dimension of V1 disparity representation, we incorporate a specific temporal response function into the disparity energy model and demonstrate that the binocular interaction of complex cells is separable into a Gabor disparity function and a positive time function. We then investigate how the model simple and complex cells respond to widely used time-varying stimuli, including motion-in-depth patterns, drifting gratings, moving bars, moving random-dot stereograms, and dynamic random-dot stereograms. It is found that both model simple and complex cells show more reliable disparity tuning to time-varying stimuli than to static stimuli, but similarities in the disparity tuning between simple and complex cells depend on the stimulus. Specifically, the disparity tuning curves of the two cell types are similar to each other for either drifting sinusoidal gratings or moving bars. In contrast, when the stimuli are dynamic random-dot stereograms, the disparity tuning of simple cells is highly variable, whereas the tuning of complex cells remains reliable. Moreover, cells with similar motion preferences in the two eyes cannot be truly tuned to motion in depth regardless of the stimulus types. These simulation results are consistent with a large body of extant physiological data, and provide some specific, testable predictions.