This paper studies low-delay Wyner-Ziv coding, i.e., lossy source coding with side information at the decoder, with emphasis on the extreme of zero delay. To achieve zero delay, a scalar quantizer is followed by scalar coding of quantization indices. In the fixed-length coding scenario, under high-resolution assumptions and appropriately defined decodability constraints, the optimal quantization level density is conjectured to be periodic. This conjecture, which is provable when the correlation is high, allows for a precise analysis of the rate-distortion tradeoff. The performance of variable-length coding with periodic quantization is also characterized. The results are then incorporated in predictive Wyner-Ziv coding for Gaussian sources with memory, and optimal prediction filters are numerically designed so as to strike a balance between maximally exploiting both temporal and spatial correlation and limiting the propagation of distortion due to occasional decoding errors. Finally, the zero-delay schemes are also employed in transform coding with small block lengths, where the Gaussian source and side information are transformed separately with the premise that corresponding transform coefficient pairs exhibit good spatial correlation and minimal temporal correlation. For the specific source-side information pairs studied, it is shown that transform coding, even with a small block-length, outperforms predictive coding. Performances of both predictive and transform coding are also compared with the asymptotic rate-distortion bounds.