A robust video communication system based on layered coding and unequal error protection is developed in this work. We consider two video communication scenarios. First, for pre-compressed video bitstreams, a channel code rate allocation scheme is proposed to minimize the expected mean square error subject to a constraint on the overall bit budget. Second, for real-time video transmission, we jointly optimize the quantization parameters and the channel coding rates according to channel conditions. To this end, we develop a simple rate-distortion model for general video coders using DCT and motion compensation, so that the rate and the distortion can be estimated without an expensive encoding procedure. Simulation results show that the proposed algorithms provide acceptable image quality even in high bit error rate environments.