Mobile video streaming is fundamental to advanced applications in the fifth generation (5G) networks. Millimeter wave (mmWave) communication represents a leading 5G technology, which provides rich bandwidth and, therefore, great potentials for high-quality mobile video streaming. However, mobile video streaming in mmWave 5G networks faces fundamental challenges due to mmWave antenna directivity and high user mobility. As such, users typically have short connection durations and frequent handoffs, making video streaming suffer from long handoff delays and connection latency. In this paper, we tackle the issues by developing a caching-based mmWave framework, which precaches video contents at the base station for handoff users and thus significantly reduces the connection and retrieval delays. As a result, high-mobility users with frequent handoffs can enjoy continuous high-quality video streaming. Specifically, we model the proposed system as a cache management problem and attain optimal video streaming quality by using Markov decision process to dynamically allocate proper cache memory space of each base station to mobile users. A cell-by-cell decomposition method is proposed to solve the dynamic programming problem with significantly reduced computational complexity. Using extensive simulations, we demonstrate that the proposed solution can effectively maintain high-quality mobile video streaming for high-mobility 5G users moving among mmWave small cells with directional antenna.