In visible light communication (VLC), using multiple colors is an efficient way to enhance data rate, leading to multi-color VLC (MC-VLC). However, the performance of MC-VLC is jeopardized by the spectral overlaps of different colors. Thin-film optical filters, as the key component of MC-VLC systems, are usually adopted to separate colors. The passband bandwidth (BW) and center wavelength (CWL) of optical filters are critical to mitigate the crosstalk among colors and, thus, must be carefully designed. Moreover, due to the intrinsic wavelength shift of the CWL with the varying of the angle of incidence, it is challenging to support mobility for MC-VLC. In this paper, we consider a joint design of multiple optical filters for MC-VLC by properly selecting the BW and CWL of each filter. We first investigate the optical filter design for a fixed receiver location. Then, to support mobility, we propose two robust optical filter designs, namely, statistically and worst case robust designs, which do not rely on the exact receiver location. Efficient methods are developed to solve the corresponding design problems and obtain the optimized optical filters. Compared with the existing optical filters, the proposed optical filters exhibits much better performance in various scenarios.