Considering the complementarity of Moderate Resolution Imaging Spectroradiometer (MODIS) and Multiangle Imaging Spectroradiometer (MISR) images on temporal and angular resolutions, we propose a fusion method to generate frequent time series MISR images. Thereby, the fusion results can give full play to their respective advantages in the inversion of surface or atmospheric parameters. Based on sparse representation, the proposed method includes two stages: the spectral dictionary-pair training stage and the MISR image prediction stage. In the training stage, we establish a corresponding relationship between the basic MODIS and MISR representation atoms in the spectral domain by learning a dictionary pair from the prior image pairs. In the prediction stage, the MISR images are predicted from the corresponding MODIS images via sparse coding. Experimental results on Baltimore–Washington, DC metropolitan area demonstrate the effectiveness of the proposed method with approximately 7% prediction errors.