An image camouflage encryption scheme based on fiber specklegram and polarization multiplexing technology is proposed and demonstrated via simulation. The camouflaged original image is coupled into the multimode fiber (MMF) through different polarized light fields, and the specklegram caused by strong intermodal interference provides excellent chaos and randomness required for encryption. Since the optical fiber specklegram has a strong dependence on the configuration, transmission matrixes used as the key have high security and unpredictability. The results demonstrate the huge keyspace and excellent key sensitivity of the scheme. The ability to resist key disclosure and differential attack is analyzed. In addition, the scheme shows great security and reliability in several common noise attacks.