A camouflaged cryptosystem based on optical diffractive imaging is proposed in this paper. During the encryption process, a camouflaged image is first encoded into an intensity image by optical diffractive imaging, which is stored and transmitted as a fake ciphertext. Then a secret image is encoded into two generated phase masks(GPMs) with the fake ciphertext as the amplitude constraint by cascading phase encoding. The ciphertext and random phase masks(RPMs) used in the optical diffractive imaging are transmitted to the receivers via a common channel, while GPMs are transmitted via a private channel. The common channel can be accessed by both authorized receivers and potential illegal intruders. Therefore, illegal intruders can just obtain the camouflaged image by the fake ciphertext and RPMs, while only authorized receivers with the fake ciphertext and GPMs can retrieve the secret image. The decoded camouflaged image would confuse the illegal users and provide additional security protection for the secret image. The feasibility and security of this cryptosystem would by verified by several simulation experiments.