AbstractA novel image encryption algorithm based on hyperchaos and DNA encoding is proposed in this paper. Unlike the traditional DNA‐based encoding method, a more fine‐grained encryption approach is adopted, which enhances the security of the system. To address the inherent binary arithmetic limitation of traditional DNA computing, the authors develop a novel symmetric and reversible DNA subtraction operation that fully utilizes the encryption advantage of DNA computing. To enhance security, the authors introduce a DNA crossover mechanism based on pixel, which can generate different ciphertexts even for the same pixel values. In the algorithm, the authors propose a diffusion strategy that combines left diffusion, right diffusion, and XOR parameters. This strategy not only ensures the avalanche effect, where even a single pixel change in the plaintext image will cause a drastic change in the ciphertext image, but also greatly enhances the security of the algorithm. Experimental results and performance analysis indicate that the authors’ proposed system has good encryption performance and the ability to resist noise attacks and clipping attacks.