In the current era of digital communication, the security of data transmitted relies heavily on cryptographic algorithms. Transmission of digital images through an insecure channel is liable for various types of cryptanalytic attacks. Therefore, there is a strong need for efficient encryption algorithms to protect data from adversaries. In this paper, we have proposed a new technique of color image encryption and decryption using chaos theory and algebra. The algorithm is designed by using two stages of affine Hill cipher (AHC) over SLm(Fp) and Mm(Fp) domains, before and next to two-dimensional discrete chaotic Hénon map. The non-linear nature, high sensitiveness to its parameters, and initial conditions, the Hénon map employs complex permutations. The diffusion is achieved by the affine Hill cipher technique to dissipate the statistical structure. The proposed system is not only key-dependent but their arrangement is fundamental, therefore, the cryptanalyst cannot hack the system by knowing only the key parameters. Simulation results and performance analysis are presented to verify the feasibility and efficiency of the proposed technique. Further, the security analysis describes that the system is robust against diverse cryptanalytic attacks and the comparison reveals that the proposed cryptosystem outperforms some existing and traditional encryption techniques.