This paper proposes an asymmetric scanning holography cryptosystem based on the Elgamal algorithm. The method encodes images with sine and cosine holograms. Subsequently, each hologram is divided into a signed bit matrix and an unsigned hologram matrix, both encrypted using the sender’s private key and the receiver’s public key. The resulting ciphertext matrices are then transmitted to the receiver. Upon receipt, the receiver decrypts the ciphertext matrices using their private key and the sender’s public key. We employ an asymmetric single-image encryption method for key management and dispatch for securing imaging and transmission. Furthermore, we conducted a sensitivity analysis of the encryption system. The image encryption metrics, including histograms of holograms, adjacent pixel correlation, image correlation, the peak signal-to-noise ratio, and the structural similarity index, were also examined. The results demonstrate the security and stability of the proposed method.