SummaryIn this article, an advanced reversible stenographic method is proposed for image and video frames by adapting dual tree complex wavelet transform and chaotic whale optimization algorithm. Here, the cover image and secret message are taken as input. In this image, the edges are identified by Canny edge detection system and embedded by the process of dual‐tree complex wave transformation method (DT‐CWT‐CWOA‐SDH) and pixel edges are optimized using the chaos whale optimization algorithm (CWOA), it optimizes the edge pixel variations and maximizes the robustness of the image and visual quality of stego image. The objective of this process is to increase the security of secret data hiding the secret message on cover image and forming the stego image by the embedding procedure and to maximize the peak signal‐to‐noise ratio (PSNR) and reduce the execution time. The proposed method runs on the MATLAB work site and the performance of proposed method is evaluated using different existing methods. The simulation outcomes illustrate that the proposed DT‐CWT‐CWOA‐SDH shows higher PSNR of 32.01%, 24.79%; lower mean square error (MSE) of 21.13%, 45.185%, normalized correlation (NC) of 11.24%, 19.04% compared with existing method.