Electrocardiogram (ECG) signals are one of the fundamental health indicators used in the continuous monitoring and diagnosis of severe heart diseases. The acquisition of long-term ECG signals generates large data that negatively affect the proficiency of the transmission channel and storage devices. In this regard, this work presents a novel compression algorithm by combining discrete cosine transform (DCT) and optimized tunable-Q wavelet transform (TQWT) for compression of 2D ECG signals. The proposed algorithm is applied to columns and rows of 2D ECG array, which improves the compression by increasing sparsity in the transform domain. Thereafter, transform coefficients are approximated using an optimized midtread quantizer. TQWT and quantizer’s parameters both are optimized using a recently developed COOT bird optimization algorithm, which simulates the behavior of COOT birds. The obtained quantized coefficients are encoded by adaptive Huffman encoding. The investigational work is tested on MIT-BIH arrhythmia database. The optimization performance of COOT algorithm is also validated by comparing it with some existing algorithms. Similarly, compression performance is also compared with the state-of-art compression techniques in terms of compression ratio, signal-to-noise ratio, percent root-mean-square difference, and quality score. The obtained average values of these parameters are 30.7826, 27.7073 dB, 4.5162 %, and 8.1349, respectively. Attained results strengthen the applicability of proposed method in telemedicine or mobile health-based monitoring systems and memory devices.