Efficient Compression Technique Research Articles

Efficient Video Compression Using Afterimage Representation.

Recent advancements in large-scale video data have highlighted the growing need for efficient data compression techniques to enhance video processing performance. In this paper, we propose an afterimage-based video compression method that significantly reduces video data volume while maintaining analytical performance. The proposed approach utilizes optical flow to adaptively select the number of keyframes based on scene complexity, optimizing compression efficiency. Additionally, object movement masks extracted from keyframes are accumulated over time using alpha blending to generate the final afterimage. Experiments on the UCF-Crime dataset demonstrated that the proposed method achieved a 95.97% compression ratio. In binary classification experiments on normal/abnormal behaviors, the compressed videos maintained performance comparable to the original videos, while in multi-class classification, they outperformed the originals. Notably, classification experiments focused exclusively on abnormal behaviors exhibited a significant 4.25% improvement in performance. Moreover, further experiments showed that large language models (LLMs) can interpret the temporal context of original videos from single afterimages. These findings confirm that the proposed afterimage-based compression technique effectively preserves spatiotemporal information while significantly reducing data size.

Comparative Analysis of Huffman Coding Implementations for Efficient Data Communication Using Greedy and Divide-and-Conquer Techniques

Efficient data compression techniques are required to minimize storage and processing overhead due to modern systems' growing amount of data. Huffman Coding is a lossless compression technique that maintains data integrity by assigning shorter bit codes to characters appearing frequently, reducing size. Our analysis focuses on two implementation methodologies: greedy technique and divide and conquer. To find efficient solutions, divide-and-conquer algorithms partition problems into smaller components. In contrast, greedy algorithms strive to attain the utmost attainable result at each level. Our extensive investigation centers on the timing and space intricacies of diverse methodologies, enabling a comparative analysis that underscores their respective merits and drawbacks.

Robust Mixed-Rate Region-of-Interest-Aware Video Compressive Sensing for Transmission Line Surveillance Video

Classic video compression methods usually suffer from long encode time and requires large memories, making it hard to deploy on edge devices; thus, video compressive sensing, which requires less resources during encoding, is receiving more attention. We propose a robust mixed-rate ROI-aware video compressive sensing algorithm for transmission line surveillance video compression. The proposed method compresses foreground targets and background frames separately and uses reversible neural network to reconstruct original frames. The result on transmission line surveillance video data shows that the proposed compressive sensing method can achieve 26.47, 34.71 PSNR and 0.6839, 0.9320 SSIM higher than existing methods on 1.5% and 15% measurement rates, and the proposed ROI extraction net can precisely retrieve regions under high noise levels. This research not only demonstrates the potential for a more efficient video compression technique in resource-constrained environments, but also lays a foundation for future advancements in video compressive sensing techniques and their applications in various real-time surveillance systems.

An efficient data compression and storage technique with key management authentication in cloud space

Cloud computing is one of the promising technologies that offers cost-effective choices for processing and storing the huge volumes of data. In today’s world, data is the most important asset that one can have but it needs to be handled and protected properly. Portability of data can be increased by reducing the size of the data to be stored because of the limited storage space. As a result, data compression has arisen significantly. Data compression is a useful technique for reducing data size and increasing the effectiveness of data transit and storage. Data compression reduces the size of a data file while using lossy or lossless compression. One of the newest techniques for data compression is data duplication, which can reduce the amount of data saved while removing unnecessary data and maintaining an exact copy of the data. This analysis presents an Efficient data compression and storage technique with key management authentication in cloud space. This approach uses Regressive probabilistic key encryption (RPKE) to encrypt the cloud data and Lempel-Ziv-77-Huffman coding (LZ77-HM) is used to compress the huge amounts of cloud data. The Performance of presented approach is evaluated in terms of compression ratio and compression rate.

Wavelet-based Filter Bank Approach for Image Compression and Enhancement in Alzheimer\'s Disease Diagnosis Using Medical Imaging

Abstract: This research presents a wavelet-based filter bank approach for image compression and enhancement in the context of Alzheimer's disease diagnosis using medical imaging. The proposed system leverages the PyWavelets and OpenCV libraries in Python to implement Discrete Wavelet Transform (DWT) and Inverse Discrete Wavelet Transform (IDWT) algorithms. The primary objective is to develop an efficient image compression technique while preserving visual quality and enhancing relevant features for improved diagnostic accuracy. The system utilizes filter banks to process wavelet coefficients, enabling selective enhancement of patterns or biomarkers associated with Alzheimer's disease. Results show that the decompressed images closely resemble the originals, with low Mean Squared Error (MSE) and high Peak Signal-to-Noise Ratio (PSNR) values. Visual inspection and quantitative metrics confirm the successful preservation of image quality and structural details. The wavelet coefficients visualization highlights the captured significant features crucial for reconstruction

Memory‐efficient compression of 𝒟ℋ2‐matrices for high‐frequency Helmholtz problems

AbstractDirectional interpolation is a fast and efficient compression technique for high‐frequency Helmholtz boundary integral equations, but requires a very large amount of storage in its original form. Algebraic recompression can significantly reduce the storage requirements and speed up the solution process accordingly. During the recompression process, weight matrices are required to correctly measure the influence of different basis vectors on the final result, and for highly accurate approximations, these weight matrices require more storage than the final compressed matrix. We present a compression method for the weight matrices and demonstrate that it introduces only a controllable error to the overall approximation. Numerical experiments show that the new method leads to a significant reduction in storage requirements.

Collaborative Knowledge Distillation via Multiknowledge Transfer.

Knowledge distillation (KD), as an efficient and effective model compression technique, has received considerable attention in deep learning. The key to its success is about transferring knowledge from a large teacher network to a small student network. However, most existing KD methods consider only one type of knowledge learned from either instance features or relations via a specific distillation strategy, failing to explore the idea of transferring different types of knowledge with different distillation strategies. Moreover, the widely used offline distillation also suffers from a limited learning capacity due to the fixed large-to-small teacher-student architecture. In this article, we devise a collaborative KD via multiknowledge transfer (CKD-MKT) that prompts both self-learning and collaborative learning in a unified framework. Specifically, CKD-MKT utilizes a multiple knowledge transfer framework that assembles self and online distillation strategies to effectively: 1) fuse different kinds of knowledge, which allows multiple students to learn knowledge from both individual instances and instance relations, and 2) guide each other by learning from themselves using collaborative and self-learning. Experiments and ablation studies on six image datasets demonstrate that the proposed CKD-MKT significantly outperforms recent state-of-the-art methods for KD.

EFFICIENT VIDEO COMPRESSION USING HEVC AND NON-LINEAR CONVOLUTIONAL MOBILEVNET BASED RATE-DISTORTION OPTIMIZATION

With the surge in demand for high-quality video content over various platforms, efficient video compression techniques have become indispensable. High-Efficiency Video Coding (HEVC) has been a cornerstone, yet further enhancements are essential for optimal compression. Despite HEVC’s advancements, achieving optimal compression while maintaining video quality remains challenging. Additionally, existing methods often overlook the computational complexity, hindering real-time applications. We propose a novel approach integrating HEVC with Non-Linear Convolutional MobileNet (NLCM) for enhanced compression efficiency. Our method employs a rate-distortion optimization framework, leveraging the capabilities of both HEVC and NLCM to achieve superior compression performance. NLCM provides adaptive filtering, enhancing spatial and temporal correlations, while HEVC ensures high compression efficiency. Through experimentation on standard video datasets, our method demonstrates significant improvements over existing techniques. Compared to HEVC alone, our approach achieves up to 30% reduction in bitrate at equivalent perceptual quality levels. Moreover, computational complexity is reduced by 15%, enabling real- time applications without compromising performance. The proposed method exhibits competitive results across various resolutions and frame rates, making it versatile for diverse video compression scenarios.

Arithmetic N-gram: an efficient data compression technique

Due to the increase in the growth of data in this era of the digital world and limited resources, there is a need for more efficient data compression techniques for storing and transmitting data. Data compression can significantly reduce the amount of storage space and transmission time to store and transmit given data. More specifically, text compression has got more attention for effectively managing and processing data due to the increased use of the internet, digital devices, data transfer, etc. Over the years, various algorithms have been used for text compression such as Huffman coding, Lempel-Ziv-Welch (LZW) coding, arithmetic coding, etc. However, these methods have a limited compression ratio specifically for data storage applications where a considerable amount of data must be compressed to use storage resources efficiently. They consider individual characters to compress data. It can be more advantageous to consider words or sequences of words rather than individual characters to get a better compression ratio. Compressing individual characters results in a sizeable compressed representation due to their less repetition and structure in the data. In this paper, we proposed the ArthNgram model, in which the N-gram language model coupled with arithmetic coding is used to compress data more efficiently for data storage applications. The performance of the proposed model is evaluated based on compression ratio and compression speed. Results show that the proposed model performs better than traditional techniques.

CPRNC: Channels pruning via reverse neuron crowding for model compression

Channel pruning is an efficient technique for model compression, removing redundant parts of a convolutional neural network with minor degradation in classification accuracy. Previous criteria of channel pruning ignore neurons’ intrinsic relationship and the high correlation with input samples. Inspired by the visual crowding phenomenon in neuroscience, this paper presents a novel channel pruning method via reverse neuron crowding, dubbed CPRNC, to address this issue. First, CPRNC involves a neuron crowding degree measure (NCDM) module, which builds the relationship model among all artificial neurons by observing their crowding behaviors. Subsequently, each channel’s importance is evaluated by the crowding degree of corresponding channels. Considering that the channel importance is affected by the characteristic of input samples, CPRNC designs a neuron crowding degree recalibrate (NCDR) module. NCDR emphasizes discriminative samples to recalibrate the channel priority list generated by NCDM, further enhancing the precision of the pruning criterion. Experimental results show that CPRNC achieves performance that competes with state-of-the-art pruning methods, including dynamic channel pruning and learning-based pruning. For example, we prune ResNet-50 with 56.7% FLOPs on the large-scale dataset ImageNet1K with only a 0.19% decrease in accuracy. At low pruning rates, CPRNC achieves lossless compression, e.g., the pruned ResNet-56 on CIFAR-10 increases accuracy by 0.13% over the baseline model at 56.3% FLOPs reduction.

Multi-codec rate adaptive point cloud streaming for holographic-type communication

Point cloud videos play a crucial role in immersive applications enabled by holographic-type communication, which has been identified as an important service for 6G and beyond wireless systems and the metaverse. The significant volume of point cloud video demands efficient compression and transmission techniques to support the Quality of Experience (QoE) requirements of real-time immersive applications. A few Point Cloud Compression (PCC) techniques, such as MPEG PCC and Draco, have emerged in recent years, and studies have shown that each technique has its strengths and weaknesses under different system settings. This paper proposes a multi-codec rate adaptive point cloud streaming method to satisfy the QoE requirements of interactive and live applications considering available system resources. The proposed method leverages three common PCC techniques: MPEG V-PCC, MPEG G-PCC, and Draco. The performance of each PCC technique is evaluated under various test conditions, and then estimation models are constructed to predict the bit rate, the decoding time, and the quality of the reconstructed point cloud. Based on the user's quality requirements and available computational and communication resources, the proposed streaming method selects a codec along with appropriate compression parameters that can provide the minimum latency for streaming. Evaluation results demonstrate that the proposed method can provide better QoE than benchmark methods under various bandwidth and computation scenarios.

A new efficient referential genome compression technique for FastQ files.

Hospitals and medical laboratories create a tremendous amount of genome sequence data every day for use in research, surgery, and illness diagnosis. To make storage comprehensible, compression is therefore essential for the storage, monitoring, and distribution of all these data. A novel data compression technique is required to reduce the time as well as the cost of storage, transmission, and data processing. General-purpose compression techniques do not perform so well for these data due to their special features: a large number of repeats (tandem and palindrome), small alphabets, and highly similar, and specific file formats. In this study, we provide a method for compressing FastQ files that uses a reference genome as a backup without sacrificing data quality. FastQ files are initially split into three streams (identifier, sequence, and quality score), each of which receives its own compression technique. A novel quick and lightweight mapping mechanism is also presented to effectively compress the sequence stream. As shown by experiments, the suggested methods, both the compression ratio and the compression/decompression duration of NGS data compressed using RBFQC, are superior to those achieved by other state-of-the-art genome compression methods. In comparison to GZIP, RBFQC may achieve a compression ratio of 80-140% for fixed-length datasets and 80-125% for variable-length datasets. Compared to domain-specific FastQ file referential genome compression techniques, RBFQC has a compression and decompression speed (total) improvement of 10-25%.

SbMBR Tree—A Spatiotemporal Data Indexing and Compression Algorithm for Data Analysis and Mining

In the field of data analysis and mining, adopting efficient data indexing and compression techniques to spatiotemporal data can significantly reduce computational and storage overhead for the abilities to control the volume of data and exploit the spatiotemporal characteristics. However, traditional lossy compression techniques are hardly suitable due to their inherently random nature. They often impose unpredictable damage to scientific data, which affects the results of data mining and analysis tasks that require certain precision. In this paper, we propose a similarity-based minimum bounding rectangle (SbMBR) tree, a tree-based indexing and compression method, to address the aforementioned problem. Our method can hierarchically select appropriate minimum bounding rectangles according to the given maximum acceptable errors and use the average value contained in each selected MBR to replace the original data to achieve data compression with multi-layer loss control. This paper also provides the corresponding tree construction algorithm and range query processing algorithm for the indexing structure mentioned above. To evaluate the data quality preservation in cross-domain data analysis and mining scenarios, we use mutual information as the estimation metric. Experimental results emphasize the superiority of our method over some of the typical indexing and compression algorithms.

Design approach of FPGA based efficient data compression technique (TS $${D}^{2}$$ NIS) in FC technology for data transmission

Design approach of FPGA based efficient data compression technique (TS $${D}^{2}$$ NIS) in FC technology for data transmission

From Pixels to Bits: A State of the Art of Deep Learning Approaches for Natural Image Compression

With the ever-increasing volume of digital imagery and the growing demand for efficient storage and transmission, image compression has become a crucial aspect of multimedia processing. In recent years, deep learning models have emerged as powerful tools for a wide range of computer vision tasks, including image compression. This research article presents a comprehensive state-of-the-art review of natural image compression techniques leveraging deep learning architectures.
 
 This research article presents a state-of-the-art review of natural image compression using deep learning models. With the exponential growth of digital imagery, efficient compression techniques are essential. Deep learning, particularly Convolutional Neural Networks (CNNs), Generative Adversarial Networks (GANs), and Autoencoders, has shown promise in this area. The paper explores various architectures, discussing their adaptation to image compression tasks. Both lossless and lossy compression approaches are surveyed, considering the trade-off between compression ratios and visual quality. The article identifies challenges, such as computational complexity, scalability, and real-world applications, while suggesting future directions.

Distributed ℋ 2 -Matrices for Boundary Element Methods

Standard discretization techniques for boundary integral equations, e.g., the Galerkin boundary element method, lead to large densely populated matrices that require fast and efficient compression techniques like the fast multipole method or hierarchical matrices. If the underlying mesh is very large, running the corresponding algorithms on a distributed computer is attractive, e.g., since distributed computers frequently are cost-effective and offer a high accumulated memory bandwidth. Compared to the closely related particle methods, for which distributed algorithms are well-established, the Galerkin discretization poses a challenge, since the supports of the basis functions influence the block structure of the matrix and therefore the flow of data in the corresponding algorithms. This article introduces distributed ℋ 2 -matrices, a class of hierarchical matrices that is closely related to fast multipole methods and particularly well-suited for distributed computing. While earlier efforts required the global tree structure of the ℋ 2 -matrix to be stored in every node of the distributed system, the new approach needs only local multilevel information that can be obtained via a simple distributed algorithm, allowing us to scale to significantly larger systems. Experiments show that this approach can handle very large meshes with more than 130 million triangles efficiently.

An efficient and secure compression technique for data protection using burrows-wheeler transform algorithm

Data stored on physical storage devices and transmitted over communication channels often have a lot of redundant information, which can be reduced through compression techniques to conserve space and reduce the time it takes to transmit the data. The need for adequate security measures, such as secret key control in specific techniques, raises concerns about data exposure to potential attacks. Encryption plays a vital role in safeguarding information and maintaining its confidentiality by utilizing a secret key to make the data unreadable and unalterable. The focus of this paper is to tackle the challenge of simultaneously compressing and encrypting data without affecting the efficacy of either process. The authors propose an efficient and secure compression method incorporating a secret key to accomplish this goal. Encoding input data involves scrambling it with a generated key and then transforming it through the Burrows-Wheeler Transform (BWT). Subsequently, the output from the BWT is compressed through both Move-To-Front Transform and Run-Length Encoding. This method blends the cryptographic principles of confusion and diffusion into the compression process, enhancing its performance. The proposed technique is geared towards providing robust encryption and sufficient compression. Experimentation results show that it outperforms other techniques in terms of compression ratio. A security analysis of the technique has determined that it is susceptible to the secret key and plaintext, as measured by the unicity distance. Additionally, the results of the proposed technique showed a significant improvement with a compression ratio close to 90% after passing all the test text files.

Error-Resilient Data Compression With Tunstall Codes

Data compression has been commonly employed to reduce the required memory size for emerging applications with large storage needs like Big Data and Machine Learning (ML). When considering the flexibility of decompression and its hardware implementation, variable-to-fixed length codes (e.g., Tunstall codes) are usually selected. However, memories are prone to suffer different types of errors, causing the stored data to be corrupted; if an error affects the compressed data, it can propagate and cause corruption in a sequence of bits of the decompressed data. Therefore, error resilience should be built-in as part of the memory design to provide reliable data, especially for safety-critical applications. However, Error Correction Codes (ECCs) that are widely used for memory protection, are not very efficient to protect compressed data, because ECCs further increase the memory size and the additional decoding process can impact the latency to decompress the stored data. In this paper, an efficient error-resilient data compression technique with Tunstall codes is proposed; it requires almost no memory overhead and can correct most errors during the decompression process by introducing a conversion table. An enhanced design is also presented to reduce the impact of errors when they cannot be corrected. The proposed scheme has been implemented and evaluated on three ML datasets; results show that it can deal with up to 99.98% errors with almost no memory overhead when Tunstall codes with smaller than 16-bit symbols are employed. The scheme has also been evaluated for two ML applications; results show that even though a small number of errors cannot be corrected in the proposed scheme, they have an extremely low impact on the classification results and the protection overhead is significantly lower than existing ECC techniques

Pricing High-Dimensional Bermudan Options with Hierarchical Tensor Formats

An efficient compression technique based on hierarchical tensors for popular option pricing methods is presented. It is shown that the “curse of dimensionality” can be alleviated for the computation of Bermudan option prices with the Monte Carlo least-squares approach as well as the dual martingale method, both using high-dimensional tensorized polynomial expansions. Complexity estimates are provided as well as a description of the optimization procedures in the tensor train format. Numerical experiments illustrate the favorable accuracy of the proposed methods. The dynamical programming method yields results comparable to recent neural network based methods.

BPG-Based Lossy Compression of Three-Channel Noisy Images with Prediction of Optimal Operation Existence and Its Parameters

Nowadays, there is a clear trend toward increasing the number of remote-sensing images acquired and their average size. This leads to the need to compress the images for storage, dissemination, and transfer over communication lines where lossy compression techniques are more popular. The images to be compressed or some of their components are often noisy. They must therefore be compressed taking into account the properties of the noise. Due to the noise filtering effect obtained during lossy compression of noisy images, an optimal operating point (OOP) may exist. The OOP is a parameter that controls the compression for which the quality of the compressed image is closer (closest) to the corresponding noise-free image than the quality of the noisy (original, uncompressed) image according to some quantitative criterion (metric). In practice, it is important to know whether the OOP exists for a given image, because if the OOP exists, it is appropriate to perform the compression in the OOP or at least in its neighborhood. Since the real image is absent in practice, it is impossible to determine a priori whether the OOP exists or not. Here, we focus on three-channel-remote-sensing images and show that it is possible to easily predict the existence of the OOP. Furthermore, it is possible to predict the metric values or their improvements with appropriate accuracy for practical use. The BPG (better portable graphics) encoder is considered a special case of an efficient compression technique. As an initial design step, the case of additive white Gaussian noise with equal variance in the three components is considered. While previous research was mainly focused on predicting the improvement (reduction) of the PSNR and PSNR-HVS-M metrics, here we focus on the modern visual quality metrics, namely PSNR-HA and MDSI. We also discuss what to do if, according to the prediction, an OOP is absent. Examples of lossy compression of noisy three-channel remote sensing images are given. It is also shown that the use of three-dimensional compression provides a compression ratio increase by several times compared with component-wise compression in the OOP.