Effective Compression Algorithm Research Articles

A Comparison of Machine Learning-Based and Conventional Technologies for Video Compression

The growing demand for high-quality video transmission over bandwidth-constrained networks and the increasing availability of video content have led to the need for efficient storage and distribution of large video files. To improve the latter, this article offers a comparison of six video compression methods without loss of quality. Particularly, H.255, VP9, AV1, convolutional neural network (CNN), recurrent neural network (RNN), and deep autoencoder (DAE). The proposed decision is to use a dataset of high-quality videos to implement and compare the performance of classical compression algorithms and algorithms based on machine learning. Evaluations of the compression efficiency and the quality of the received images were made on the basis of two metrics: PSNR and SSIM. This comparison revealed the strengths and weaknesses of each approach and provided insights into how machine learning algorithms can be optimized in future research. In general, it contributed to the development of more efficient and effective video compression algorithms that can be useful for a wide range of applications.

Synthetic Aperture Radar Image Compression Based on Low-Frequency Rejection and Quality Map Guidance

Synthetic Aperture Radar (SAR) images are widely utilized in the field of remote sensing. However, there is a limited body of literature specifically addressing the compression of SAR learning images. To address the escalating volume of SAR image data for storage and transmission, which necessitates more effective compression algorithms, this paper proposes a novel framework for compressing SAR images. Experimental validation is performed using a representative low-resolution Sentinel-1 dataset and the high-resolution QiLu-1 dataset. Initially, we introduce a novel two-stage transformation-based approach aimed at suppressing the low-frequency components of the input data, thereby achieving a high information entropy and minimizing quantization losses. Subsequently, a quality map guidance image compression algorithm is introduced, involving the fusion of the input SAR images with a target-aware map. This fusion involves convolutional transformations to generate a compact latent representation, effectively exploring redundancies between focused and non-focused areas. To assess the algorithm’s performance, experiments are carried out on both the low-resolution Sentinel-1 dataset and the high-resolution QiLu-1 dataset. The results indicate that the low-frequency suppression algorithm significantly outperforms traditional processing algorithms by 3–8 dB when quantifying the input data, effectively preserving image features and improving image performance metrics. Furthermore, the quality map guidance image compression algorithm demonstrates a superior performance compared to the baseline model.

AFC: An adaptive lossless floating-point compression algorithm in time series database

The time series database is a specialized type of database specifically designed for storing and analyzing time series data. Compression of time series data is crucial for its performance. However, efficiently compressing time series data, particularly floating-point data, remains challenging. Existing compression algorithms are efficient for only a limited range of data patterns, indicating a lack of self-adaptation. In this paper, we propose an effective and Adaptive lossless Floating-point Compression algorithm AFC for time series databases. We devise four unique compression strategies, and based on the data patterns, AFC dynamically selects the appropriate strategy. These strategies handle data compression for diverse data patterns, enhancing the compression ratio and efficiency. The most suitable strategy is employed to achieve an optimal compression ratio. We compared our AFC algorithm with four state-of-the-art compression algorithms, namely Gorilla, FPC, TSXor, and Chimp, as well as various general-purpose compression algorithms such as LZ4 and Snappy. Experimental results demonstrate that our algorithm achieves an improvement of at least 20% in compression ratio and even up to 100% on certain datasets.

Hierarchical Residual Encoding for Multiresolution Time Series Compression

Data compression is a key technique for reducing the cost of data transfer from storage to compute nodes. Increasingly, modern data scales necessitate lossy compression techniques, where exactness is sacrificed for a smaller compressed representation. One challenge in lossy compression is that different applications may have different accuracy demands. Today's compression techniques struggle in this setting either forcing the user to compress at the strictest accuracy demand, or to re-encode the data at multiple resolutions. This paper proposes a simple, but effective multiresolution compression algorithm for time series data, where a single encoding can effectively be decompressed at multiple output resolutions. There are a number of benefits over current state-of-the-art techniques for time series compression. (1) The storage footprint of this encoding is smaller than re-encoding the data at multiple resolutions. (2) Similarly, the compression latency is generally smaller than re-encoding at multiple resolutions. (3) Finally, the decompression latency of our encoding is significantly faster than single encodings at the strictest accuracy demand.

Deep Learning Architecture Improvement Based on Dynamic Pruning and Layer Fusion

The heavy workload of current deep learning architectures significantly impedes the application of deep learning, especially on resource-constrained devices. Pruning has provided a promising solution to compressing the bloated deep learning models by removing the redundancies of the networks. However, existing pruning methods mainly focus on compressing the superfluous channels without considering layer-level redundancies, which results in the channel-pruned models still suffering from serious redundancies. To mitigate this problem, we propose an effective compression algorithm for deep learning models that uses both the channel-level and layer-level compression techniques to optimize the enormous deep learning models. In detail, the channels are dynamically pruned first, and then the model is further optimized by fusing the redundant layers. Only a minor performance loss results. The experimental results show that the computations of ResNet-110 are reduced by 80.05%, yet the accuracy is only decreased by 0.72%. Forty-eight convolutional layers could be discarded from ResNet-110 with no loss of performance, which fully demonstrates the efficiency of the proposal.

ANALYSIS OF VIDEO DATA COMPRESSION ALGORITHMS AND OPTIMIZED FOR USE IN REMOTELY CONTROLLED DRONES

Due to the widespread use of unmanned aerial vehicles (UAVs) in the civil sphere, there is a need to improve the video image generation quality and transmission technologies, changing the video streams coding to reduce their size and improve quality. Video streaming technologies are moving further towards complicating the video transmission systems used, which helps improve the received video data quality. Higher of transmitted video signal quality – more requirements for frequency band increase used. This fact leads us to use effective video compression algorithms that allow us to combine high image quality with a narrow bandwidth of frequencies used. In this paper, currently existing coding algorithms, their efficiency and computational complexity are studied. Among the algorithms under consideration there will only be those whose effectiveness has been proven by finding their application in modern realities. Based on the research, conclusions will be drawn regarding the feasibility of using each encoding algorithm for transmitting video data in real time.

Compressing AIS Trajectory Data Based on the Multi-Objective Peak Douglas–Peucker Algorithm

The automatic identification system (AIS) provides a massive database for ocean science. The original AIS data are redundant. Direct use will cause a waste of data storage space and computation costs; hence, data compression must be performed. The Douglas–Peucker algorithm (DP) is an effective trajectory compression algorithm that can well preserve the spatial characteristics of a trajectory but has the following shortcomings: first, it has poor track recovery when compressing multi-turn routes; second, it does not consider the ship speed and heading; and third, it may have the wrong result of the compressed trajectory crossing the obstacle. To address these situations, this study proposes a multi-objective peak DP algorithm (MPDP) that adopts a peak sampling strategy, considers three optimization objectives (spatial characteristics, heading and speed) of trajectory and adds an obstacle detection mechanism to realize a compression algorithm more suitable for curved trajectories. The classical DP algorithm is compared with the MPDP algorithm by simulating trajectory and real trajectory experiments. The results show that the MPDP algorithm optimizes the length loss rate, simultaneous Euclidean distance, and average deviations of the speed and the heading while maintaining a high compression rate similar to that of the DP algorithm. Moreover, it can also successfully avoid obstacles. The optimization effect is most obvious for the multi-turn or hovering trajectory. The optimization rate of length loss, synchronous Euclidean distance, and average deviation of the heading can reach 40%.

An efficient compression technique for Foetal phonocardiogram signals in remote healthcare monitoring systems

Remote Healthcare Monitoring Systems (RHMs) that employ fetal phonocardiography (fPCG) signals are highly efficient technologies for monitoring continuous and long-term fetal heart rate. Wearable devices used in RHMs still face a challenge that decreases their efficacy in terms of energy consumption because these devices have limited storage and are powered by batteries. This paper proposes an effective fPCG compression algorithm to reduce RHM energy consumption. In the proposed algorithm, the Discrete Orthogonal Charlier Moment (DOCMs) is used to extract features of the signal. The householder orthonormalization method (HOM) is used with the Charlier Moment to overcome the propagation of numerical errors that occur when computing high-order Charlier polynomials. The proposed algorithm’s performance is evaluated in terms of CR, PRD, SNR, PSNR, and QS and provides the average values 18.33, 0.21, 48.85, 68.86, and 90.88, respectively. The results of the comparison demonstrate the proposed compression algorithm’s superiority over other algorithms. It also tested in terms of compression speed and computational efficiency. The results indicate that the proposed algorithm has a high Compression speed (218.672 bps) and high computational efficiency (21.33). Additionally, the results reveal that the proposed algorithm decreases the energy consumption of a wearable device due to the transmission time decreasing for data by 3.68 s.

Analysis of the Correlation Properties of the Wavelet Transform Coefficients of Typical Images

The increasing flow of photo and video information transmitted through the channels of infocommunication systems and complexes stimulates the search for effective compression algorithms that can significantly reduce the volume of transmitted traffic, while maintaining its quality. In the general case, the compression algorithms are based on the operations of converting the correlated brightness values of the pixels of the image matrix into their uncorrelated parameters, followed by encoding the obtained conversion coefficients. Since the main known decorrelating transformations are quasi-optimal, the task of finding transformations that take into account changes in the statistical characteristics of compressed video data is still relevant. These circumstances determined the direction of the study, related to the analysis of the decorrelating properties of the generated wavelet coefficients obtained as a result of multi-scale image transformation. The main result of the study was to establish the fact that the wavelet coefficients of the multi-scale transformation have the structure of nested matrices defined as submatrices. Therefore, it is advisable to carry out the correlation analysis of the wavelet transformation coefficients separately for the elements of each submatrix at each level of decomposition (decomposition). The main theoretical result is the proof that the core of each subsequent level of the multi-scale transformation is a matrix consisting of the wavelet coefficients of the previous level of decomposition. It is this fact that makes it possible to draw a conclusion about the dependence of the corresponding elements of neighboring levels. In addition, it has been found that there is a linear relationship between the wavelet coefficients within the local area of ​​the image with a size of 8×8 pixels. In this case, the maximum correlation of submatrix elements is directly determined by the form of their representation, and is observed between neighboring elements located, respectively, in a row, column or diagonally, which is confirmed by the nature of the scattering. The obtained results were confirmed by the analysis of samples from more than two hundred typical images. At the same time, it is substantiated that between the low-frequency wavelet coefficients of the multi-scale transformation of the upper level of the expansion, approximately the same dependences are preserved uniformly in all directions. The practical significance of the study is determined by the fact that all the results obtained in the course of its implementation confirm the presence of characteristic dependencies between the wavelet transform coefficients at different levels of image decomposition. This fact indicates the possibility of achieving higher compression ratios of video data in the course of their encoding. The authors associate further research with the development of a mathematical model for adaptive arithmetic coding of video data and images, which takes into account the correlation properties of wavelet coefficients of a multi-scale transformation.

Multiscale partition-based Kolmogorov-Sinai Entropy: a preliminary HRV study on Heart Failure vs. Atrial Fibrillation.

Several approaches for estimating complexity in physiological time series at various time scales have recently been developed, with a special focus on heart rate variability (HRV) series. While numerous multiscale complexity quantifiers have been investigated, a multiscale Kolmogorov-Sinai (K-S) entropy for the characterization of cardiovascular dynamics still has to be properly assessed. In this pilot study, we investigate the Algorithmic Information Content, which is calculated using an effective compression algorithm, to quantify multiscale partition- based K-S entropy on experimental HRV series. Data were gathered from publicly available datasets comprising long-term, unstructured recordings from 10 healthy subjects, as well as 10 patients with congestive heart failure (CHF) and 10 patients with atrial fibrillation. Results show that multiple time scales and domain partitions statistically discern healthy vs. pathological cardiovascular dynamics. We conclude that the proposed multiscale partition-based K-S entropy may constitute a viable tool for the complexity assessment of cardiovascular variability series.

A Multiscale Partition-Based Kolmogorov-Sinai Entropy for the Complexity Assessment of Heartbeat Dynamics.

Background: Several methods have been proposed to estimate complexity in physiological time series observed at different time scales, with a particular focus on heart rate variability (HRV) series. In this frame, while several complexity quantifiers defined in the multiscale domain have already been investigated, the effectiveness of a multiscale Kolmogorov–Sinai (K-S) entropy has not been evaluated yet for the characterization of heartbeat dynamics. Methods: The use of the algorithmic information content, which is estimated through an effective compression algorithm, is investigated to quantify multiscale partition-based K-S entropy on publicly available experimental HRV series gathered from young and elderly subjects undergoing a visual elicitation task (Fantasia). Moreover, publicly available HRV series gathered from healthy subjects, as well as patients with atrial fibrillation and congestive heart failure in unstructured conditions have been analyzed as well. Results: Elderly people are associated with a lower HRV complexity and a more predictable cardiovascular dynamics, with significantly lower partition-based K-S entropy than the young adults. Major differences between these groups occur at partitions greater than six. In case of partition cardinality greater than 5, patients with congestive heart failure show a minimal predictability, while atrial fibrillation shows a higher variability, and hence complexity, which is actually reduced by the time coarse-graining procedure. Conclusions: The proposed multiscale partition-based K-S entropy is a viable tool to investigate complex cardiovascular dynamics in different physiopathological states.

Quantifying partition-based Kolmogorov-Sinai Entropy on Heart Rate Variability: a young vs. elderly study.

In the last decades, a considerable effort has been devoted to quantify complexity in physiological time series, with a particular focus on heart rate variability (HRV). To this end, exemplary quantifiers including Approximate Entropy and Sample Entropy have successfully been applied by leveraging on statistical approximation and further parametrization through the definition of tolerance and embedding dimension, among others. In this study, we investigate the use of the Algorithmic Information Content, which is estimated through an effective compression algorithm, to quantify partition-based Kolmogorov-Sinai (K-S) entropy on HRV series. We test such a K-S estimate on real data gathered from the Fantasia database, aiming to discern young vs. elderly complex dynamics. Experimental results show that elderly people are associated with a lower HRV complexity and a more predictable behavior, with significantly lower partition-based K-S entropy than the young adults. We conclude that partition-based K-S entropy may effectively be used to investigate pathological conditions in the cardiovascular system, complementing state-of-the-art methods for complexity assessment.

Byte-Select Compression

Cache-block compression is a highly effective technique for both reducing accesses to lower levels in the memory hierarchy (cache compression) and minimizing data transfers (link compression). While many effective cache-block compression algorithms have been proposed, the design of these algorithms is largely ad hoc and manual and relies on human recognition of patterns. In this article, we take an entirely different approach. We introduce a class of “byte-select” compression algorithms, as well as an automated methodology for generating compression algorithms in this class. We argue that, based on upper bounds within the class, the study of this class of byte-select algorithms has potential to yield algorithms with better performance than existing cache-block compression algorithms. The upper bound we establish on the compression ratio is 2X that of any existing algorithm. We then offer a generalized representation of a subset of byte-select compression algorithms and search through the resulting space guided by a set of training data traces. Using this automated process, we find efficient and effective algorithms for various hardware applications. We find that the resulting algorithms exploit novel patterns that can inform future algorithm designs. The generated byte-select algorithms are evaluated against a separate set of traces and evaluations show that Byte-Select has a 23% higher compression ratio on average. While no previous algorithm performs best for all our data sets which include CPU and GPU applications, our generated algorithms do. Using an automated hardware generator for these algorithms, we show that their decompression and compression latency is one and two cycles respectively, much lower than any existing algorithm with a competitive compression ratio.

An effective compression algorithm for real-time transmission data using predictive coding with mixed models of LSTM and XGBoost

In this paper, a compression and decompression algorithm is proposed for real-time transmission data by combining discrete wavelet decomposition and reconstruction, hybrid model prediction of long short-term memory (LSTM) and extreme gradient boosting (XGBoost), and the development of quantization sequences. The forward compression and reverse decompression are achieved by continuously acquiring the prediction error and data series with quantization error through the iterative shifting approach. To accurately forecast data series with quantization errors, three classes of data with specific distribution characteristics collected from the measurement-while-drilling (MWD) operation field are decomposed into approximate data and detailed data, which are then predicted by autoregressive integrated moving average (ARIMA), support vector regression (SVR), XGBoost, backpropagation neural network (BPNN), radial basis function neural network (RBFNN), and LSTM models. Particularly, the LSTM model and the XGBoost model are proven to be suitable strategies for working with approximate data and detailed data separately by comparing with conventional approaches in terms of the mean absolute error (MAE), root mean square error (RMSE), and coefficient of determination (R2) predictive indexes. Besides, experiment studies show that almost 50% compression performance can be reached under one in ten thousand distortions. It is verified that the proposed algorithm is appropriate and efficient for real-time data transmission applications with high performance.

An Evolving TinyML Compression Algorithm for IoT Environments Based on Data Eccentricity.

Currently, the applications of the Internet of Things (IoT) generate a large amount of sensor data at a very high pace, making it a challenge to collect and store the data. This scenario brings about the need for effective data compression algorithms to make the data manageable among tiny and battery-powered devices and, more importantly, shareable across the network. Additionally, considering that, very often, wireless communications (e.g., low-power wide-area networks) are adopted to connect field devices, user payload compression can also provide benefits derived from better spectrum usage, which in turn can result in advantages for high-density application scenarios. As a result of this increase in the number of connected devices, a new concept has emerged, called TinyML. It enables the use of machine learning on tiny, computationally restrained devices. This allows intelligent devices to analyze and interpret data locally and in real time. Therefore, this work presents a new data compression solution (algorithm) for the IoT that leverages the TinyML perspective. The new approach is called the Tiny Anomaly Compressor (TAC) and is based on data eccentricity. TAC does not require previously established mathematical models or any assumptions about the underlying data distribution. In order to test the effectiveness of the proposed solution and validate it, a comparative analysis was performed on two real-world datasets with two other algorithms from the literature (namely Swing Door Trending (SDT) and the Discrete Cosine Transform (DCT)). It was found that the TAC algorithm showed promising results, achieving a maximum compression rate of 98.33%. Additionally, it also surpassed the two other models regarding the compression error and peak signal-to-noise ratio in all cases.

An advanced image compression technique by using a coupled compression algorithms depend on different wavelet methods

Digital images need a large storage capacity, and as a result they need a large bandwidth for data transmission to deliver to the desired destination over the network. Image compression technologies not only reduce the size of stored data, but also maintain as much as possible the output image quality. In the proposed research we review a technique for image compression that uses a distinct two-stage image encoding method using different compression algorithms and wavelet transform methods, which combines two types of effective compression algorithms that give more ability to compress image data. The proposed compression technique which coupled two image compression algorithms that put to use premium characteristics from each algorithm. The wavelet transform methods contribute effectively to finding suitable solutions to supply better compression ratios for images with high resolution. The complete series of compression includes repeated stages of encoding and decoding, in addition to the wavelet processing itself. This study will have carried out an advanced compression technique that contain a coupled compression algorithms relying on the preferred wavelets to this work from practical experiments they are, biorthogonal and Haar wavelet transform, the performance metrics for tested true HD color image will be studied. The challenge for image compression algorithms is to detect a best solution between a low compression ratio and good visual perception results. An essential measure of achieved image compression process is taken by compression ratio CR and the ratio of bit-per-pixel BPP. The CR and BPP metrics are important components in image compression techniques. Through the results of the image compression metrics in two stages, the best practical results were obtained when the compression ratio metric CR was equal to 2.3%, and this metric indicates that the compressed image can be stored using 2.3% of the original image data size. While the BPP which represent the bit number that used to store one pixel of true color image is equal to 0.575.

An advanced image compression technique by using a coupled compression algorithms depend on different wavelet methods

Digital images need a large storage capacity, and as a result they need a large bandwidth for data transmission to deliver to the desired destination over the network. Image compression technologies not only reduce the size of stored data, but also maintain as much as possible the output image quality. In the proposed research we review a technique for image compression that uses a distinct two-stage image encoding method using different compression algorithms and wavelet transform methods, which combines two types of effective compression algorithms that give more ability to compress image data. The proposed compression technique which coupled two image compression algorithms that put to use premium characteristics from each algorithm. The wavelet transform methods contribute effectively to finding suitable solutions to supply better compression ratios for images with high resolution. The complete series of compression includes repeated stages of encoding and decoding, in addition to the wavelet processing itself. This study will have carried out an advanced compression technique that contain a coupled compression algorithms relying on the preferred wavelets to this work from practical experiments they are, biorthogonal and Haar wavelet transform, the performance metrics for tested true HD color image will be studied. The challenge for image compression algorithms is to detect a best solution between a low compression ratio and good visual perception results. An essential measure of achieved image compression process is taken by compression ratio CR and the ratio of bit-per-pixel BPP. The CR and BPP metrics are important components in image compression techniques. Through the results of the image compression metrics in two stages, the best practical results were obtained when the compression ratio metric CR was equal to 2.3%, and this metric indicates that the compressed image can be stored using 2.3% of the original image data size. While the BPP which represent the bit number that used to store one pixel of true color image is equal to 0.575.

Iterative Enhanced Multivariance Products Representation for Effective Compression of Hyperspectral Images

Effective compression of hyperspectral (HS) images is essential due to their large data volume. Since these images are high dimensional, processing them is also another challenging issue. In this work, an efficient lossy HS image compression method based on enhanced multivariance products representation (EMPR) is proposed. As an efficient data decomposition method, EMPR enables us to represent the given multidimensional data with lower-dimensional entities. EMPR, as a finite expansion with relevant approximations, can be acquired by truncating this expansion at certain levels. Thus, EMPR can be utilized as a highly effective lossy compression algorithm for hyper spectral images. In addition to these, an efficient variety of EMPR is also introduced in this article, in order to increase the compression efficiency. The results are benchmarked with several state-of-the-art lossy compression methods. It is observed that both higher peak signal-to-noise ratio values and improved classification accuracy are achieved from EMPR-based methods.

A hybrid adaptive spatiotemporal compression scheme in cluster‐based wireless sensor networks

SummaryEnergy efficiency and data reliability are the crucial issues to be explored in wireless sensor networks (WSNs). A large amount of power consumption is depleted by each sensor node to transmit its sensed data which typically induces high temporal and spatial dependencies. Therefore, efficient compression approaches are designed to exploit the data correlation in both space and time so as to discard information redundancies with accurate recovery. In this paper, we propose a hybrid and adaptive spatiotemporal compression approach named Distributed Temporal Source Coding, designed to extend the lifespan of cluster based WSNs. The proposed scheme takes advantage of the performed Distributed Source Coding as an effective spatial compression algorithm and adaptively exploits the Lightweight Temporal Compression as a promising solution to reduce the temporally correlated information. The obtained results have revealed that the proposed algorithm outperforms other compression methods, promotes the network battery life, and ensures compelling data accuracy. The proposed scheme has succeeded to reduce the gap between the energy efficiency and data reliability in WSNs.

Comparison of Image Compression Methods for Image Transmission Over Wireless Sensor Network

Unlike conventional networks, in Wireless Sensor Network the nodes have constrained energy, memory and processing capabilities. These nodes deployed in a constrained environment monitor any changes in surrounding environment and transfer the changes to the cluster heads. Each node has its own memory, battery, and transceivers. Efficient utilization of these resources can result in the enhancement of network lifetime. In order to securely transfer the data in the form of images, an efficient and cost effective image compression algorithm is required. Hence, in this paper, a detailed review of image compression algorithms has been carried out. The selected algorithms are compared in terms of various performance metrics such as compression ratio, compression time, speed, type of data, etc. The results showed that algorithm proposed by Borici and Arber is the best in case of compression ratio, as it provides better compression ratio in comparison to other algorithms.