Terms Of Compression Efficiency Research Articles

Paired decision trees for fast intra decision in H.266/VVC

The Versatile Video Coding (H.266/VVC) has demonstrated significant improvements to its predecessor High-Efficiency Video Coding (H.265/HEVC) in terms of compression efficiency. One of its advancements is attributed to the introduction of the QuadTree with nested Multi-type Tree (QTMT). However, the accompanying computing complexity challenges practical applications. To this end, this paper proposes a fast encoding algorithm to accelerate the encoding process of H.266/VVC while maintaining its compression efficiency as much as possible. Essentially, the Coding Unit (CU) partition can be modeled as a classification problem, and thus conventional machine learning methods such as the decision tree can be used to solve it. Previous decision tree-based works generally use one decision tree to dyadically select one partition from a set of candidate partition modes, easily leading to a local optimum. In contrast, this work models the classification from a new viewpoint and devises Paired Decision Trees (PDT) to tackle it. Specifically, the first decision tree determines whether to divide the current CU or not, and the second decision tree further determines whether to divide the CU horizontally or vertically. However, inaccurate results in the first decision tree can be propagated to the second one, resulting in error accumulation. To address this issue, we introduce the Indeterminate Space to both decision trees and perform Rate-Distortion Optimization (RDO) on samples falling within this space. Experimental results show that compared with H.266/VVC reference software VTM-15.0, the proposed PDT method saves 52.86% encoding time on average while maintaining a small BDBR increase of only 1.36%, which significantly outperforms previous works.

Fast and Efficient Entropy Coding Architectures for Massive Data Compression

The compression of data is fundamental to alleviating the costs of transmitting and storing massive datasets employed in myriad fields of our society. Most compression systems employ an entropy coder in their coding pipeline to remove the redundancy of coded symbols. The entropy-coding stage needs to be efficient, to yield high compression ratios, and fast, to process large amounts of data rapidly. Despite their widespread use, entropy coders are commonly assessed for some particular scenario or coding system. This work provides a general framework to assess and optimize different entropy coders. First, the paper describes three main families of entropy coders, namely those based on variable-to-variable length codes (V2VLC), arithmetic coding (AC), and tabled asymmetric numeral systems (tANS). Then, a low-complexity architecture for the most representative coder(s) of each family is presented—more precisely, a general version of V2VLC, the MQ, M, and a fixed-length version of AC and two different implementations of tANS. These coders are evaluated under different coding conditions in terms of compression efficiency and computational throughput. The results obtained suggest that V2VLC and tANS achieve the highest compression ratios for most coding rates and that the AC coder that uses fixed-length codewords attains the highest throughput. The experimental evaluation discloses the advantages and shortcomings of each entropy-coding scheme, providing insights that may help to select this stage in forthcoming compression systems.

TERSE/PROLIX (TRPX) - a new algorithm for fast and lossless compression and decompression of diffraction and cryo-EM data.

High-throughput data collection in crystallography poses significant challenges in handling massive amounts of data. Here, TERSE/PROLIX (or TRPX for short) is presented, a novel lossless compression algorithm specifically designed for diffraction data. The algorithm is compared with established lossless compression algorithms implemented in gzip, bzip2, CBF (crystallographic binary file), Zstandard(zstd), LZ4 and HDF5 with gzip, LZF and bitshuffle+LZ4 filters, in terms of compression efficiency and speed, using continuous-rotation electron diffraction data of an inorganic compound and raw cryo-EM data. The results show that TRPX significantly outperforms all these algorithms in terms of speed and compression rate. It was 60 times faster than bzip2 (which achieved a similar compression rate), and more than 3 times faster than LZ4, which was the runner-up in terms of speed, but had a much worse compression rate. TRPX files are byte-order independent and upon compilation the algorithm occupies very little memory. It can therefore be readily implemented in hardware. By providing a tailored solution for diffraction and raw cryo-EM data, TRPX facilitates more efficient data analysis and interpretation while mitigating storage and transmission concerns. The C++20 compression/decompression code, custom TIFF library and an ImageJ/Fiji Java plugin for reading TRPX files are open-sourced on GitHub under the permissive MIT license.

CACTUS: Content-Aware Compression and Transmission Using Semantics for Automotive LiDAR Data

Many recent cloud or edge computing strategies for automotive applications require transmitting huge amounts of Light Detection and Ranging (LiDAR) data from terminals to centralized processing units. As a matter of fact, the development of effective Point Cloud (PC) compression strategies that preserve semantic information, which is critical for scene understanding, proves to be crucial. Segmentation and compression have always been treated as two independent tasks; however, since not all the semantic classes are equally important for the end task, this information can be used to guide data transmission. In this paper, we propose Content-Aware Compression and Transmission Using Semantics (CACTUS), which is a coding framework that exploits semantic information to optimize the data transmission, partitioning the original point set into separate data streams. Experimental results show that differently from traditional strategies, the independent coding of semantically consistent point sets preserves class information. Additionally, whenever semantic information needs to be transmitted to the receiver, using the CACTUS strategy leads to gains in terms of compression efficiency, and more in general, it improves the speed and flexibility of the baseline codec used to compress the data.

Compression Reconstruction and Fault Diagnosis of Diesel Engine Vibration Signal Based on Optimizing Block Sparse Bayesian Learning.

It is critical to deploy wireless data transmission technologies remotely, in real-time, to monitor the health state of diesel engines dynamically. The usual approach to data compression is to collect data first, then compress it; however, we cannot ensure the correctness and efficiency of the data. Based on sparse Bayesian optimization block learning, this research provides a method for compression reconstruction and fault diagnostics of diesel engine vibration data. This method’s essential contribution is combining compressive sensing technology with fault diagnosis. To achieve a better diagnosis effect, we can effectively improve the wireless transmission efficiency of the vibration signal. First, the dictionary is dynamically updated by learning the dictionary using singular value decomposition to produce the ideal sparse form. Second, a block sparse Bayesian learning boundary optimization approach is utilized to recover structured non-sparse signals rapidly. A detailed assessment index of the data compression effect is created. Finally, the experimental findings reveal that the approach provided in this study outperforms standard compression methods in terms of compression efficiency and accuracy and its ability to produce the desired fault diagnostic effect, proving the usefulness of the proposed method.

Immersive Video Coding: Should Geometry Information Be Transmitted as Depth Maps?

Immersive video often refers to multiple views with texture and scene geometry information, from which different viewports can be synthesized on the client side. To design efficient immersive video coding solutions, it is desirable to minimize bitrate, pixel rate and complexity. We investigate whether the classical approach of sending the geometry of a scene as depth maps is appropriate to serve this purpose. Previous work shows that bypassing depth transmission entirely and estimating depth at the client side improves the synthesis performance while saving bitrate and pixel rate. In order to understand if the encoder side depth maps contain information that is beneficial to be transmitted, we first explore a hybrid approach which enables partial depth map transmission using a block-based RD-based decision in the depth coding process. This approach reveals that partial depth map transmission may improve the rendering performance but does not present a good compromise in terms of compression efficiency. This led us to address the remaining drawbacks of decoder side depth estimation: complexity and depth map inaccuracy. We propose a novel system that takes advantage of high quality depth maps at the server side by encoding them into lightweight features that support the depth estimator at the client side. These features allow reducing the amount of data that has to be handled during decoder side depth estimation by 88%, which significantly speeds up the cost computation and the energy minimization of the depth estimator. Furthermore, −46.0% and −37.9% average synthesis BD-Rate gains are achieved compared to the classical approach with depth maps estimated at the encoder.

Efficient Light Field Image Compression with Enhanced Random Access

In light field image compression, facilitating random access to individual views plays a significant role in decoding views quickly, reducing memory footprint, and decreasing the bandwidth requirement for transmission. Highly efficient light field image compression methods mainly use inter view prediction. Therefore, they typically do not provide random access to individual views. On the other hand, methods that provide full random access usually reduce compression efficiency. To address this trade-off, a light field image encoding method that favors random access is proposed in this paper. Light field image views are grouped into independent (3× 3) views, which are called Macro View Images (MVIs) . To encode MVIs, the central view is used as a reference to compress its adjacent neighboring views using a hierarchical reference structure. To encode the central view of each MVI, the most central view along with the center of a maximum of three MVIs, are used as reference images for the disparity estimation. In addition, the proposed method allows the use of parallel processing to reduce the maximum encoding/decoding time-complexity in multi-core processors. Tile partitioning can also be used to randomly access different regions of the light field images. The simulation results show that the proposed method outperforms other state-of-the-art methods in terms of compression efficiency while providing random access to both views and regions of interest.

Performance comparison among popular implementations of H.264 encoders

Remote sensing videos captured by Unmanned Aerial Vehicle (UAV) air-born high-resolution cameras require an efficient compression scheme that preserves the details of the visual contents of the videos while reducing the total size of the data to be managed in real-time. This paper presents a detailed comparison between different open-source implementations for the H.264 video compression scheme. While the high-resolution videos allow analysts to extract more descriptive interpretations and draw more conclusive results, the increase in the consequent data size consumes more storage, resulting in more channel bandwidth, more power, and encounters an extra delay in transmission time. An efficient implementation of video compression can alleviate these large data size effects. In this paper, we analyze and compare the JM-encoder, the X264, the FFmpeg, and Cisco’s OpenH264 open-source implementations in terms of compression efficiency, video quality, and computational load. Moreover, we present the rate-distortion curves in terms of PSNR as a quality metric against the bit-rate for a combination of 20 videos with various resolutions and dynamic contents. Albeit H.64 is superseded by H.265, till now H.264 is used in more than 65% of video coding applications. For example, YouTube only allows H.264 for live streaming.

Shipboard Data Compression Method for Sustainable Real-Time Maritime Communication in Remote Voyage Monitoring of Autonomous Ships

Due to the ever-increasing amount of data collected and the requirements for the rapid and reliable exchange of information across many interconnected communication devices, land-based communications networks are experiencing continuous progress and improvement of existing infrastructures. However, maritime communications are still characterized by slow communication speeds and limited communication capacity, despite a similar trend of increasing demand for information exchange. These limitations are particularly evident in digital data exchange, which is still limited to relatively slow and expensive narrowband satellite transmission. Furthermore, with the increasing digitalization of ships and introducing the sustainable concept of autonomous ship operation, large amounts of collected data need to be transmitted in real-time to enable remote voyage monitoring and control, putting additional pressure on the already strained means of maritime communications. In this paper, an adaptive shipboard data compression method based on differential binary encoding is proposed for real-time maritime data transmission. The proposed approach is verified on the actual data collected on board a training ship equipped with the latest data acquisition system. The obtained results show that the proposed data encoding method efficiently reduces the transmitted data size to an average of 3.4% of the original shipboard data, thus significantly reducing the required data transmission rate. Moreover, the proposed method outperforms several other tested competing methods for shipboard data encoding by up to 69.6% in terms of compression efficiency. Therefore, this study suggests that the proposed data compression approach can be a viable and efficient solution for transmitting large amounts of digital shipboard data in sustainable maritime real-time communications.

A novel lossless compression framework for facial depth images in expression recognition

With the development of AR and VR, depth images are widely used for facial expression analysis and recognition. To reduce the storage size and save bandwidth, an efficient compression framework is desired. In this paper, we propose a novel lossless compression framework for facial depth images in expression recognition. In the proposed framework, two steps are designed to remove the redundancy in the facial depth images, which are data preparing and bitstream encoding operations. In the data preparing operation, the original image is represented by the same and different parts between the left and right sides. In the bitstream encoding operation, these parts are compressed to get the final bitstream. The proposed framework is implemented and examined on the BU-3DFE Database. Experimental result shows that the proposed technique outperforms existing lossless compression frameworks in terms of compression efficiency, and the average data size is reduced to 25.27% by the proposed framework.

New variable-length data compression scheme for solution representation of meta-heuristics

Solution representation is an important aspect of the development of a meta-heuristic. However, most meta-heuristics focus on the algorithm aspect rather than the solution representation. As the scale of the optimization problem under study rises, the solution representation becomes ever more crucial because the computational performance tends to deteriorate for three reasons: (1) increased landscape complexity, (2) exponential growth of search space, and (3) costly function evaluation. In this study, data compression of solution representation is explored to improve the performance of meta-heuristics in terms of compression efficiency and solution quality. A proposed new data compression scheme – variable-length solution representation compression for meta-heuristics – is developed and tested against benchmarked problem instances from the quadratic assignment problem library (QAPLIB) and additional large problem instances. The analyzed results indicate that the compressed solution representation scheme under a meta-heuristic framework performs well against the QAPLIB and large problem instances with less space complexity and computation time.

Reliability Analysis of the SHyLoC CCSDS123 IP Core for Lossless Hyperspectral Image Compression Using COTS FPGAs

Hyperspectral images can comprise hundreds of spectral bands, which means that they can represent a large volume of data difficult to manage with the available on-board resources. Lossless compression solutions are interesting for reducing the amount of information stored or transmitted while preserving it at the same time. The Hyperspectral Lossless Compressor for space applications (SHyLoC), which is part of the European Space Agency (ESA) IP core’s library, has been demonstrated to meet the requirements of space missions in terms of compression efficiency, low complexity and high throughput. Currently, there is a trend to use Commercial Off-The-Shelf (COTS) on-board electronic devices on small satellites. Moreover, commercial Field-Programmable Gate Arrays (FPGAs) have been used in a number of them. Hence, a reliability analysis is required to ensure the robustness of the applications to Single Event Upsets (SEUs) in the configuration memory. In this work, we present a reliability analysis of this hyperspectral image compression module as a first step towards the development of ad-hoc fault-tolerant protection techniques for the SHyLoC IP core. The reliability analysis is performed using a fault-injection-based experimental set-up in which a hardware implementation of the Consultative Committee for Space Data Systems (CCSDS) 123.0-B-1 lossless compression standard is tested against configuration memory errors in a Xilinx Zynq XC7Z020 System-on-Chip. The results obtained for unhardened and redundancy-based protected versions of the module are put into perspective in terms of area/power consumption and availability/protection coverage gained to provide insight into the development of more efficient knowledge-based protection schemes.

Compact structure for sparse undirected graphs based on a clique graph partition

Compressing real-world graphs has many benefits such as improving or enabling the visualization in small memory devices, graph query processing, community search, and mining algorithms. This work proposes a novel compact representation for real sparse and clustered undirected graphs. The approach lists all the maximal cliques by using a fast algorithm and defines a clique graph based on its maximal cliques. Further, the method defines a fast and effective heuristic for finding a clique graph partition that avoids the construction of the clique graph. Finally, this partition is used to define a compact representation of the input graph. The experimental evaluation shows that this approach is competitive with the state-of-the-art methods in terms of compression efficiency and access times for neighbor queries, and that it recovers all the maximal cliques faster than using the original graph. Moreover, the approach makes it possible to query maximal cliques, which is useful for community detection.

A novel lossless compression encoding framework for SAR remote sensing images

The resolution of SAR (synthetic-aperture radar) remote sensing images becomes higher to provide more details, but these images contain more data, which creates a limitation in terms of transport and storage. Most of existing image data compression frameworks are lossy or designed for spectral images. In this paper, we propose a novel lossless compression encoding framework for SAR remote sensing images. In the proposed framework, an outline of the image and the high-frequency components are separated and processed separately to increase the relativity of adjacent pixels. So the accuracy of prediction is improved, which makes the data compression more effective. The outline image is down-sampled to reduce data size, and the nonlocally centralized sparse representation-based super-resolution method is used to predict pixel values using the information in nonlocal similar regions. The proposed framework is evaluated with the ground range detected and PauliRGB images captured by SAR satellites. The results show that the proposed technique can get an efficient compression performance and it outperforms existing lossless compression frameworks in terms of compression efficiency.

Optimal deep learning based image compression technique for data transmission on industrial Internet of things applications

AbstractRecently, industrial Internet of things becomes more popular and it involves a group of intelligent devices linked to create systems which observe, gather, communicate, and investigate data. In this view, the demand for compression techniques in remote sensing images is increasing since low complexity technique is required in spacecraft. Deep learning, for instance, convolutional neural network (CNN) has gained more attention in the domain of computer vision, particularly for high‐level applications like detection along with interpretation. At the same time, it is difficult to resolve the low‐level applications like image compression and it is investigated in this article. This article presents an optimal compression technique using CNNs for remote sensing images. The proposed method uses CNN for learning the compact representation of the original image which held the structural data and was then coded by Lempel Ziv Markov chain algorithm. Next, the encoded image was reconstructed to retrieve the original image with high reconstructed image quality. The proposed optimal compression technique is compatible with the available image codec standards. Wide range of experiments was carried out and the results were compared with binary tree and optimized truncation, JPEG, and JPEG2000 in terms of compression efficiency, reconstructed image quality, and space saving (SS). The obtained results apparently proved the effectiveness of the presented method, which attains an average peak signal to noise ratio of 49.90 dB and SS of 89.38%.

Study on Capabilities of Different Segmentation Algorithms in Detecting and Reducing Brain Tumor Size in Magnetic Resonance Imaging for Effective Telemedicine Services

Over the past decade, different image segmentation algorithms have been developed and employed in segmenting or analyzing brain magnetic resonance imaging (MRI) scans in the clinical applications for the detection of brain tumor. However, accurate detection, compression and transmission of brain tumor data remain parts of the challenging tasks militating against brain tumor telemedicine services due to the complex nature of brain tumor MRI scans. In overcoming this challenge, five different brain tumor segmentation algorithms were developed for this study. The algorithms were developed using MATLAB scripts. The developed algorithms were evaluated using patients’ data retrieved from Mayfield website. The result of the comparative performance compression rate efficiency evaluation test carried out shows that the developed hybrid threshold-watershed segmentation algorithm outperforms others in terms of compression efficiency. The result implies that the usage of the developed hybrid threshold-watershed segmentation algorithm in transmitting brain tumor patients’ data transmission over wireless communication system will require limited bandwidth resource.

Study on Capabilities of Different Segmentation Algorithms in Detecting and Reducing Brain Tumor Size in Magnetic Resonance Imaging for Effective Telemedicine Services

Over the past decade, different image segmentation algorithms have been developed and employed in segmenting or analyzing brain magnetic resonance imaging (MRI) scans in the clinical applications for the detection of brain tumor. However, accurate detection, compression and transmission of brain tumor data remain parts of the challenging tasks militating against brain tumor telemedicine services due to the complex nature of brain tumor MRI scans. In overcoming this challenge, five different brain tumor segmentation algorithms were developed for this study. The algorithms were developed using MATLAB scripts. The developed algorithms were evaluated using patients’ data retrieved from Mayfield website. The result of the comparative performance compression rate efficiency evaluation test carried out shows that the developed hybrid threshold-watershed segmentation algorithm outperforms others in terms of compression efficiency. The result implies that the usage of the developed hybrid threshold-watershed segmentation algorithm in transmitting brain tumor patients’ data transmission over wireless communication system will require limited bandwidth resource.

An alphabet reduction algorithm for lossless compression of images with sparse histograms

In this paper, we propose a new adaptive arithmetic coding for lossless image compression applying an alphabet reduction algorithm. The algorithm is a reduction mechanism of the alphabet set within each block by assigning to each one an as small as possible symbol set including all the really present symbols called active symbols, instead of using the nominal alphabet set. The method can be considered as away to address the well-known zero-frequency problem which appears especially for images with sparse and locally sparse histograms. The analytical expression of the expected gain in terms of compression efficiency when using the block active symbol sets is derived. We show experimentally that the proposed method, in conjunction with adaptive arithmetic coding order-0 model applied for images with sparse and locally sparse histograms, provides promising compression ratios and outperforms several state-of-the-art lossless image compression standards such as JPEG2000, JPEG-LS and CALIC.

Multi-threaded compression of Earth observation time-series data

ABSTRACTEarth observation data are typically compressed using general-purpose single-threaded compression algorithms that operate at a fraction of the bandwidth of modern storage and processing systems. We present evidence that recently developed multi-threaded compression codecs offer substantial benefits over widely used single-threaded codecs in terms of compression efficiency when applied to a selection of moderate resolution imaging spectroradiometer (MODIS) datasets stored in the HDF5 format. Compression codecs from the LZ77 and Rice families are shown to vary in efficacy when applied to different MODIS data products, highlighting the need for compression strategies to be tailored to different classes of data. We also introduce LPC-Rice, a new multi-threaded codec, that performs particularly well when applied to time-series data.

Bit Rate Requirements for High Dynamic Range Video

High dynamic range (HDR) video can offer consumers a much improved viewing experience compared to current broadcast video. The dynamic range of current television images, referred to as standard dynamic range (SDR), is governed by cathode ray tube physics first documented about eighty years ago. The standards include the Electro-Optical Transfer Function (EOTF) and the Opto-Electrical Transfer Function (OETF), as defined in International Telecommunications Union Radiocommunication (ITU-R) Recommendations BT.1886 and ITU-R BT.709, respectively. Alternative transfer functions have been defined to support the transmission and rendering of HDR video signals. These transfer functions aim to provide perceptually uniform mapping of video signals to the higher luminance range of HDR displays while maintaining video signal bit depths used across current broadcast infrastructures. In consequence, these new transfer functions exhibit much higher non-linearity compared to the transfer functions used in today's SDR systems. This could lead to several implications, such as an increase in bit rates required to transmit HDR services; changes to the existing broadcast infrastructure, including graphics equipment and vision mixers; and the compatibility of HDR services with existing SDR displays. This paper studies the impact of such transfer functions on the efficiency of the video compression used for delivery to the final user. Results, in terms of compression efficiency and subjective picture quality, using single-layer High Efficiency Video Coding (HEVC, also known as H.265 and MPEG-H Part 2) video compression algorithms are presented. This seeks to answer the question of what bit rates will be required to provide HDR services using existing video compression technology.