Image Sequence Compression Research Articles

Proposed Extensions to the Third Edition of JPEG XS (ISO/IEC 21122) Standard

At its 94th meeting (17–21 January 2022), International Organization for Standardization (ISO)/IEC SC29WG1 (also known as JPEG Committee) decided to create a third edition of the JPEG XS standard (ISO/IEC 21122) for lightweight, low-latency image coding. While existing coding tools in the standard already support the compression of natural content and color filter array (CFA) RAW Bayer pattern data well, this third edition will extend the previous editions with coding tools that improve its performance on the compression of screen content and image sequences with static background. The first experiments demonstrate that the proposed coding tools can improve the average peak signal-to-noise ratio (PSNR) performance of JPEG XS on such sequences of more than 10 dB, and approximately 3 dB on computer-generated content. In this work, the authors will present the proposed coding tools and will evaluate their performance objectively by PSNR.

A Novel Coding Architecture for Multi-Line LiDAR Point Clouds Based on Clustering and Convolutional LSTM Network

Light detection and ranging (LiDAR) plays an indispensable role in autonomous driving technologies, such as localization, map building, navigation and object avoidance. However, due to the vast amount of data, transmission and storage could become an important bottleneck. In this article, we propose a novel compression architecture for multi-line LiDAR point cloud sequences based on clustering and convolutional long short-term memory (LSTM) networks. LiDAR point clouds are structured, which provides an opportunity to convert the 3D data to 2D array, represented as range images. Thus, we cast the 3D point clouds compression as a range image sequence compression problem. Inspired by the high efficiency video coding (HEVC) algorithm, we design a novel compression framework for LiDAR data that includes two main techniques: intra-prediction and inter-prediction. For intra-frames, inspired by the depth modeling modes (DMM) adopted in 3D-HEVC, we develop a clustering-based intra-prediction technique, which can utilize the spatial structure characteristics of point clouds to remove the spatial redundancy. For inter-frames, we design a prediction network model using convolutional LSTM cells. The network model is capable of predicting future inter-frames using the encoded intra-frames. As a result, temporal redundancy can be removed. Experiments on the KITTI dataset demonstrate that the proposed method achieves an impressive compression ratio (CR), with 4.10% at millimeter precision, which means the point clouds can compress to nearly 1/25 of their original size. Additionally, compared with the well-known octree, Google Draco, and MPEG TMC13 methods, our algorithm yields better performance in compression ratio.

Lossless Compression of Medical Image Sequences Using a Resolution Independent Predictor and Block Adaptive Encoding

The proposed block-based lossless coding technique presented in this paper targets at compression of volumetric medical images of 8-bit and 16-bit depth. The novelty of the proposed technique lies in its ability of threshold selection for prediction and optimal block size for encoding. A resolution independent gradient edge detector is used along with the block adaptive arithmetic encoding algorithm with extensive experimental tests to find a universal threshold value and optimal block size independent of image resolution and modality. Performance of the proposed technique is demonstrated and compared with benchmark lossless compression algorithms. BPP values obtained from the proposed algorithm show that it is capable of effective reduction of inter-pixel and coding redundancy. In terms of coding efficiency, the proposed technique for volumetric medical images outperforms CALIC and JPEG-LS by 0.70 % and 4.62 %, respectively.

A novel resolution independent gradient edge predictor for lossless compression of medical image sequences

ABSTRACT Digital visualization of human body in terms of medical images with high resolution and bit depth generates tremendous amount of data. In the field of medical diagnosis, lossless compression technique is preferred that facilitates efficient archiving and transmission of medical images avoiding false diagnosis. Among various approaches to lossless compression of medical images, predictive coding techniques have high coding efficiency and low complexity. Gradient Edge Detector (GED) used in predictive coding is based on threshold value for prediction and choice of threshold is very important for efficient prediction. However, no specific method is adopted in the literature for threshold value selection. This paper presents an efficient prediction solution targeted at lossless compression of 8 bits and higher bit depth volumetric medical images up to 16 bits. Novelty of the proposed technique is developing Resolution Independent Gradient Edge Predictor (RIGED) algorithm to support 8- and 16-bit depth medical images. Percentage improvement of the proposed model is 30.39% over state-of-the-art Median Edge Detector (MED) and 0.92% over Gradient Adaptive Predictor (GAP) in terms of entropy for medical image dataset of different modalities having different resolutions and bit depths.

Meteosat image sequence coding in the Radon field

The high demand for Meteosat image sequences for various fields and the high rate of presentation of methods dealing with these latter is facing with the low rate of presentation of remote sensing image sequence compression methods generally for different imagers. To tackle this issue, a novel Meteosat image sequence coding method in the Radon field is presented in this paper. The idea behind this paper is to show the contribution of the coding in Radon field to an efficient lossless compression. The Radon field, represented here by a new fast Mojette transform (FMT), gives a homogeneous field through a Mojette matrix engendered by the intra-projections and inter-projections redundancy. As a consequence, the use of an efficient block-based motion estimation (ME) algorithm such as adaptive rood pattern search (ARPS) becomes possible in a such environment, which can help to avoid many problems encountered when using the same techniques for remote sensing image as a block effect by way of example. In order to achieve a high lossless compression, the Sorted Run Length Coding (SRLC) is applied to encode besides the motion vectors (MVs), the residual error represented by separate long sequences of zeros in addition to other repeated symbols. The test phase showed that our proposed scheme exceeds the performances of the JPEG-LS standard. The proposed scheme can also be used as pre-technique for methods using Meteosat images in general and their sequences in particular after the decompression process such as meteorological phenomena tracking, segmentation, and so on.

3D Reconstruction and 3D Point Cloud Compression

3D reconstruction and 3D Point Cloud Compression of image sequences is an important application in computer graphics and image processing, especially in biomedical engineering. Based on the introduction of the basic principle of 3D reconstruction, this paper presents a simple and easy implementation process of 3D reconstruction, and focuses on the key technologies of point cloud acquisition, finally, the future research directions of 3D Point Cloud Compression is discussed.

Adaptive Vector Quantization for Lossy Compression of Image Sequences

In this work, we present a scheme for the lossy compression of image sequences, based on the Adaptive Vector Quantization (AVQ) algorithm. The AVQ algorithm is a lossy compression algorithm for grayscale images, which processes the input data in a single-pass, by using the properties of the vector quantization to approximate data. First, we review the key aspects of the AVQ algorithm and, subsequently, we outline the basic concepts and the design choices behind the proposed scheme. Finally, we report the experimental results, which highlight an improvement in compression performances when our scheme is compared with the AVQ algorithm.

COMPRESSION OF IMAGE SEQUENCES IN INTERACTIVE MEDICAL TELECONSULTATIONS

Interactive medical teleconsultations are an important tool in the modern medical practice. Their applications include remote diagnostics, conferences, workshops and classes for students. In many cases standard medium or low-end machines are employed and the teleconsultation systems must be able to provide high quality of user experience with very limited resources. Particularly problematic are large datasets, consisting of image sequences, which need to be accessed fluently. The main issue is insufficient internal memory, therefore proper compression methods are crucial. However, a scenario where image sequences are kept in a compressed format in the internal memory and decompressed on-the-fly when displayed, is difficult to implement due to performance issues. In this paper we present methods for both lossy and lossless compression of medical image sequences, which require only compatibility with Pixel Shader 2.0 standard, which is present even on relatively old, low-end devices. Based on the evaluation of quality, size reduction and performance, the methods are proved to be suitable and beneficial for the medical teleconsultation applications.

Improved Image Sequence Compression Efficiency Through Various Techniques

Improved Image Sequence Compression Efficiency Through Various Techniques

A novel compression algorithm for infrared thermal image sequence based on K-means method

High resolution in space and time is becoming the new trend of thermographic inspection of equipments, therefore, the development of a fast and precise processing and data store technique of high resolution thermal image should be well studied. This article will propose a novel global compression algorithm, which will provide an effective way to improve the precision and processing speed of thermal image data. This new algorithm is based on the decay of the temperature of thermograph and the feature of thermal image morphology. Firstly, it will sort the data in space according to K-means method. Then it will employ classic fitting calculation to fit all the typical temperature decay curves. At last, it will use the fitting parameters of the curves as the parameters for compression and reconstruction of thermal image sequence to achieve the method for which the thermal image sequence can be compressed in space and time simultaneously. To validate the proposed new algorithm, the authors used two embedded defective specimens made of different materials to do the experiment. The results show that the proposed infrared thermal image sequence compression processing algorithm is an effective solution with high speed and high precision. Compared to the conventional method, the global compression algorithm is not only noise resistant but also can improve the computing speed in hundreds of times.

Research on the Compression Algorithm of the Infrared Thermal Image Sequence Based on Differential Evolution and Double Exponential Decay Model

This paper has proposed a new thermal wave image sequence compression algorithm by combining double exponential decay fitting model and differential evolution algorithm. This study benchmarked fitting compression results and precision of the proposed method was benchmarked to that of the traditional methods via experiment; it investigated the fitting compression performance under the long time series and improved model and validated the algorithm by practical thermal image sequence compression and reconstruction. The results show that the proposed algorithm is a fast and highly precise infrared image data processing method.

Screen image sequence compression method utilizing adaptive block size coding and hierarchical GOP structure

To compress screen image sequence in real-time remote and interactive applications, a novel compression method is proposed. The proposed method is named as CABHG. CABHG employs hybrid coding schemes that consist of intra-frame and inter-frame coding modes. The intra-frame coding is a rate-distortion optimized adaptive block size that can be also used for the compression of a single screen image. The inter-frame coding utilizes hierarchical group of pictures (GOP) structure to improve system performance during random accesses and fast-backward scans. Experimental results demonstrate that the proposed CABHG method has approximately 47%–48% higher compression ratio and 46%–53% lower CPU utilization than professional screen image sequence codecs such as TechSmith Ensharpen codec and Sorenson 3 codec. Compared with general video codecs such as H.264 codec, XviD MPEG-4 codec and Apple’s Animation codec, CABHG also shows 87%–88% higher compression ratio and 64%–81% lower CPU utilization than these general video codecs.

ENCODING VIDEO SEQUENCES IN FRACTAL-BASED COMPRESSION

With the rapid increase in the use of computers and the Internet, the demand for higher transmission and better storage is increasing as well. This paper describes the different techniques for data (image-video) compression in general and, in particular, the new compression technique called fractal image compression. Fractal image compression is based on self-similarity, where one part of an image is similar to the other part of the same image. Low bit rate color image sequence coding is very important for video transmission and storage applications. The most significant aspect of this work is the development of color images using fractal-based color image compression, since little work has been done previously in this area. The results obtained show that the fractal-based compression works for the color images works as well as for the gray-scale images. Nevertheless, the encoding of the color images takes more time than the gray-scale images. Color images are usually compressed in a luminance-chrominance coordinate space, with the compression performed independently for each coordinate by applying the monochrome image processing techniques. For image sequence compression, the design of an accurate and efficient algorithm for computing motion to exploit the temporal redundancy has been one of the most active research areas in computer vision and image compression. Pixel-based motion estimation algorithms address pixel correspondence directly by identifying a set of local features and computing a match between these features across the frames. These direct techniques share the common pitfall of high computation complexity resulting from the dense vector fields produced. For block matching motion estimation algorithms, the quad-tree data structure is frequently used in image coding to recursively decompose an image plane into four non-overlapping rectangular blocks.

Hardware implementation of a disparity estimation scheme for real-time compression in 3D imaging applications

This paper presents a novel hardware implementation of a disparity estimation scheme targeted to real-time Integral Photography (IP) image and video sequence compression. The software developed for IP image compression achieves high quality ratios over classic methodologies by exploiting the inherent redundancy that is present in IP images. However, there are certain time constraints to the software approach that must be confronted in order to address real-time applications. Our main effort is to achieve real-time performance by implementing in hardware the most time-consuming parts of the compression algorithm. The proposed novel digital architecture features minimized memory read operations and extensive simultaneous processing, while taking into concern the memory and data bandwidth limitations of a single FPGA implementation. Our results demonstrate that the implemented hardware system can successfully process high resolution IP video sequences in real-time, addressing a vast range of applications, from mobile systems to demanding desktop displays.

Synthetic Image Sequence Compression.

This paper describes a technique for compressing computer screen shots into a GIF animation file. The goal is to distribute the animations, to a variety of browsers, without requiring a plug-in or helper application. We seek to minimize the size of the image sequence, while maximizing the signal to noise ratio of the sequence. The GIF animation format has several constraints; images may have a maximum of 256 colors, and the images must all be of the same size. Further, to minimize overhead, we seek to make use of a single color lookup table for the entire animation. Several color quantization algorithms are compared, using SNR (Signal to Noise Ratio) as the metric of quality (as well as subjective appearance). We present the design of an interface, written in Java, and distributed freely using Java Web Start, that employs a well know neural network program and a color quantization algorithm to capture screen shots and save them to the GIF animation. GIF animations represent silent movies, as they have no sound to accompany them. However, they are still in wide use and have applications in entertainment and education. The techniques described are a part of the JSnap project, a joint project between the skunk works of DocJava, Inc. and Fairfield University.

A Wavelet-Based Sprite Codec

The International Standards Organization (ISO) has proposed a family of standards for compression of image and video sequences, including the JPEG, MPEG-1, and MPEG-2. The latest MPEG-4 standard has many new dimensions to coding and manipulation of visual content. A video sequence usually contains a background object and many foreground objects. Portions of this background may not be visible in certain frames due to the occlusion of the foreground objects or camera motion. MPEG-4 introduces the novel concepts of video object planes (VOPs) and Sprites. A VOP is a visual representation of real world objects with shapes that need not be rectangular. Sprite is a large image composed of pixels belonging to a video object visible throughout a video segment. Since a sprite contains all parts of the background that were at least visible once, it can be used for direct reconstruction of the background VOP. Sprite reconstruction is dependent on the mode in which it is transmitted. In the static sprite mode, the entire sprite is decoded as an Intra VOP before decoding the individual VOPs. Since sprites consist of the information needed to display multiple frames of a video sequence, they are typically much larger than a single frame of video. Therefore, a static sprite can be considered as a large static image. In this paper, a novel solution to address the problem of spatial scalability has been proposed, where the sprite is encoded in discrete wavelet transform (DWT). A lifting kernel method of DWT implementation has been used for encoding and decoding sprites. Modifying the existing lifting scheme while maintaining it to be shape-adaptive results in a reduced complexity. The proposed scheme has the advantages of: 1) avoiding the need for any extensions to image or tile border pixels and is hence superior to the discrete cosine transform-based low latency scheme (used in the current MPEG-4 verification model) and 2) mapping the in place computed wavelet coefficients into a zero-tree structure without actually rearranging them, thereby saving allocation of additional memory. The proposed solutions provide efficient implementation of the sprite decoder, making possible a VLSI realization with a reduced real estate.

A wavelet multiresolution compression technique for 3D stereoscopic image sequence based on mixed-resolution psychophysical experiments

Three-dimensional visual communications, made through the use of stereoscopic images, are able to achieve total display realism. In order to create a 3D system with two images (left and right) that should be transmitted simultaneously, the large amount of information contained in the images should be reduced, and an efficient coding appropriate for stereoscopic images is developed. Stereoscopic image sequence compression involves the exploitation of the spatial redundancy between the left and right image frames to achieve the compression ratio higher than that are by the independent compression of the two frames. In order to further achieve the higher compression ratio, in this paper, we employ the mixed-resolution psychophysical experiments to the stereo image compression. The experiments have shown that a stereo image pair with one high-resolution image and one lower-resolution image is sufficient to provide good stereoscopic depth perception. Thus, one image sequence is compressed independent of the other sequence using the motion compensation, while the other sequence is estimated at a lower resolution from this stream using the low-resolution disparity compensation. To implement the mixed-resolution coding, a wavelet multiresolution framework has been adopted to facilitate such an estimation of motion and disparity vectors at different resolutions. Experimental results indicate that the compression ratio for a typical stereoscopic image sequence is about 90, without any significant loss in the perceived 3D stereoscopic image quality.

Image sequence compression using adapted three-dimensional transform: Application to scintigraphic images

Abstract Compression results obtained by using the three-dimensional discrete cosine transform are presented. The key aspects of the paper concern the adaptation to the 3D case of transform coding algorithms that are most often used in the 2D case. This work focussed on two particular aspects that are the original contribution of this paper: (i) the adaptation to the 3D case of transform coefficients quantization method; (ii) the application of an octree analysis algorithm before transform. The algorithm was especially designed for tomographic images issued from scintigraphic measures, we finally obtained a compression rate of 100 for a quality measured by a 40 dB PSNR.

Adaptive Fractal Image Sequence Coding by Variable Shape Decomposition

A two-phase fractal image sequence compression system is proposed. In the classification phase, according to the texture attribution a testing solid image block is assigned to its corresponding texture class. The texture attribution is derived from the tomographic block projection classification for the finite projection directions at the three-dimensional (3D) space. In the adaptive coding phase, both the algorithm of the 3D projection classification and the 3D variable shape decomposition are incorporated into the variable shape block transformation for image sequence. By applying this variable shape block transformation algorithm to fractal image sequence coding scheme, we can obtain a promising performance.

New techniques on deformed image motion estimation and compensation

In this paper, new techniques for deformed image motion estimation and compensation using variable-size block-matching are proposed, which can be applied to an image sequence compression system or a moving object recognition system. The motion estimation and compensation techniques have been successfully applied in the area of image sequence coding. Many research papers on improving the performance of these techniques have been published; many directions are proposed, which can all lead to better performance than the conventional techniques. Among them, both generalized block-matching and variable-size block-matching are successfully applied in reducing the data rate of compensation error and motion information, respectively. These two algorithms have their merits, but suffer from their drawbacks. Moreover, reducing the data rate in compensation error is sometimes increasing the data rate in motion information, or vice versa. Based on these two algorithms, we propose and examine several algorithms which are effective in reducing the data rate. We then incorporate these algorithms into a system, in which they work together to overcome the disadvantages to individual and keep their merits at the same time. The proposed system can optimally balance the amount of data rate in two aspects (i.e., compensation error and motion information). Experimental results show that the proposed system outweighs the conventional techniques. Since we propose a recovery operation which tries to recover the incorrect motion vectors from the global motion, this proposed system can also be applied to the moving object recognition in image sequences.