Lossy Image Research Articles

Variable-length image compression based on controllable learning network

While neural-network-based lossy image compression methods have shown impressive performance, most of them output a fixed-length coding using a trained-specific network. However, it is essential to support the variable-length compression or meet a target rate with a high-coding performance in practice. This paper steps forward the neural-network-based image compression method, making it possible for a single network model to generate variable compression rates. Our network model combines an auto-encoder (AE) and a generative adversarial network (GAN) for generative compression. We introduce a noise interference mechanism to train the feature representation produced by the encoder, making the feature nodes training controllable and distributed from top to bottom according to their importance in feature expression. Based on this importance distribution, the latent nodes are quantized into bits and the variable-length compression can be achieved by discarding bits of those less-important feature nodes to meet the compression target. We propose several noise interference methods, and the experiments confirm the feasibility of method Random-add and Dropout in controllable learning. Further experiments illustrate that our compression method can not only achieve variable-length compression but also can recover high-quality compressed images at extremely low bit rates, outperforming that with a fixed rate.

Reduced-Complexity End-to-End Variational Autoencoder for on Board Satellite Image Compression

Recently, convolutional neural networks have been successfully applied to lossy image compression. End-to-end optimized autoencoders, possibly variational, are able to dramatically outperform traditional transform coding schemes in terms of rate-distortion trade-off; however, this is at the cost of a higher computational complexity. An intensive training step on huge databases allows autoencoders to learn jointly the image representation and its probability distribution, possibly using a non-parametric density model or a hyperprior auxiliary autoencoder to eliminate the need for prior knowledge. However, in the context of on board satellite compression, time and memory complexities are submitted to strong constraints. The aim of this paper is to design a complexity-reduced variational autoencoder in order to meet these constraints while maintaining the performance. Apart from a network dimension reduction that systematically targets each parameter of the analysis and synthesis transforms, we propose a simplified entropy model that preserves the adaptability to the input image. Indeed, a statistical analysis performed on satellite images shows that the Laplacian distribution fits most features of their representation. A complex non parametric distribution fitting or a cumbersome hyperprior auxiliary autoencoder can thus be replaced by a simple parametric estimation. The proposed complexity-reduced autoencoder outperforms the Consultative Committee for Space Data Systems standard (CCSDS 122.0-B) while maintaining a competitive performance, in terms of rate-distortion trade-off, in comparison with the state-of-the-art learned image compression schemes.

Two-Channel 3D Range Geometry Compression with Virtual Plane Encoding

Modern computing and imaging technologies have allowed for many recent advances to be made in the field of 3D range imaging: range data can now be acquired at speeds much faster than real-time, with sub-millimeter precision. However, these benefits come at the cost of an increased quantity of data being generated by 3D range imaging systems, potentially limiting the number of applications that can take advantage of this technology. One common approach to the compression of 3D range data is to encode it within the three color channels of a traditional 24-bit RGB image. This paper presents a novel method for the modification and compression of 3D range data such that the original depth information can be stored within, and recovered from, only two channels of a traditional 2D RGB image. Storage within a traditional image format allows for further compression to be realized via lossless or lossy image compression techniques. For example, when JPEG 80 was used to store the encoded output image, this method was able to achieve an 18.2% reduction in file size when compared to a similar three-channel, image-base compression method, with only a corresponding 0.17% reduction in global reconstruction accuracy.

Hybrid Encoding Scheme for AMBTC Compressed Images Using Ternary Representation Technique

Absolute moment block truncated coding (AMBTC) is a lossy image compression technique aiming at low computational cost, and has been widely studied. Previous studies have investigated the performance improvement of AMBTC; however, they often over describe the details of image blocks during encoding, causing an increase in bitrate. In this paper, we propose an efficient method to improve the compression performance by classifying image blocks into flat, smooth, and complex blocks according to their complexity. Flat blocks are encoded by their block means, while smooth blocks are encoded by a pair of adjusted quantized values and an index pointing to one of the k representative bitmaps. Complex blocks are encoded by three quantized values and a ternary map obtained by a clustering algorithm. Ternary indicators are used to specify the encoding cases. In our method, the details of most blocks can be retained without significantly increasing the bitrate. Experimental results show that, compared with prior works, the proposed method achieves higher image quality at a better compression ratio for all of the test images.

Mez: An Adaptive Messaging System for Latency-Sensitive Multi-Camera Machine Vision at the IoT Edge

Mez is a novel publish-subscribe messaging system for latency sensitive multi-camera machine vision applications at the IoT Edge. The unlicensed wireless communication in IoT Edge systems are characterized by large latency variations due to intermittent channel interference. To achieve user specified latency in the presence of wireless channel interference, Mez takes advantage of the ability of machine vision applications to temporarily tolerate lower quality video frames if overall application accuracy is not too adversely affected. Control knobs that involve lossy image transformation techniques that modify the frame size, and thereby the video frame transfer latency, are identified. Mez implements a network latency feedback controller that adapts to channel conditions by dynamically adjusting the video frame quality using the image transformation control knobs, so as to simultaneously satisfy latency and application accuracy requirements. Additionally, Mez uses an application domain specific design of the storage layer to provide low latency operations. Experimental evaluation on an IoT Edge testbed with a pedestrian detection machine vision application indicates that Mez is able to tolerate latency variations of up to 10x with a worst-case reduction of 4.2% of the application accuracy F1 score metric. The performance of Mez is also experimentally evaluated against state-of-the-art low latency NATS messaging system.

Rate-Distortion Optimized Encoding for Deep Image Compression

Deep-learned variational auto-encoders (VAE) have shown remarkable capabilities for lossy image compression. These neural networks typically employ non-linear convolutional layers for finding a compressible representation of the input image. Advanced techniques such as vector quantization, context-adaptive arithmetic coding and variable-rate compression have been implemented in these auto-encoders. Notably, these networks rely on an end-to-end approach, which fundamentally differs from hybrid, block-based video coding systems. Therefore, signal-dependent encoder optimizations have not been thoroughly investigated for VAEs yet. However, rate-distortion optimized encoding heavily determines the compression performance of state-of-the-art video codecs. Designing such optimizations for non-linear, multi-layered networks requires to understand the relationship between the quantization, the bit allocation of the features and the distortion. Therefore, this paper examines the rate-distortion performance of a variable-rate VAE. In particular, one demonstrates that the trained encoder network typically finds features with a near-optimal bit allocation across the channels. Furthermore, one approximates the relationship between distortion and quantization by a higher-order polynomial, whose coefficients can be robustly estimated. Based on these considerations, the authors investigate an encoding algorithm for the Lagrange optimization, which significantly improves the coding efficiency.

Distributed Learning and Inference With Compressed Images.

Modern computer vision requires processing large amounts of data, both while training the model and/or during inference, once the model is deployed. Scenarios where images are captured and processed in physically separated locations are increasingly common (e.g. autonomous vehicles, cloud computing, smartphones). In addition, many devices suffer from limited resources to store or transmit data (e.g. storage space, channel capacity). In these scenarios, lossy image compression plays a crucial role to effectively increase the number of images collected under such constraints. However, lossy compression entails some undesired degradation of the data that may harm the performance of the downstream analysis task at hand, since important semantic information may be lost in the process. Moreover, we may only have compressed images at training time but are able to use original images at inference time (i.e. test), or vice versa, and in such a case, the downstream model suffers from covariate shift. In this paper, we analyze this phenomenon, with a special focus on vision-based perception for autonomous driving as a paradigmatic scenario. We see that loss of semantic information and covariate shift do indeed exist, resulting in a drop in performance that depends on the compression rate. In order to address the problem, we propose dataset restoration, based on image restoration with generative adversarial networks (GANs). Our method is agnostic to both the particular image compression method and the downstream task; and has the advantage of not adding additional cost to the deployed models, which is particularly important in resource-limited devices. The presented experiments focus on semantic segmentation as a challenging use case, cover a broad range of compression rates and diverse datasets, and show how our method is able to significantly alleviate the negative effects of compression on the downstream visual task.

Impact of the lossy image compression on the biometric system accuracy: a case study of hand biometrics

Impact of the lossy image compression on the biometric system accuracy: a case study of hand biometrics

Reversible Data Hiding Scheme Based on VQ Prediction and Adaptive Parametric Binary Tree Labeling for Encrypted Images

In this paper, we propose a reversible data hiding scheme for the encrypted images (RDHEI) based on vector quantization (VQ) prediction and parametric binary tree labeling (PBTL). VQ compression is a lossy image compression method, the difference between the original image and the decompressed image is small when the length of codebook is sufficient. Thus, VQ can be applied as a tool for pixel value prediction. Based on VQ prediction, PBTL method is applied to label the embeddable and non-embeddable pixels. Through adaptive setting of parameters, the modified PBTL can provide optimal pixel labeling strategies and thus maximize the overall embedding capacity. Furthermore, the VQ index and the secret data are stream ciphered to avoid leakage of the image content and secret information. Different metrics are used to show that the marked encrypted images are highly secure. In comparison with several state-of-the-art schemes, our scheme outperforms the related works in embedding rate for two commonly applied image databases. In addition, extraction of the secret data and recovery of the original image can be operated separately according to authorization.

Learning End-to-End Lossy Image Compression: A Benchmark.

Image compression is one of the most fundamental techniques and commonly used applications in the image and video processing field. Earlier methods built a well-designed pipeline, and efforts were made to improve all modules of the pipeline by handcrafted tuning. Later, tremendous contributions were made, especially when data-driven methods revitalized the domain with their excellent modeling capacities and flexibility in incorporating newly designed modules and constraints. Despite great progress, a systematic benchmark and comprehensive analysis of end-to-end learned image compression methods are lacking. In this paper, we first conduct a comprehensive literature survey of learned image compression methods. The literature is organized based on several aspects to jointly optimize the rate-distortion performance with a neural network, i.e., network architecture, entropy model and rate control. We describe milestones in cutting-edge learned image-compression methods, review a broad range of existing works, and provide insights into their historical development routes. With this survey, the main challenges of image compression methods are revealed, along with opportunities to address the related issues with recent advanced learning methods. This analysis provides an opportunity to take a further step towards higher-efficiency image compression. By introducing a coarse-to-fine hyperprior model for entropy estimation and signal reconstruction, we achieve improved rate-distortion performance, especially on high-resolution images. Extensive benchmark experiments demonstrate the superiority of our model in rate-distortion performance and time complexity on multi-core CPUs and GPUs.

A Novel Multilevel Lossy Compression Algorithm for Grayscale Images Inspired by the Synthesization of Biological Protein Sequences

Enormous number of images are generated daily in all areas of life, including social media, medical and navigation images. Moreover, the development of smart phones among other specialized media-capturing devices has witnessed great advances during the last decade. Consequently, the storage, transmission, and analysis of images become essential and frequent tasks. Thus, various research efforts tried to address the image compression problem from different computational perspectives. This article presents a novel multilevel lossy compression algorithm for grayscale images, namely Image-as-Protein ( IaP ), that is inspired by the translation of DNA sequences into protein sequences that occurs inside live beings. Because of the high similarity of the resulting textual protein sequence, it can be tackled by general text compression techniques with competitive compression ratios. Various qualitative comparisons and quantitative measures such as BPP , SSIM and PSNR have been carried out on multiple grayscale image benchmark datasets. The experimental results showed that the proposed algorithm is promising compared to the famous JPEG lossy image compression standard.

Improvement of JPEG XL Lossy Image Coding Using Region Adaptive DCT Block Partitioning Structure

The Joint Picture Expert Group (JPEG) committee has been standardizing next-generation image compression, called JPEG XL, to meet the specific needs for a responsive web, wide color gamut, and high dynamic range. JPEG XL supports lossy and lossless compression. A variable-sized discrete cosine transform (DCT) block is used for lossy compression. A block partitioning method is regarded as a critical function for the performance of JPEG XL. The current DCT block partitioning method used in JPEG XL is highly dependent on the compression rate and tends to assign small-sized DCT blocks to homogeneously textured regions (HTRs) having similar or regular patterns. We propose a region-adaptive DCT block partitioning method that assigns larger blocks to the HTR. The proposed method identifies the HTRs by using a combined metric employing a sum-modified Laplacian, zero-crossing, and colorfulness metric for measuring the region homogeneity. Objective, subjective, and visual comparison evaluations with the ten images recommended by the JPEG working group were provided to show the improvement in coding performance. The proposed method shows its superiority in terms of the compression efficiency evaluated using six objective metrics, subjective tests with 15 participants, visual comparison improvements in the HTR, and gains in the execution time.

Impact of the lossy image compression on the biometric system accuracy: a case study of hand biometrics

Biometric recognition systems are used in several cases, to recognise people using images. Storing of large images require large storage space. To reduce the storage space, compression methods are employed. In this paper, we analyse the effect of lossy image compression on the performance of biometric identification systems. We propose a scheme to evaluate the recognition performance at low bitrates of hand images. The images are compressed using set partitioning in hierarchical trees (SPIHT) encoding. A powerful feature extraction algorithm based on quantising the phase information of the local Fourier transform is used. The nearest neighbour (NN) classifier and the support vector machine (SVM) classifier are employed to classify the feature extraction. The obtained results show at the compression does not significantly affect the performance of recognition operation at low bitrate for unimodal and multimodal systems. Thus, the low bitrate images perform equivalent to uncompressed images in the recognition system.

Lossy Image Compression Techniques

In recent years, domain image processing plays a most significant role in realtime applications. Due to reserved bandwidth and capacity, images are needed to be compressed, enhance, and noise-free to reduce the storage space as well as the transmission time. Compression overcomes the problem such as insufficient bandwidth of network and storage of memory device. Nowadays, many techniques are used to compress an image with a good quality. In this paper, I briefly discuss the concept of image compression and review the techniques which are used in loss image compression. This technique losses some data by using transform codes. Two codes are used to compress an image such as wavelet and fractal. The techniques scalar and vector quantization are used to quantize an input and partition into sub-images. I briefly discussed about the 2D-Discrete wavelet transform codes with examples which were done by Mat lab software.

BINet: A binary inpainting network for deep patch-based image compression

Recent deep learning models outperform standard lossy image compression codecs. However, applying these models on a patch-by-patch basis requires that each image patch be encoded and decoded independently. The influence from adjacent patches is therefore lost, leading to block artefacts at low bitrates. We propose the Binary Inpainting Network (BINet), an autoencoder framework which incorporates binary inpainting to reinstate interdependencies between adjacent patches, for improved patch-based compression of still images. When decoding a patch, BINet additionally uses the binarised encodings from surrounding patches to guide its reconstruction. In contrast to sequential inpainting methods where patches are decoded based on previous reconstructions, BINet operates directly on the binary codes of surrounding patches without access to the original or reconstructed image data. Encoding and decoding can therefore be performed in parallel. We demonstrate that BINet improves the compression quality of a competitive deep image codec across a range of compression levels.

Near-lossless image compression using an improved edge adaptive hierarchical interpolation

<span>In medical and scientific imaging, lossless image compression is recommended because the loss of minor details subject to medical diagnosis can lead to wrong diagniosis. On the other hand, lossy compression of medical images is required in the long run because a huge quantity of medical data needs remote storage. This, in turn, takes long time to search and transfer an image. Instead of thinking lossless or lossy image compression methods, near-loss image compression mehod can be used to compromise the two conflicting requirements. In the previous work, an edge adaptive hierarchical interpolation (EAHINT) was proposed for resolution scalable lossless compression of images. In this paper, it was enhanced for scalable near-less image compression. The interpolator of this arlgorithm swiches among one-directional, multi-directional and non-directional linear interpolators adaptively based on the strength of the edge in a 3x3 local casual context of the current pixel being predicted. The strength of the edge in local window was estimated using the variance of the the pixels in the local window. Although the actual predictors are still linear functions, the switching mechanism tried to deal with non-linear structures like edges. Simulation results demonstrate that the improved interpolation algorithm has better compression ratio over the the exsisting the original EAHINT algorithm and JPEG-Ls image compression standard. </span>

New Quality Metric for Compressed Images

The field of image processing has several applications in our daily life. The image quality can be affected by a wide variety of deformations during image acquisition, transmission, compression, etc. Image compression is one of the applications where the quality of the image plays an important role since it can be used to evaluate the performance of various image compression techniques. Many image quality assessment metrics have been proposed. This paper proposes a new metric to assess the quality of compressed images. The principle idea of this metric is to estimate the amount of lost information during image compression process using three components: error magnitude, error location and error distribution. We denote this metric as MLD, which combines the objective assessment (error magnitude) and the subjective assessment (error location and error distribution). First, the metric is used to estimate the quality of compressed images using the JPEG algorithm as this is a standard lossy image compression technique. Then, the metric is used to estimate the quality of compressed images using other compression techniques. The results illustrate that the proposed quality metric is correlated with the subjective assessment better than other well-known objective quality metrics such as SSIM, MSE and PSNR. Moreover, using the proposed metric the JPEG2000 algorithm produces better quality results as compared to the JPEG algorithm especially for higher compression ratios

Forward-looking sonar image compression by integrating keypoint clustering and morphological skeleton

Forward-Looking Sonar (FLS) is one of the most effective devices for underwater exploration which provides high-resolution images that can be used for several tasks in marine research, oceanographic, and deep-sea exploration. The limitation of current underwater acoustic channels does not allow transmitting these images in real-time, therefore image compression is required. Since acoustic images are characterized by speckle noise, an important challenge, in this area, is how to perform the compression while preserving relevant information. In this paper, a novel lossy forward-looking acoustic image compression method based on the combination between keypoint clustering and Morphological Skeleton (MS) is proposed. Keypoints are extracted by using A-KAZE feature extractor, while Density-Based Spatial Clustering of Application with Noise (DBSCAN) is used to find keypoint clusters representing a region-of-interest (ROI). Then, MS is executed to compact the ROI. The rest of the image is down-sampled and quantized through K-Means Clustering and represented via colour indexing. Finally, the information is compressed by using Brotli data compression. The experimental results on real FLS images demonstrate that our method achieves good outcomes in terms of quality metrics and compression ratio.

Towards PDE-Based Video Compression with Optimal Masks Prolongated by Optic Flow

Lossy image compression methods based on partial differential equations have received much attention in recent years. They may yield high-quality results but rely on the computationally expensive task of finding an optimal selection of data. For the possible extension to video compression, this data selection is a crucial issue. In this context, one could either analyse the video sequence as a whole or perform a frame-by-frame optimisation strategy. Both approaches are prohibitive in terms of memory and run time. In this work, we propose to restrict the expensive computation of optimal data to a single frame and to approximate the optimal reconstruction data for the remaining frames by prolongating it by means of an optic flow field. In this way, we achieve a notable decrease in the computational complexity. As a proof-of-concept, we evaluate the proposed approach for multiple sequences with different characteristics. In doing this, we discuss in detail the influence of possible computational setups. We show that the approach preserves a reasonable quality in the reconstruction and is very robust against errors in the flow field.

Augmenting Colonoscopy using Extended and Directional CycleGAN for Lossy Image Translation.

Colorectal cancer screening modalities, such as optical colonoscopy (OC) and virtual colonoscopy (VC), are critical for diagnosing and ultimately removing polyps (precursors of colon cancer). The non-invasive VC is normally used to inspect a 3D reconstructed colon (from CT scans) for polyps and if found, the OC procedure is performed to physically traverse the colon via endoscope and remove these polyps. In this paper, we present a deep learning framework, Extended and Directional CycleGAN, for lossy unpaired image-to-image translation between OC and VC to augment OC video sequences with scale-consistent depth information from VC, and augment VC with patient-specific textures, color and specular highlights from OC (e.g, for realistic polyp synthesis). Both OC and VC contain structural information, but it is obscured in OC by additional patient-specific texture and specular highlights, hence making the translation from OC to VC lossy. The existing CycleGAN approaches do not handle lossy transformations. To address this shortcoming, we introduce an extended cycle consistency loss, which compares the geometric structures from OC in the VC domain. This loss removes the need for the CycleGAN to embed OC information in the VC domain. To handle a stronger removal of the textures and lighting, a Directional Discriminator is introduced to differentiate the direction of translation (by creating paired information for the discriminator), as opposed to the standard CycleGAN which is direction-agnostic. Combining the extended cycle consistency loss and the Directional Discriminator, we show state-of-the-art results on scale-consistent depth inference for phantom, textured VC and for real polyp and normal colon video sequences. We also present results for realistic pendunculated and flat polyp synthesis from bumps introduced in 3D VC models.