Portable Graphics Research Articles

Classification of BPG-Based Lossy Compressed Noisy Images

Acquired remote sensing images can be noisy. This fact has to be taken into account in their lossy compression and classification. In particular, a specific noise filtering effect is usually observed due to lossy compression and this can be positive for classification. Classification can be also influenced by methodology of classifier learning. In this paper, we consider peculiarities of lossy compression of three-channel noisy images by better portable graphics (BPG) encoder and their further classification. It is demonstrated that improvement of data classification accuracy is not observed if a given image is compressed in the neighborhood of optimal operation point (OOP) and the classifier training is performed for the noisy image. Performance of neural network based classifier is studied. As demonstrated, its training for compressed remote sensing data is able to provide certain benefits compared to training for noisy (uncompressed) data. Examples for Sentinel data used in simulations are offered.

Semantic-empowered visible light communication for image transmission based on deep convolutional generative adversarial network.

In this paper, a semantic communication-based scheme was proposed to tackle the optimization challenge of transmission efficiency and link stability in indoor visible light communication (VLC) systems utilizing light-emitting diodes for image transmission. The semantic model, established by deep convolutional generative adversarial network (DCGAN) and vector quantization method, can effectively extract the essential characteristics of images. In addition, indoor VLC channel models including line-of-sight (LOS) and non-line-of-sight (NLOS) links are established in a 5*5*3 room, while incorporating noise interference encountered during signal transmission into the training process of the semantic model to enhance its anti-interference capability. Besides, the performance disparity between the semantic VLC system and the conventional VLC system using Better Portable Graphics (BPG) is assessed across various modulation formats, transmission distances, and receiving locations. The simulation results demonstrate that the semantic VLC system effectively enhances link stability and achieves a signal-to-noise ratio (SNR) gain exceeding 6 dB. Additionally, the semantic VLC system achieves superior performance compared to the conventional VLC system employing convolutional code at a rate of 1/2, without incurring additional bits consumption for error correction. Moreover, the practicality of the semantic VLC system is experimentally validated on an indoor physical testbed online enabled by the field programmable gate array (FPGA) with up to 1 Gbps data rate. In comparison with existing schemes, the semantic VLC system effectively reduces communication overhead by 40% while maintaining similar received image quality. Furthermore, it significantly enlarges the communication coverage of the VLC system by a factor of two without necessitating any hardware modifications and ensures stable transmission of images in high-speed scenarios.

BPG-Based Lossy Compression of Three-Channel Remote Sensing Images with Visual Quality Control

A tendency to increase the number of acquired remote sensing images and to make their average size larger has been observed. To manage such data, compression is needed, and lossy compression is often preferable. Since lossy compression introduces distortions, this results in worse classification and object detection. Therefore, lossy compression must be controlled, i.e., the introduced distortions must be under a certain limit. The distortions and the limit can be characterized by different metrics (quantitative criteria). Here, we consider the case of using the HaarPSI metric, which has a very high correlation with visual quality and human attention (saliency map), for three-channel optical band images compressed by the better portable graphics (BPG) encoder, one of the best modern compression techniques. We analyze a two-step procedure of providing a desired visual quality and show its peculiarities for the modes 4:4:4, 4:2:2, and 4:2:0 of image compression. We show how the HaarPSI metric relates to other known metrics of image visual quality and thresholds of distortion visibility. It is demonstrated that the two-step procedure provides about three times better accuracy in providing the desired visual quality compared to the fixed setting of parameter Q that controls compression for the BPG encoder. The provided accuracy is close to the reachable limit determined by the integer value setting of the Q parameter. We also briefly analyze the influence of compression on the classification accuracy of real-life remote sensing data.

Lossy Compression of Single-channel Noisy Images by Modern Coders

Lossy compression of remote-sensing images is a typical stage in their processing chain. In design or selection of methods for lossy compression, it is commonly assumed that images are noise-free. Meanwhile, there are many practical situations where an image or a set of its components are noisy. This fact needs to be taken into account since noise presence leads to specific effects in lossy compressed data. The main effect is the possible existence of the optimal operation point (OOP) shown for JPEG, JPEG2000, some coders based on the discrete cosine transform (DCT), and the better portable graphics (BPG) encoder. However, the performance of such modern coders as AVIF and HEIF with application to noisy images has not been studied yet. In this paper, analysis is carried out for the case of additive white Gaussian noise. We demonstrate that OOP can exist for AVIF and HEIF and the performance characteristics in it are quite similar to those for the BPG encoder. OOP exists with a higher probability for images of simpler structure and/or high-intensity noise, and this takes place according to different metrics including visual quality ones. The problems of providing lossy compression by AVIF or HEIF are shown and an initial solution is proposed. Examples for test and real-life remote-sensing images are presented.

PECULIARITIES OF NOISY IMAGE LOSSY COMPRESSION

This paper deals with lossy compression applied to grayscale noisy images using several coders. The case of additive white Gaussian noise is considered as the first step in studying the problem. A special attention is paid to quite new coders AVIF and HEIF for which, to the best of our knowledge, lossy compression of noisy images of different complexity has not been thoroughly considered yet. It is shown that optimal operation point is possible for all coders under certain conditions according to different quality metrics including conventional peak signal-to-noise ratio and visual quality metrics such as PSNR-HVS-M and MS-SSIM. The compression parameters (metrics) in optimal operation point significantly depend on image complexity and noise intensity where the existence of optimal operation point is more probable for simple structure images and/or intensive noise. Comparison of metric values in optimal operation point for the considered coders shows that slightly better characteristics are provided by better portable graphics (BPG) and advanced discrete cosine transform (ADCT) coders. The results for JPEG are significantly worse. The AVIF and HEIF encoders provide similar results and they outperform JPEG significantly. Optimal operation point for all studied coders is observed more rarely for visual quality metrics than for peak signal-to-noise ratio. General tendencies concerning dependence of optimal compression parameters on noise intensity are presented. It is shown that compression ratio in optimal operation point is often larger than 20 and can be as large as 60. The problem with AVIF and HEIF is that it is currently unclear how to choose quality factor for them to carry out lossy image compression in the neighborhood of the corresponding optimal operation point. Meanwhile, there is the tendency to optimal quality factor reduction if intensity of additive white Gaussian noise increases. The directions of further research are discussed.

Post-processing of compressed noisy images by BM3D filter

Acquired images are often noisy. Since the amount of such images increases, they should be compressed where lossy compression is often applied for several reasons. Such compression is associated with the phenomena of specific image filtering due to lossy compression and the possible existence of an optimal operation point (OOP). However, such filtering is not perfect, and residual noise can be quite intensive even if an image is compressed at the so-called optimal operation point. Then, additional post-filtering can be applied. Thus, the basic subject of this paper is the post-processing of noisy images compressed in a lossy manner. The main goal of this paper is to consider the possible application of a block-matching 3-dimensional (BM3D) filter to images corrupted by additive white Gaussian noise compressed by a better portable graphics (BPG) coder with a compression ratio smaller than that for the optimal operation point and in OOP neighborhood. The tasks of this paper are to analyze the efficiency of compressed image post-processing depending on noise intensity, image complexity, coder compression parameter Q, and filter threshold parameter β according to different quality metrics and to provide practical recommendations on setting the filter and coder parameters. The main result is that the post-processing efficiency decreases when the coder compression parameter increases and becomes negligible for a coder compression parameter slightly larger than its value for OOP. The post-processing efficiency is larger for simpler structure images and larger noise intensity. Compressed image quality due to post-processing improves according to the standard criterion peak signal-to-noise ratio and visual quality metrics. For larger coder compression parameters, the optimal threshold shifts toward smaller values. In conclusion, we demonstrate the efficiency of post-processing and show that the BM3D filter outperforms the standard discrete cosine-based (DCT) filter. We also provide recommendations for filter parameter setting. We also outline possible research directions for the future.

Analysis of the potential efficiency of post-filtering noisy images after lossy compression

An increase in the number of images and their average size is the general trend nowadays. This increase leads to certain problems with data storage and transfer via communication lines. A common way to solve this problem is to apply lossy compression that provides sufficiently larger compression ratios compared to lossless compression approaches. However, lossy compression has several peculiarities, especially if a compressed image is corrupted by quite intensive noise. First, a specific noise-filtering effect is observed. Second, an optimal operational point (OOP) might exist where the quality of a compressed image is closer to the corresponding noise-free image than the quality of the original image according to a chosen quality metric. In this case, it is worth compressing this image in the OOP or its closest neighborhood. These peculiarities have been earlier studied and their positive impact on image quality improvement has been demonstrated. Filtering of noisy images due to lossy compression is not perfect. Because of this, it is worth checking can additional quality improvement be reached using such an approach as post-filtering. In this study, we attempt to answer the questions: “is it worth to post-filter an image after lossy compression, especially in OOP’s neighborhood? And what benefit can it bring in the sense of image quality?”. The study is carried out for better portable graphics (BPG) coder and the DCT-based filter focusing mainly on one-component (grayscale) images. The quality of images is characterized by several metrics such as PSNR, PSNR-HVS-M, and FSIM. Possible image quality increasing via post-filtering is demonstrated and the recommendations for filter parameter setting are given.

BPG-Based Lossy Compression of Three-Channel Noisy Images with Prediction of Optimal Operation Existence and Its Parameters

Nowadays, there is a clear trend toward increasing the number of remote-sensing images acquired and their average size. This leads to the need to compress the images for storage, dissemination, and transfer over communication lines where lossy compression techniques are more popular. The images to be compressed or some of their components are often noisy. They must therefore be compressed taking into account the properties of the noise. Due to the noise filtering effect obtained during lossy compression of noisy images, an optimal operating point (OOP) may exist. The OOP is a parameter that controls the compression for which the quality of the compressed image is closer (closest) to the corresponding noise-free image than the quality of the noisy (original, uncompressed) image according to some quantitative criterion (metric). In practice, it is important to know whether the OOP exists for a given image, because if the OOP exists, it is appropriate to perform the compression in the OOP or at least in its neighborhood. Since the real image is absent in practice, it is impossible to determine a priori whether the OOP exists or not. Here, we focus on three-channel-remote-sensing images and show that it is possible to easily predict the existence of the OOP. Furthermore, it is possible to predict the metric values or their improvements with appropriate accuracy for practical use. The BPG (better portable graphics) encoder is considered a special case of an efficient compression technique. As an initial design step, the case of additive white Gaussian noise with equal variance in the three components is considered. While previous research was mainly focused on predicting the improvement (reduction) of the PSNR and PSNR-HVS-M metrics, here we focus on the modern visual quality metrics, namely PSNR-HA and MDSI. We also discuss what to do if, according to the prediction, an OOP is absent. Examples of lossy compression of noisy three-channel remote sensing images are given. It is also shown that the use of three-dimensional compression provides a compression ratio increase by several times compared with component-wise compression in the OOP.

Usage of different Chroma Subsampling Modes in Image Compression by BPG Coder

A BPG (better portable graphics) coder is a novel approach that aims to replace common standards of compression such as JPEG, JPEG2000 and so on. That is why, the BPG coder needs a detailed analysis of its basic characteristics from the viewpoint of visual quality and compression ratio. The BPG coder can use different modes of chroma subsampling for color and three-channel images and it is worth analyzing and comparing them. In practice, images to be compressed are often noisy. Then, lossy compression of such images has a specific noise filtering effect. In particular, optimal operation point (OOP) might exist where compressed image quality is closer to the corresponding noise-free (true) image than uncompressed (original, noisy) image quality according to certain criterion (metric). It is also needed to analyze the coder performance from compression ratio point of view. In this paper, we pay attention on impact of different chroma subsampling modes on image quality and compression ratio. Based on simulation results obtained for a set of color images, the best possible ways of compression are recommended.

BPG-Based Automatic Lossy Compression of Noisy Images with the Prediction of an Optimal Operation Existence and Its Parameters

With a resolution improvement, the size of modern remote sensing images increases. This makes it desirable to compress them, mostly by using lossy compression techniques. Often the images to be compressed (or some component images of multichannel remote sensing data) are noisy. The lossy compression of such images has several peculiarities dealing with specific noise filtering effects and evaluation of the compression technique’s performance. In particular, an optimal operation point (OOP) may exist where quality of a compressed image is closer to the corresponding noise-free (true) image than the uncompressed (original, noisy) image quality, according to certain criterion (metrics). In such a case, it is reasonable to automatically compress an image under interest in the OOP neighborhood, but without having the true image at disposal in practice, it is impossible to accurately determine if the OOP does exist. Here we show that, by a simple and fast preliminary analysis and pre-training, it is possible to predict the OOPs existence and the metric values in it with appropriate accuracy. The study is carried out for a better portable graphics (BPG) coder for additive white Gaussian noise, focusing mainly on one-component (grayscale) images. The results allow for concluding that prediction is possible for an improvement (reduction) in the quality metrics of PSNR and PSNR-HVS-M. In turn, this allows for decision-making about the existence or absence of an OOP. If an OOP is absent, a more “careful” compression is recommended. Having such rules, it then becomes possible to carry out the compression automatically. Additionally, possible modifications for the cases of signal-dependent noise and the joint compression of three-component images are considered and the possible existence of an OOP for these cases is demonstrated.

Prediction of Parameters in Optimal Operation Point for BPG-based Lossy Compression of Noisy Images

Lossy compression of images corrupted by noise has several peculiarities. First, a specific noise filtering effect is observed. Second, optimal operation point (OOP) can be observed, i.e. such coder parameter (e.g., quantization step) value can exist that quality of compressed image calculated with respect to noise-free image can be better compared to quality of uncompressed (original noisy) image. If OOP exists, it is worth compressing a given image in OOP, if no, other recommendations on coder parameter setting are reasonable. Since noise-free image is not available in practice, it is not possible to determine does OOP exist and what is image quality in it. In this paper, we show that OOP existence for several quality metrics can be predicted quite easily and quickly for grayscale images corrupted by additive white Gaussian noise and compressed by better portable graphics (BPG) encoder. Such a prediction is based on analysis of statistics of discrete cosine transform (DCT) coefficients calculated for a limited number of 8x8 pixel blocks. A scatter-plot of metric improvement (reduction) depending upon these statistics is obtained in advance and prediction curve fitting is performed. Recommendations on encoder parameter setting for cases of OOP absence are given.

OFDM-Guided Deep Joint Source Channel Coding for Wireless Multipath Fading Channels

We investigate joint source channel coding (JSCC) for wireless image transmission over multipath fading channels. Inspired by recent works on deep learning based JSCC and model-based learning methods, we combine an autoencoder with orthogonal frequency division multiplexing (OFDM) to cope with multipath fading. The proposed encoder and decoder use convolutional neural networks (CNNs) and directly map the source images to complex-valued baseband samples for OFDM transmission. The multipath channel and OFDM are represented by non-trainable (deterministic) but differentiable layers so that the system can be trained end-to-end. Furthermore, our JSCC decoder further incorporates explicit channel estimation, equalization, and additional subnets to enhance the performance. The proposed method exhibits 2.5 – 4 dB SNR gain for the equivalent image quality compared to conventional schemes that employ state-of-the-art but separate source and channel coding such as Better Portable Graphics (BPG) and Low-Density Parity-Check (LDPC) schemes. The performance further improves when the system incorporates the channel state information (CSI) feedback. The proposed scheme is robust against OFDM signal clipping and parameter mismatch for the channel model used in training and evaluation.

Quality Control for the BPG Lossy Compression of Three-Channel Remote Sensing Images

This paper deals with providing the desired quality in the Better Portable Graphics (BPG)-based lossy compression of color and three-channel remote sensing (RS) images. Quality is described by the Mean Deviation Similarity Index (MDSI), which is proven to be one of the best metrics for characterizing compressed image quality due to its high conventional and rank-order correlation with the Mean Opinion Score (MOS) values. The MDSI properties are studied and three main areas of interest are determined. It is shown that quite different quality and compression ratios (CR) can be observed for the same values of the quality parameter Q that controls compression, depending on the compressed image complexity. To provide the desired quality, a modified two-step procedure is proposed and tested. It has a preliminary stage carried out offline (in advance). At this stage, an average rate-distortion curve (MDSI on Q) is obtained and it is available until the moment when a given image has to be compressed. Then, in the first step, an image is compressed using the starting Q determined from the average rate-distortion curve for the desired MDSI. After this, the image is decompressed and the produced MDSI is calculated. In the second step, if necessary, the parameter Q is corrected using the average rate-distortion curve, and the image is compressed with the corrected Q. Such a procedure allows a decrease in the MDSI variance by around one order after two steps compared to variance after the first step. This is important for the MDSI of approximately 0.2–0.25 corresponding to the distortion invisibility threshold. The BPG performance comparison to some other coders is performed and examples of its application to real-life RS images are presented.

Recognition-Aware Learned Image Compression

Learned image compression methods generally optimize a rate-distortion loss, trading off improvements in visual distortion for added bitrate. Increasingly, however, compressed imagery is used as an input to deep learning networks for various tasks such as classification, object detection, and superresolution. We propose a recognition-aware learned compression method, which optimizes a rate-distortion loss alongside a task-specific loss, jointly learning compression and recognition networks. We augment a hierarchical autoencoder-based compression network with an EfficientNet recognition model and use two hyperparameters to trade off between distortion, bitrate, and recognition performance. We characterize the classification accuracy of our proposed method as a function of bitrate and find that for low bitrates our method achieves as much as 26% higher recognition accuracy at equivalent bitrates compared to traditional methods such as Better Portable Graphics (BPG).

Human Perception Measurement by Electroencephalography for Facial Image Compression

Facial images are the main focused contents in video conferences and many other applications. Therefore, it turns to a critical issue to measure and maintain the perceptual quality of facial images if transmitted over a bandwidth-limited communication system. In this letter, we propose a regional distortion perceptual threshold measurement model based on electroencephalography (EEG) to establish the relationship between image quality and human perception. Then, a facial image compression method is presented based on the model to improve the perceptual quality. Specifically, we construct a facial image dataset with regional distortion using the better portable graphics (BPG) compression. With the dataset, we design an EEG experiment and collect the brain responses to measure the human perception on regional distortions. By this method, a regional distortion perceptual threshold map (RDPTM) is constructed to guide the data rate allocation process for different regions of facial images. The experimental results show that our method can measure the human perception of regional distortion using EEG and improve the image perceptual quality by data rate allocation based on the RDPTM.

Використання візуальних метрик для аналізу стиснення з втратами зашумлених зображень

The subject of the article is to analyze the effectiveness of lossy image compression using a BPG encoder using visual metrics as a quality criterion. The aim is to confirm the existence of an operating point for images of varying complexity for visual quality metrics. The objectives of the paper are the following: to analyze for a set of images of varying complexity, where images are distorted by additive white Gaussian noise with different variance values, build and analyze dependencies for visual image quality metrics, provide recommendations on the choice of parameters for compression in the vicinity of the operating point. The methods used are the following: methods of mathematical statistics; methods of digital image processing. The following results were obtained. Dependencies of visual quality metrics for images of various degrees of complexity affected by noise with variance equal to 64, 100, and 196. It can be seen from the constructed dependence that a working point is present for images of medium and low complexity for both the PSNR-HVS-M and MS-SSIM metrics. Recommendations are given for choosing a parameter for compression based on the obtained dependencies. Conclusions. Scientific novelty of the obtained results is the following: for a new compression method using Better Portable Graphics (BPG), research has been conducted and the existence of an operating point for visual quality metrics has been proven, previously such studies were conducted only for the PSNR metric.The test images were distorted by additive white Gaussian noise and then compressed using the methods implemented in the BPG encoder. The images were compressed with different values of the Q parameter, which made it possible to estimate the image compression quality at different values of compression ratio. The resulting data made it possible to visualize the dependence of the visual image quality metric on the Q parameter. Based on the obtained dependencies, it can be concluded that the operating point is present both for the PSNR-HVS-M metric and for the MS-SSIM for images of medium and low complexity, it is also worth noting that, especially clearly, the operating point is noticeable at large noise variance values. As a recommendation, a formula is presented for calculating the value of the compression control parameter (for the case with the BPG encoder, it is the Q parameter) for images distorted by noise with variance varying within a wide range, on the assumption that the noise variance is a priori known or estimated with high accuracy.

An entropy minimization histogram mergence scheme and its application in image compression

In this paper, we propose an entropy minimization histogram mergence (EMHM) scheme that can significantly reduce the number of grayscales with nonzero pixel populations (GSNPP) without visible loss to image quality. We proved in theory that the entropy of an image is reduced after histogram mergence and that the reduction in entropy is maximized using our EMHM. The reduction in image entropy is good for entropy encoding considering that the minimum average code word length per source symbol is the entropy of the source signal according to Shannon’s first theorem. Extensive experimental results show that our EMHM can significantly reduce the code length of entropy coding, such as Huffman, Shannon, and arithmetic coding, by over 20% while preserving the image subjective and objective quality very well. Moreover, the performance of some classic lossy image compression techniques, such as the Joint Photographic Experts Group (JPEG), JPEG2000, and Better Portable Graphics (BPG), can be improved by preprocessing images using our EMHM.

Image compression based on Gaussian mixture model constrained using Markov random field

We introduce a Gaussian Mixture Model (GMM) constrained by Markov Random Field (MRF) framework for image compression in this paper. The image is predicted using GMM with MRF and the parameters of the GMM are estimated using an adjusted Expectation-Maximization (EM) algorithm. Mixture Model Optimization (MMO) is used in this framework to select the optimal number of distributions and avoid local optimum of EM at the same time. Parameters are encoded using fixed-length bits. A codebook is used to improve the coding efficiency of the covariance parameters. The residual between the original image and the prediction is encoded using High Efficiency Video Coding (HEVC) intra coding. Experimental results show that our method performs better than our previous work, HEVC, JPEG 2000 and Better Portable Graphics (BPG) which is an improved version of HEVC.

Artificial neural network applications for data compression in video data transfer protocols

The article describes the current state of data transfer protocols and methods of image and video compression through the use of artificial neural networks, namely convolutional multilayer networks and deep structured learning. Based on recent publications, a comparative analysis of the performance of classical compression methods and methods based on neural networks was performed. The most effective are those compression methods which are based on decorrelation transforms, namely discrete cosine (JPEG standard) and wavelet (JPEG-2000 standard) transforms. The transform coefficients have a well-understood physical content of spatial frequencies and can be further quantized for a more optimal representation of components that are less important for human perception. The HEVC standard guarantees a more efficient image compression scheme that further takes advantage of the similarity of adjacent blocks and uses interpolation (intracoding). Based on the HEVC standard, the BPG (better portable graphics) format was developed to be used on the Internet as an alternative to JPEG, which is much more efficient than other standards. An overview of the current state of open standards, provided in the article, gives an explanation of what properties of neural networks can be applied to image compression. There are two approaches towards the compression using neural networks: in case of the first approach neural network is used as a part of an existing algorithm (hybrid coding), and in case of the second approach the neural network gives a concise representation of the data (compression network). The final conclusions were made as regards to the application of these algorithms in H.265 protocol (HEVC) and the possibility of creating a new protocol which is completely based on the neural network. Protocols using neural network show better results during image compression, but are currently hard to be subjected to standardization in order to obtain the expected result in case of different network architects. We may expect and predict an increase in the need for video transmission in the future, which will bump into the imitating nature of classical approaches. At the same time, the development of specialized processors for parallel data processing and implementation of neural networks is currently underway. These two factors indicate that neural networks must be embedded into the industrial data standards.

Technique for lossless compression of color images based on hierarchical prediction, inversion, and context adaptive coding

Among the variety of multimedia formats, color images play a prominent role. A technique for lossless compression of color images is introduced. The technique is composed of first transforming a red, green, and blue image into luminance and chrominance domain (Y C u C v). Then, the luminance channel Y is compressed with a context-based, adaptive, lossless image coding technique (CALIC). After processing the chrominance channels with a hierarchical prediction technique that was introduced earlier, Burrows–Wheeler inversion coder or JPEG 2000 is used in compression of those C u and C v channels. It is demonstrated that, on a wide variety of images, particularly on medical images, the technique achieves substantial compression gains over other well-known compression schemes, such as CALIC, M-CALIC, Better Portable Graphics, JPEG-LS, JPEG 2000, and the previously proposed hierarchical prediction and context-adaptive coding technique LCIC.