Kodak Dataset Research Articles

Contrast enhancement of digital images using dragonfly algorithm

Contrast enhancement aims to amplify the visual quality of images by modifying the contrast level because digital images may get distorted by casual acquisition. The article deals with contrast enhancement as an optimization problem and uses the Dragonfly Algorithm (DA) to find the optimal grey-level intensity values. The DA for contrast enhancement uses five control parameters (entropy, number of edges, total intensities of edges, the variance of the probability of occurrence of each grey value, and the number of grey levels) to generate an objective function. An ablation study is also performed to understand how different controlling parameter combinations contribute to determining the optimal solution. The proposed approach considers 24 grey-scale images from the Kodak dataset and metrics as Peak Signal-to-Noise Ratio (PSNR), Visual Information Fidelity (VIF), and Structural Similarity Index Measure (SSIM) to verify the output's performance. The PSNR, VIF, and SSIM values in the experiments are 30.87, 0.7451, and 0.9523, respectively. The experimental observations reveal that the proposed DA-based image contrast enhancement produces high-quality images from its low-contrast counterparts. Comparisons with state-of-art methods ensure the superiority of the proposed algorithm. The Python implementation of the proposed approach is available in this Github repository.

Reversible Data Hiding for Color Images Using Channel Reference Mapping and Adaptive Pixel Prediction

Reversible data hiding (RDH) is a technique that embeds secret data into digital media while preserving the integrity of the original media and the secret data. RDH has a wide range of application scenarios in industrial image processing, such as intellectual property protection and data integrity verification. However, with the increasing prevalence of color images in industrial applications, traditional RDH methods for grayscale images are inadequate to meet the requirements of image fidelity. This paper proposes an RDH method for color images based on channel reference mapping (CRM) and adaptive pixel prediction. Initially, the CRM mode for a color image is established based on the pixel variation correlation between the RGB channels. Then, the pixel local complexity context is adaptively selected using the CRM mode. Next, each pixel value is adaptively predicted based on the features and characteristics of adjacent pixels and reference channels, and then data is embedded by expanding the prediction error. Finally, we compare seven existing RDH algorithms on the standard image dataset and the Kodak dataset to validate the advantages of our method. The experimental results demonstrate that our approach achieves average peak signal-to-noise ratio (PSNR) values of 63.61 and 60.53 dB when embedding 20,000 and 40,000 bits of data, respectively. These PSNR values surpass those of other RDH methods. These findings indicate that our method can effectively preserve the visual quality of images even under high embedding capacities.

Image Compression Scheme based on Optimized K-means Clustering and Higher-Level Decomposed DWT

Image processing is a vital research domain essential in day-to-day digital activities. Image compression reduces memory space, transmission bandwidth requirement, transmission time, and computational complexity. This article aims to construct and compare hybrid image compression schemes using DWT, PCA, and K-means clustering techniques. This research proposes an image compression scheme based on optimized K-means clustering and higher-level decomposed DWT (K-DWT). Also, we have developed the hybrid image compression schemes based on PCA and DWT (P-DWT), PCA, K-means, and DWT (P-K-DWT) and analyzed the performance. Initially, we implemented compression algorithms using K-means Clustering, PCA, and DWT. Further, using the python platform, K-DWT, P-DWT, and P-K-DWT compression models have been executed on the images of the kodak dataset. The research objective is to obtain good image quality and better compression algorithms. The image quality and compression efficiency are evaluated using Peak Signal to Noise Ratio (PSNR), Structural Similarity Index Measurement (SSIM), and Compression Ratio (CR). The K-DWT methods have achieved a higher compression ratio, i.e., 15.3870, compared with other methods. Also, the outcomes show that the K-DWT method has provided the required values of PSNR and SSIM.

PVO-Based Reversible Data Hiding Using Global Sorting and Fixed 2D Mapping Modification

Pixel-value-ordering (PVO) is one of the most popular methods in reversible data hiding (RDH). In PVO based methods, pixels are processed in a block-wise way, so that the local similarities of the images are considered but the global statistical characteristics are ignored. To better utilize the correlations of pixels, this paper proposes a global sorting strategy to combine utilizations of local and global characteristics of the images. For each pixel, its prediction value and local complexity are first calculated based on its local characteristics. Then the image pixels are sorted globally according to their prediction values to generate a single-sorted pixel sequence, in which the pixels with the equal prediction values are sorted again by referring to their local complexities. In such a way, the spatial distances of image pixels are broken so that the global statistical characteristics can be well exploited. With the proposed sorting strategy, we can obtain a more regular 2D histogram by segmenting the sorted sequence for the location-based PVO (LPVO) predictor. Owe to the regular 2D histogram, we have designed an efficient 2D mapping to achieve perfect performance for all the tested images. With the proposed RDH scheme, the PSNR of the image Lena is as high as 61.86 dB and the average PSNR of the Kodak dataset reaches 63.55 dB after embedding 10,000 bits. The superiority of the proposed method has been verified by comparing with recent state-of-the-art RDH methods.

Autoencoder-based joint image compression and encryption

Recently, learning-based methods have developed rapidly, with promising results in many areas. And deep learning models are expected to be the next-generation optimal image compression solutions. This paper proposes a new image compression and encryption framework that integrates encryption algorithms with a deep-learning compression network. Our work employs Auto-Encoder (AE) based compression network as the backbone. After extracting semantic features from the learning network, encryption layers are added to introduce end-to-end learning to image encryption. And for a higher level of visual security, the parameters of the synthesis network are replaced by a new parameter matrix based on a logistic map controlled by a secret key. The encryption key of the system is derived from the image content, which will be embedded in the deep feature vectors to save the cost of sending the key for different images. Another secret key, which will be shared among other images, is used for controlling the embedding process. And to learn the entropy model from the scrambled feature maps, an attention scheme is exploited in estimating parameters to achieve more effective compression. Experimental results and the security analysis using Kodak dataset show that the proposed scheme can achieve a sufficiently high-security level with high compression efficiency.

A Universal Optimization Framework for Learning-based Image Codec

Recently, machine learning-based image compression has attracted increasing interests and is approaching the state-of-the-art compression ratio. But unlike traditional codec, it lacks a universal optimization method to seek efficient representation for different images. In this paper, we develop a plug-and-play optimization framework for seeking higher compression ratio, which can be flexibly applied to existing and potential future compression networks. To make the latent representation more efficient, we propose a novel latent optimization algorithm to adaptively remove the redundancy for each image. Additionally, inspired by the potential of side information for traditional codecs, we introduce side information into our framework, and integrate side information optimization with latent optimization to further enhance the compression ratio. In particular, with the joint side information and latent optimization, we can achieve fine rate control using only single model instead of training different models for different rate-distortion trade-offs, which significantly reduces the training and storage cost to support multiple bit rates. Experimental results demonstrate that our proposed framework can remarkably boost the machine learning-based compression ratio, achieving more than 10% additional bit rate saving on three different representative network structures. With the proposed optimization framework, we can achieve 7.6% bit rate saving against the latest traditional coding standard VVC on Kodak dataset, yielding the state-of-the-art compression ratio.

Taylor series based dual-branch transformation for learned image compression

Nonlinear transformations (NTs) are a crucial element of learned image compression (LIC) as they eliminate correlations from images. However, there has been limited focus on developing efficient NTs. This paper aims to propose a design approach for efficient NTs. In this paper, firstly, we model the current NT methods with a unified formulation and find that the difference in the NTs lies in the representation of the scaling function. Subsequently, we analyze the essential elements of designing scaling functions from the perspective of the Taylor series of derivable functions to satisfy the conditions of the Taylor expansion. Lastly, we develop an efficient two-branch nonlinear transformation architecture based on the Taylor series of sinusoidal functions. We compare the performance of various existing nonlinear transformations in the same backbone network. The experimental results demonstrate that our method achieves advanced performance among existing NT methods without a significant increase in encoding and decoding time. For instance, our method achieves a 20.24% BD-rate reduction over BPG444 on the Kodak dataset. The generic capability of our method is validated through experimental results on multiple backbone architectures.

MR-Net: Multiresolution sinusoidal neural networks

We present MR-Net, a general architecture for multiresolution sinusoidal neural networks, and a framework for imaging applications based on this architecture. We extend sinusoidal networks, and we build an infrastructure to train networks to represent signals in multiresolution. Our coordinate-based networks, namely L-Net, M-Net, and S-Net, are continuous both in space and in scale as they are composed of multiple stages that progressively add finer details. Currently, band-limited coordinate networks (BACON) are able to represent signals at multiscale by limiting their Fourier spectra. However, this approach introduces artifacts leading to an image with a ringing effect. We show that MR-Net can represent more faithfully what is expected of sequentially applying low-pass filters in a high-resolution image. Our experiments on the Kodak Dataset show that MR-Net can reach comparable Peak Signal-to-Noise Ratio (PSNR) to other architectures, on image reconstruction, while needing fewer additional parameters for multiresolution. Along with MR-Net, we detail our architecture’s mathematical foundations and general ideas, and show examples of applications to texture magnification, minification, and antialiasing. Lastly, we compare our three MR-Net subclasses.

Nonlinear Transforms in Learned Image Compression From a Communication Perspective

Recently, remarkable progress has been made in learned image compression (LIC), in which nonlinear transforms (NTs) play a crucial role. Although there are many NT methods for improving the rate distortion performance, all the existing methods sacrifice the computational complexity and the number of parameters of the transformation. This paper provides a fundamental novel viewpoint on nonlinear transforms from a communication perspective, and shows how this idea can be extended to design efficient NT methods. In particular, the nonlinear transforms are inferred as signal modulation modules. Under this extrapolation, the current NTs are generalized as amplitude modulation that only varies the amplitude of the carrier wave. Therefore, a nonlinear modulation-like transform (NMLT) which varies the phase angle of the carrier is proposed. Moreover, this concept is extended by introducing In-phase/Quadrature (IQ) modulation, which is a boosting technique in communication field, in order to enhance NMLT. Furthermore, the Bit-interleaved technique in communication is used to guide the optimization of NTML with IQ. The experimental results on different datasets and backbone architectures verify the efficiency and robustness of the proposed methods. For example, when backbone architecture is hyperprior model, our method achieves 19.37% BD-rate reduction over GDN on the Kodak dataset. In addition, our method with channel wise autoregressive model leads to the state-of-the-art rate-distortion performance.

Fuzzy Rule Based Adaptive Block Compressive Sensing for WSN Application

Transmission of high volume of data in a restricted wireless sensor network (WSN) has come up as a challenge due to high-energy consumption and larger bandwidth requirement. To address the issues of high-energy consumption and efficient data transmission adaptive block compressive sensing (ABCS) is one of the optimum solution. ABCS framework is well capable to adapt the sampling rate depending on the block’s features information that offers higher sampling rate for less compressible blocks and lower sampling rate for more compressible blocks In this paper, we have proposed a novel fuzzy rule based adaptive compressive sensing approach by leveraging the saliency and the edge features of the image making the sampling rate selection completely automatic. Adaptivity of the block sampling ratio has been decided based on the fuzzy logic system (FLS) by considering two important features i.e., edge and saliency information. The proposed framework is experimented on standard dataset, Kodak data set, CCTV images and the Set5 data set images. It achieved an average PSNR of 34.26 and 33.2 and an average SSIM of 0.87 and 0.865 for standard images and CCTV images respectively. Again for high resolution Kodak data set and Set 5 dataset images, it achieved an average PSNR of 32.95 and 31.72 and SSIM of 0.832 and 0.8 respectively. The experiments and the result analysis show that proposed method is efficacious than the state of the art methods in both subjective and objective evaluation metrics.

Improving Image Compression With Adjacent Attention and Refinement Block

Recently, learned image compression algorithms have shown incredible performance compared to classic hand-crafted image codecs. Despite its considerable achievements, the fundamental disadvantage is not optimized for retaining local redundancies, particularly non-repetitive patterns, which have a detrimental influence on the reconstruction quality. This paper introduces the autoencoder-style network-based efficient image compression method, which contains three novel blocks, i.e., adjacent attention block, Gaussian merge block, and decoded image refinement block, to improve the overall image compression performance. The adjacent attention block allocates the additional bits required to capture spatial correlations (both vertical and horizontal) and effectively remove worthless information. The Gaussian merge block assists the rate-distortion optimization performance, while the decoded image refinement block improves the defects in low-resolution reconstructed images. A comprehensive ablation study analyzes and evaluates the qualitative and quantitative capabilities of the proposed model. Experimental results on two publicly available datasets reveal that our method outperforms the state-of-the-art methods on the KODAK dataset (by around 4dB and 5dB) and CLIC dataset (by about 4dB and 3dB) in terms of PSNR and MS-SSIM.

Image sequence decomposition via sigma-delta cellular neural network having coupled cells

In this paper, an image sequence decomposition by the sigma-delta cellular neural network (SD-CNN) with coupled cells and its composition framework is proposed. The SD-CNN, having coupled cells inspired by the second-order sigma-delta modulator, is employed for image sequence decomposition, and it enables effective image decomposition by the effects of complex dynamics. In our method, pixel luminance is described by the number of spikes within a discrete time window that corresponds to the number of SD-CNN iterations. For efficient image decomposition, a high luminance which requires a long time window, should be compensated. To solve this problem, we introduce the cumulated luminance integrator enables compensating a high luminance even with a short time window. Experimental results on various test images in the Kodak dataset and the Waterloo dataset support that all the test images can be mathematically lossless reconstructed from an image sequence decomposed via our method. It is also confirmed that the proposed method improves transmission efficiency for the Kodak dataset by approximately 80%, which is an issue with our previous method.

ICE: Implicit Coordinate Encoder for Multiple Image Neural Representation.

In recent years, implicit neural representations (INR) have shown their great potential to solve many computer graphics and computer vision problems. With this technique, signals such as 2D images or 3D shapes can be fit by training multi-layer perceptrons (MLP) on continuous functions, providing many advantages over conventional discrete representations. Despite being considered a promising approach to 2D image encoding and compression, the application of INR to image collections remains a challenge, since the number of parameters needed rapidly grow with the number of images. In this paper, we propose a fully implicit approach to INR which drastically reduces the size of MLP models in multiple image representation tasks. We introduce the concept of implicit coordinate encoder (ICE) and show it can be used to scale INR with the image number; specifically, by learning a common feature space between images. Furthermore, we show that our method is valid not only for image collections but also for large (gigapixel) images by applying a "divide-and-conquer" strategy. We propose an auto-encoder deep neural network architecture, with a single ICE (encoder) and multiple MLP (decoders), which are jointly trained following a multi-task learning strategy. We demonstrate the benefits coming from ICE when it is implemented as a one-dimensional convolutional encoder, including a better performance of the downstream MLP models with an order of magnitude fewer parameters. Our method is the first one to make use of convolutional blocks in INR networks, unlike the conventional approach of using MLP architectures only. We show the benefits of ICE in two experimental scenarios: a collection of twenty-four small ( 768×512 ) images (Kodak dataset), and a single large ( 3072×3072 ) image (dwarf planet Pluto), achieving better quality than previous fully-implicit methods, using up to 50% fewer parameters.

Causal Contextual Prediction for Learned Image Compression

Over the past several years, we have witnessed impressive progress in the field of learned image compression. Recent learned image codecs are commonly based on autoencoders, that first encode an image into low-dimensional latent representations and then decode them for reconstruction purposes. To capture spatial dependencies in the latent space, prior works exploit hyperprior and spatial context model to build an entropy model, which estimates the bit-rate for end-to-end rate-distortion optimization. However, such an entropy model is suboptimal from two aspects: (1) It fails to capture global-scope spatial correlations among the latents. (2) Cross-channel relationships of the latents remain unexplored. In this paper, we propose the concept of separate entropy coding to leverage a serial decoding process for causal contextual entropy prediction in the latent space. A <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">causal context model</i> is proposed that separates the latents across channels and makes use of channel-wise relationships to generate highly informative adjacent contexts. Furthermore, we propose a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">causal global prediction model</i> to find global reference points for accurate predictions of undecoded points. Both these two models facilitate entropy estimation without the transmission of overhead. In addition, we further adopt a new group-separated attention module to build more powerful transform networks. Experimental results demonstrate that our full image compression model outperforms standard VVC/H.266 codec on Kodak dataset in terms of both PSNR and MS-SSIM, yielding the state-of-the-art rate-distortion performance.

DePhaseNet: A deep convolutional network using phase differentiated layers and frequency based custom loss for RGBW image sensor demosaicing

Panchromatic Color Filter Arrays with white signal were introduced a while ago, such as RGBW Kodak (CFA2.0) array, assuming to have better resolution in lowlight due to panchromatic signal. However, there is no successful RGBW image sensor in the industry targeting mobile cameras until now. In this work, we introduce a novel Samsung RGBW image sensor and we study its performance in a popular remosaic scenario. We propose a DePhaseNet - a deep fully convolutional network to solve RGBW remosaicing or demosaicing problem. We propose to have 3 layers of phase differentiated inputs and custom frequency based loss function for each layer. Proposed method successfully suppress False Colors inherent to RGBW sensor due to heavily under-sampled colors. By using this method, we were able not only to increase details preservation, but also increased color reproduction by 2% over conventional method. We found that RGBW sensor is beneficial not only in low light scenarios, but also in widely spread remosaic scenarios. Experiments show improvement in image quality, yielding CPSNR of 42dB for Kodak dataset, reaching the bar of Bayer CFA demosaicing result. Proposed method advances state-of-the-art in RGBW demosaic, by 6dB in CPSNR.

Visual experience recognition using adaptive support vector machine

Video has more information than the isolated images. Processing, analyzing and understanding of contents present in videos are becoming very important. Consumer videos are generally captured by amateurs using handheld cameras of events and it contains considerable camera motion, occlusion, cluttered background, and large intraclass variations within the same type of events, making their visual cues highly variable and less discriminant. So visual event recognition is an extremely challenging task in computer vision. A visual event recognition framework for consumer videos is framed by leveraging a large amount of loosely labeled web videos. The videos are divided into training and testing sets manually. A simple method called the Aligned Space-Time Pyramid Matching method was proposed to effectively measure the distances between two video clips from different domains. Each video is divided into space-time volumes over multiple levels. A new transfer learning method is referred to as Adaptive Multiple Kernel Learning fuse the information from multiple pyramid levels, features, and copes with the considerable variation in feature distributions between videos from two domains web video domain and consumer video domain.With the help of MATLAB Simulink videos are divided and compared with web domain videos. The inputs are taken from the Kodak data set and the results are given in the form of MATLAB simulation.

Multi-scale network (MsSG-CNN) for joint image and saliency map learning-based compression

Lossy image compression is likely to produce blurred images which leads to erroneous image-level understanding. The human visual system (HVS) is focused on the area of interest present in the image. Motivated by this fact, we propose a compression-decompression algorithm focusing on the important content present in different parts of the image. Using the concept of the Guided Grad-CAM (Gradient-weighted Class Activation Mapping) technique to produce heat maps with the help of a heat map generator trained on ResNet-50, a saliency-guided encoding–decoding algorithm is developed. A wider multi-scale saliency guided convolutional neural network (MsSG-CNN) is designed, in which the notion of convolution with different size filters helps to obtain unique but different features. The feature extraction followed by multilevel fusion of features helps deep neural network (DNN) to capture contextual information and obtain high quality, good resolution, visually pleasing images with fine details. The proposed algorithm is tested on the Kodak benchmark dataset, CLIC 2019 challenging dataset, and FDDB facial images dataset. At low bit rates, the MS-SSIM of the proposed algorithm is found to be superior to JPEG, JPEG2000, BPG, WebP, and Minnen’s approaches with approximately up to 60%, 24.80%, 11.43%, 23.08% & 75% gains respectively, which is quite a significant improvement, when tested on the Kodak dataset. Similarly, at high bit rates, the improvement in MS-SSIM is approximately up to 41.67%, 37.30%, 23.90% 34.21% & 13.33% when compared with JPEG, JPEG2000, BPG, WebP, and Minnen’s approaches respectively. The improvement in PSNR at low and high bit rates is approximately up to 11.32%, 5%, 5.26% and 5.6%, 10.29%, 8.7% as compared to JPEG, Balle’s, and Lee’s algorithms respectively. The PSNR-HVS has been improved by approximately up to 27.27%, 19.15%, and 28.33%, 28% as compared to JPEG and Toderici’s algorithms respectively at low and high bit-rates. A similar type of improvement is obtained with FDDB and CLIC 2019 datasets also, which is discussed in the paper.

An extended context-based entropy hybrid modeling for image compression

Recently deep learning has been introduced to the field of image compression. In this paper, we present a hybrid coding framework that combines entropy coding, deep learning, and traditional coding framework. In the base layer of the encoding, we use convolutional neural networks to learn the latent representation and importance map of the original image respectively. The importance map is then used to guide the bit allocation of the latent representation. A context model is also developed to help the entropy coding after the masked quantization. Another network is used to get a coarse reconstruction of the image in the base layer. The residual between the input and the coarse reconstruction is then obtained and encoded by the traditional BPG codec as the enhancement layer of the bit stream. We only need to train a basic model and the proposed scheme can realize image compression at different bit rates, thanks to the use of the traditional codec. Experimental results using the Kodak, Urban100 and BSD100 datasets show that the proposed scheme outperforms many deep learning-based methods and traditional codecs including BPG in MS-SSIM metric across a wide range of bit rates. It also exceeds some latest hybrid schemes in RGB444 domain on Kodak dataset in both PSNR and MS-SSIM metrics.

Under Display Camera Quad Bayer Raw Image Restoration using Deep Learning

Can a mobile camera see better through display? Under Display Camera (UDC) is the most awaited feature in mobile market in 2020 enabling more preferable user experience, however, there are technological obstacles to obtain acceptable UDC image quality. Mobile OLED panels are struggling to reach beyond 20% of light transmittance, leading to challenging capture conditions. To improve light sensitivity, some solutions use binned output losing spatial resolution. Optical diffraction of light in a panel induces contrast degradation and various visual artifacts including image ghosts, yellowish tint etc. Standard approach to address image quality issues is to improve blocks in the imaging pipeline including Image Signal Processor (ISP) and deblur block. In this work, we propose a novel approach to improve UDC image quality - we replace all blocks in UDC pipeline with all-in-one network – UDC d^Net. Proposed solution can deblur and reconstruct full resolution image directly from non-Bayer raw image, e.g. Quad Bayer, without requiring remosaic algorithm that rearranges non-Bayer to Bayer. Proposed network has a very large receptive field and can easily deal with large-scale visual artifacts including color moiré and ghosts. Experiments show significant improvement in image quality vs conventional pipeline – over 4dB in PSNR on popular benchmark - Kodak dataset.

End-to-End Optimized Versatile Image Compression With Wavelet-Like Transform.

Built on deep networks, end-to-end optimized image compression has made impressive progress in the past few years. Previous studies usually adopt a compressive auto-encoder, where the encoder part first converts image into latent features, and then quantizes the features before encoding them into bits. Both the conversion and the quantization incur information loss, resulting in a difficulty to optimally achieve arbitrary compression ratio. We propose iWave++ as a new end-to-end optimized image compression scheme, in which iWave, a trained wavelet-like transform, converts images into coefficients without any information loss. Then the coefficients are optionally quantized and encoded into bits. Different from the previous schemes, iWave++ is versatile: a single model supports both lossless and lossy compression, and also achieves arbitrary compression ratio by simply adjusting the quantization scale. iWave++ also features a carefully designed entropy coding engine to encode the coefficients progressively, and a de-quantization module for lossy compression. Experimental results show that lossy iWave++ achieves state-of-the-art compression efficiency compared with deep network-based methods; on the Kodak dataset, lossy iWave++ leads to 17.34 percent bits saving over BPG; lossless iWave++ achieves comparable or better performance than FLIF. Our code and models are available at https://github.com/mahaichuan/Versatile-Image-Compression.