Objective Image Quality Assessment Metrics Research Articles

NNCD-IQA: A new neural networks based compressed database for image quality assessment

Objective and subjective quality assessment is still a challenging problem in various image processing tasks. For instance, in the context of image compression, most of the conducted studies have focused on image datasets encoded using standard algorithms such as JPEG and JPEG2000. In this paper, we propose to further investigate the quality assessment issue in the presence of neural networks-based compressed images. More precisely, a new database of compressed images has been firstly built using JPEG2000 standard as well as four recent neural networks based coding schemes. Then, subjective experiments are performed to obtain the mean opinion scores of the generated distorted images. Finally, an extensive evaluation and analysis of objective image quality assessment metrics is achieved. For instance, in addition to conventional and machine learning metrics, we have considered different deep learning based models, which have been trained on our database. The new subjective database with its associated mean opinion scores as well as the learned models are publicly available at https://github.com/zakopz/NNCD-IQA-Database . The obtained results show the interest of deep learning based metrics in the context of neural networks-based compressed images.

Study of subjective and objective quality assessment of infrared compressed images

Abstract Given the lack of accessible infrared compressed images’ benchmarks annotated by human subjects, this work presents a new database with the aim of studying both subjective and objective image quality assessment (IQA) on compressed long wavelength infrared (LWIR) images. The database contains 20 reference (pristine) images and 200 distorted (degraded) images obtained by application of the most known compression algorithms used in multimedia and communication fields, namely: JPEG and JPEG-2000. Each compressed image is evaluated by 31 subjects having different levels of experience in LWIR images. Mean opinion scores (MOS) and natural scene statistics (NSS) of pristine and compressed images are elaborated to study the performance of the database. Five analyses are conducted on collected images and subjective scores, namely: analysis by compression type, analysis by file size, analysis by reference image, analysis by quality level and analysis by subject. Moreover, a wide set of objective IQA metrics is applied on the images and the obtained scores are compared with the collected subjective scores. Results show that objective IQA measures correlate with human subjective results with a degree of agreement up to 95 %, so this benchmark is promising to improve existing and develop new IQA measures for compressed LWIR images. Thanks to a real-world surveillance original images based on which we analyze how image compression and quality level affect the quality of compressed images, this database is primarily suitable for (military and civilian) surveillance applications. The database is accessible via the link: https://github.com/azedomar/compressed-LWIR-images-IQA-database. As a follow-up to this work, an extension of the database is underway to study other types of distortion in addition to compression.

Multimodal medical image fusion by cloud model theory

Image fusion can provide more extensive information since it combines two or more different images. Cloud model is a recently proposed theory in artificial intelligence and has the advantage of taking the randomness and fuzziness into account. In this paper, we introduce a novel multimodal medical image fusion method by cloud model theory. The proposed method fits the histograms of input images using the high-order spline function firstly and then divides intervals in line with the valley point of the fitted curve. On this basis, cloud models are generated adaptively through the reverse cloud generator. Finally, cloud reasoning rules are designed to achieve the fused image. Experimental results demonstrate that the fused images by proposed method show more image details and lesion regions than existing methods. The objective image quality assessment metrics on the fused images also show the superiority of the proposed method.

A Sampling Agnostic Software Framework for Converting Between Texture Map Representations of Virtual Environments.

We have developed a utility to both stitch cube maps into other types of texture maps (equirectangular,dual paraboloid, and octahedral), and stitch those other types back into cube maps. The utility allows for flexibility in the image size of the conversion - the user can specify the desired image width, and the height is computed (cube, paraboloid, and octahedral mappings are square, and spherical maps are generated to have 16:9 aspect ratio). Moreover, the utility is sampling-agnostic, so the user can select whether to use uniform or jittered sampling over the pixels, as well as the number of samples to use per pixel. The rest of this paper discusses the mathematical framework for projecting from cube maps to equirectangular, dual paraboloid, and octahedral environment maps, as well as the mathematical framework for the inverse projections. We also describe two sampling techniques: uniform sampling and correlated multi-jittered sampling. We perform an evaluation of the sampling techniques and a comparative analysis of the different projections using objective image quality assessment metrics.

Knowledge-based Taxonomic Scheme for Full-Reference Objective Image Quality Measurement Models

In this article, we propose a knowledge-based taxonomic scheme of the objective image quality assessment metrics including the key concepts involved for each approach. Our classification is constructed according to six criteria based on the information available at each stage of the design process. The novelty of the present classification scheme is that the six layers are linked via a single concept where each layer represents a single type of knowledge about: 1) the reference image, 2) the degradation type, 3) the visual perception field, 4) the human visual physiology and psychophysical mechanisms, 5) the processes of the visual information analysis, and finally 6) knowledge about perceptual image representation and coding. The first layer helps delineate boundaries between full-reference (FR) image quality assessment metrics, that are further classified through layers 2–6, and other families (reduced-reference [RR] and no-reference [NR]). In addition, gradual degrees are considered for knowledge about specific areas related to visual quality evaluation processes. The proposed taxonomic framework is intended to be stepwise, to help sorting out the fundamental ideas behind the development of objective image quality metrics often working on the luminance channel or marginally on the RGB channels. The aim is to congregate the already published classification schemes and to methodologically expand new aspects according to which an efficient and straightforward classification of the image quality assessment algorithms becomes possible. This is significant because of the increasing number of developed metrics. Furthermore, a systematic summarization is necessary in order to facilitate the research and application of image quality techniques. © 2016 Society for Imaging Science and Technology.

A weighted full-reference image quality assessment based on visual saliency

In full reference image quality assessment (IQA), the images without distortion are usually employed as reference, while the structures in both reference images and distorted images are ignored and all pixels are equally treated. In addition, the role of human visual system (HVS) is not taken account into subjective IQA metric. In this paper, a weighted full-reference image quality metric is proposed, where a weight imposed on each pixel indicates its importance in IQA. Furthermore, the weights can be estimated via visual saliency computation, which can approximate the subjective IQA via exploiting the HVS. In the experiments, the proposed metric is compared with several objective IQA metrics on LIVE release 2 and TID 2008 database. The results demonstrate that SROCC and PLCC of the proposed metric are 0.9647 and 0.9721, respectively,which are higher than other methods and it only takes 427.5s, which is lower than that of most other methods.

Knowledge-based Taxonomic Scheme for Full-Reference Objective Image Quality Measurement Models

In this paper, we propose a knowledge-based taxonomic scheme of the objective image quality assessment metrics including the key concepts involved for each approach. Our classification is constructed according to six criteria based on the information available at each stage of the design process. The novelty of the present classification scheme is that the six layers are linked via a single concept where each layer represents a single type of knowledge about: 1) the reference image, 2) the degradation type, 3) the visual perception field, 4) the human visual physiology and psychophysical mechanisms, 5) the processes of the visual information analysis, and finally 6) knowledge about perceptual image representation and coding. The first layer helps delineate boundaries between full-reference image quality assessment metrics, that are further classified trough layers 2 to 6, and other families (reduced-reference and no-reference). In addition, gradual degrees are considered for knowledge about specific areas related to visual quality evaluation processes. The proposed taxonomic framework is intended to be stepwise, to help sorting out the fundamental ideas behind the development of objective image quality metrics often working on the luminance channel or marginally on the RGB channels. The aim is to congregate the already published classification schemes and to methodologically expand new aspects according to which an efficient and straightforward classification of the image quality assessment algorithms becomes possible. This is significant because of the increasing number of developed metrics. Furthermore, a systematic summarisation is necessary in order to facilitate the search and application of image quality techniques.

Objective Image Quality Assessment Based on Saliency Map

The development of objective image quality assessment metrics aligned with human perception is of fundamental importance to numerous image processing applications. In this paper, an objective image quality assessment approach based on saliency map is proposed. By local shift estimation method, the retargeted image is resized to the same size as the reference image. A gradient magnitude similarity map is computed by comparing the retargeted and reference images. The more similarly, the brighter of pixels in the gradient magnitude similarity map. At the same time, a saliency map of reference image is achieved by visual attention. Finally, an overall image quality score is computed from the gradient magnitude similarity map via saliency pooling strategy. The most important step in our approach is to generate a gradient magnitude similarity map that indicates at each spatial location in the source image how the structural information is preserved in the retargeted image. There are two key contributions in this paper, one is that we add the texture feature in computing saliency map because image gradient is very sensitive to texture information, and the other is that we propose a new objective image quality metrics by introducing saliency map into image quality evaluation. Experimental results indicate that the evaluation indexes of our approach are better than existing methods in the literature.

Framework for fair objective performance evaluation of single‐image super‐resolution algorithms

Single-image super-resolution (SISR) is a technology to reconstruct a high-resolution image from a single low-resolution input image. The performance of SISR algorithms is usually evaluated by applying full-reference objective image quality assessment metrics. First, it is argued that the result of objective quality evaluation may become inconsistent with subjective quality assessment, depending on how the input low-resolution image is generated and how up-scaling during SISR is conducted. Since such inconsistency is due to subpixel-level misalignment between the original and output images, a framework is then proposed that compensates any spatial displacement between the two images and enables fair SISR performance evaluation using objective quality metrics.

DCT-based objective quality assessment metric of 2D/3D image

With the increasing growth of multimedia applications over the networking in recent years, users have put forward much higher requirements for multimedia quality of experience (QoE) than before. One of the representative requirements is the image quality. Therefore, the image quality assessment ranging from two-dimension (2D) image to three-dimension (3D) image has been getting much attention. In this paper, an efficient objective image quality assessment metric in block-based discrete cosine transform (DCT) coding is proposed. The metric incorporates properties of human visual system (HVS) to improve its validity and reliability in evaluating the quality of stereoscopic image. This is fulfilled by calculating the local pixel-based distortions in frequency domain, combining the simplified models of local visibility properties embodied in frequency domain, which consist of region of interest (ROI) mechanism (visual sensitivity), contrast sensitivity function (CSF) and contrast masking effect. The performance of the proposed metric is compared with other currently state-of-the-art objective image quality assessment metrics. The experimental results have demonstrated that the proposed metric is highly consistent with the subjective test scores. Moreover, the performance of the metric is also confirmed with the popular IRCCyN/IVC database. Therefore, the proposed metric is promising in term of the practical efficiency and reliability for real-life multimedia applications.

IrisCode Decompression Based on the Dependence between Its Bit Pairs

IrisCode is an iris recognition algorithm developed in 1993 and continuously improved by Daugman. Understanding IrisCode's properties is extremely important because over 60 million people have been mathematically enrolled by the algorithm. In this paper, IrisCode is proved to be a compression algorithm, which is to say its templates are compressed iris images. In our experiments, the compression ratio of these images is 1:655. An algorithm is designed to perform this decompression by exploiting a graph composed of the bit pairs in IrisCode, prior knowledge from iris image databases, and the theoretical results. To remove artifacts, two postprocessing techniques that carry out optimization in the Fourier domain are developed. Decompressed iris images obtained from two public iris image databases are evaluated by visual comparison, two objective image quality assessment metrics, and eight iris recognition methods. The experimental results show that the decompressed iris images retain iris texture that their quality is roughly equivalent to a JPEG quality factor of 10 and that the iris recognition methods can match the original images with the decompressed images. This paper also discusses the impacts of these theoretical and experimental findings on privacy and security.