Full-reference Image Quality Assessment Algorithms Research Articles

Saliency-Guided Local Full-Reference Image Quality Assessment

Research and development of image quality assessment (IQA) algorithms have been in the focus of the computer vision and image processing community for decades. The intent of IQA methods is to estimate the perceptual quality of digital images correlating as high as possible with human judgements. Full-reference image quality assessment algorithms, which have full access to the distortion-free images, usually contain two phases: local image quality estimation and pooling. Previous works have utilized visual saliency in the final pooling stage. In addition to this, visual saliency was utilized as weights in the weighted averaging of local image quality scores, emphasizing image regions that are salient to human observers. In contrast to this common practice, visual saliency is applied in the computation of local image quality in this study, based on the observation that local image quality is determined both by local image degradation and visual saliency simultaneously. Experimental results on KADID-10k, TID2013, TID2008, and CSIQ have shown that the proposed method was able to improve the state-of-the-art’s performance at low computational costs.

Composition-preserving deep approach to full-reference image quality assessment

Image quality assessment is an important element of a broad spectrum of applications ranging from automatic video streaming to display technology. In this study, we present a novel full-reference image quality assessment algorithm relying on a Siamese layout of pretrained convolutional neural networks (CNNs), feature pooling, and a neural network. Unlike previous methods, our algorithm handles input images without resizing, cropping, or any modifications. As a consequence, it effectively learns the fine-grained, quality-aware features of images. The proposed model derives its core performance from pretrained CNNs, being trained at a higher resolution than that in previous works. The presented architecture was trained on the recently published KADID-10k, which is the largest image quality database and contains 10,125 digital images. Experimental results on KADID-10k demonstrate that the proposed method outperforms other state-of-the-art algorithms. These results are also confirmed with cross-database tests using other publicly available datasets.

SPSIM: A Superpixel-Based Similarity Index for Full-Reference Image Quality Assessment.

Full-reference image quality assessment algorithms usually perform comparisons of features extracted from square patches. These patches do not have any visual meanings. On the contrary, a superpixel is a set of image pixels that share similar visual characteristics and is thus perceptually meaningful. Features from superpixels may improve the performance of image quality assessment. Inspired by this, we propose a new superpixel-based similarity index by extracting perceptually meaningful features and revising similarity measures. The proposed method evaluates image quality on the basis of three measurements, namely, superpixel luminance similarity, superpixel chrominance similarity, and pixel gradient similarity. The first two measurements assess the overall visual impression on local images. The third measurement quantifies structural variations. The impact of superpixel-based regional gradient consistency on image quality is also analyzed. Distorted images showing high regional gradient consistency with the corresponding reference images are visually appreciated. Therefore, the three measurements are further revised by incorporating the regional gradient consistency into their computations. A weighting function that indicates superpixel-based texture complexity is utilized in the pooling stage to obtain the final quality score. Experiments on several benchmark databases demonstrate that the proposed method is competitive with the state-of-the-art metrics.

Blind Quality Estimation by Disentangling Perceptual and Noisy Features in High Dynamic Range Images.

High dynamic range (HDR) image visual quality assessment in the absence of a reference image is challenging. This research topic has not been adequately studied largely due to the high cost of HDR display devices. Nevertheless, HDR imaging technology has attracted increasing attention, because it provides more realistic content, consistent to what the human visual system perceives. We propose a new no-reference image quality assessment (NR-IQA) model for HDR data based on convolutional neural networks. The proposed model is able to detect visual artifacts, taking into consideration perceptual masking effects, in a distorted HDR image without any reference. The error and perceptual masking values are measured separately, yet sequentially, and then processed by a mixing function to predict the perceived quality of the distorted image. Instead of using simple stimuli and psychovisual experiments, perceptual masking effects are computed from a set of annotated HDR images during our training process. Experimental results demonstrate that our proposed NR-IQA model can predict HDR image quality as accurately as state-of-the-art full-reference IQA methods.

Evaluation of a Transparent Display's Pixel Structure Regarding Subjective Quality of Diffracted See-Through Images

Transparent displays utilizing transparent windows suffer from blurred see-through images caused by diffraction; however, current studies still rely on experiments with actual display panels and human observers to investigate see-through image quality. Moreover, the influence of pixel structure on subjective see-through image quality has not been clearly demonstrated. To improve the inefficient investigation methodology and quantitatively evaluate pixel structure, we first propose a simulation method for diffracted see-through images. Next, by testing six mainstream full-reference image quality assessment algorithms, multiscale structure similarity (MS-SSIM) is revealed to be the most suitable predictor of subjective image quality for our study. Based on public image databases, the influences of aperture ratio, resolution, and the geometry of the transparent window are evaluated by combining the proposed simulation method and MS-SSIM. As a result, an aperture ratio increase of 0.1 leads to a considerable increase of subjective image quality, as more than eight percentage points. Then, a resolution increase of 100PPI only leads to an approximately three percentage point decrease of image quality. Finally, little influence of the geometry of the transparent window is demonstrated. Additionally, physical reasons why these aspects of pixel structure perform in such a manner are also given.

Similar Reference Image Quality Assessment: A New Database and A Trial with Local Feature Matching

Conventionally, the reference image for image quality assessment (IQA) is completely available (full-reference IQA) or unavailable (no-reference IQA). Even for reduced-reference IQA, the features that are used to predict image quality are still extracted from the pristine reference image. However, the pristine reference image is always unavailable in many real scenarios. In contrast, it is convenient to obtain a number of similar reference images via retrieval from the Internet. These similar reference images may share similar contents and scenes with the image to be assessed. In this paper, we attempt to discuss the image quality assessment problem from the view of similar images, i.e. similar reference IQA. Although the similar reference images share similar contents with the degraded image, the difference between them still cannot be ignored. Therefore, we propose an IQA framework based on local feature matching, which can help to identify the similar regions and structures. Then the IQA features are computed only from these similar regions to predict the final image quality score. Besides, since there is no IQA databases for the similar reference IQA problem, we establish a novel IQA database that consists of 272 images from four scenes. The experiments demonstrate that the performance of our scheme goes beyond state-of-the-art no-reference IQA methods and some full-reference IQA algorithms.

Full-Reference Stereo Image Quality Assessment Using Natural Stereo Scene Statistics

Empirical studies of the joint statistics of luminance and disparity images (or wavelet coefficients) of natural stereoscopic scenes have resulted in two important findings: a) the marginal statistics are modelled well by the generalized Gaussian distribution (GGD) and b) there exists significant correlation between them. Inspired by these findings, we propose a full-reference image quality assessment algorithm dubbed STeReoscopic Image Quality Evaluator (STRIQE). We show that the parameters of the GGD fits of luminance wavelet coefficients along with correlation values form excellent features. Importantly, we demonstrate that the use of disparity information (via correlation) results in a consistent improvement in the performance of the algorithm. The performance of our algorithm is evaluated over popular datasets and shown to be competitive with the state-of-the-art full-reference algorithms. The efficacy of the algorithm is further highlighted by its near-linear relation with subjective scores, low root mean squared error (RMSE), and consistently good performance over both symmetric and asymmetric distortions.

Microarchitectural analysis of image quality assessment algorithms

Algorithms for image quality assessment (IQA) aim to predict the qualities of images in a manner that agrees with subjective quality ratings. Over the last several decades, the major impetus in IQA research has focused on improving predictive performance; very few studies have focused on analyzing and improving the runtime performance of IQA algorithms. This paper is the first to examine IQA algorithms from the perspective of their interaction with the underlying hardware and microarchitectural resources, and to perform a systematic performance analysis using state-of-the-art tools and techniques from other computing disciplines. We implemented four popular full-reference IQA algorithms (most apparent distortion, multiscale structural similarity, visual information fidelity, and visual signal-to-noise ratio) and two no-reference algorithms (blind image integrity notator using DCT statistics and blind/referenceless image spatial quality evaluator) in C++ based on the code provided by their respective authors. We then conducted a hotspot analysis to identify sections of code that were performance bottlenecks and performed microarchitectural analysis to identify the underlying causes for these bottlenecks. Despite the fact that all six algorithms share common algorithmic operations (e.g., filterbanks and statistical computations), our results revealed that different IQA algorithms overwhelm different microarchitectural resources and give rise to different types of bottlenecks. Based on these results, we propose microarchitectural-conscious coding techniques and custom hardware recommendations for performance improvement.

Machine Learning to Design Full-reference Image Quality Assessment Algorithm

A crucial step in image compression is the evaluation of its performance, and more precisely, available ways to measure the quality of compressed images. In this paper, based on a learned classification process in order to respect human observers， a method namely Machine Learning-based Image Quality Measure (MLIQM) is proposed，which classifies the quality using multi-Support Vector Machine (SVM) classification according to the quality scale recommended by the ITU. Then, the classification process is performed to provide the final quality class of the considered image. Finally, once a quality class is associated to the considered image, a specific SVM regression is performed to score its quality. Obtained results are compared with the one obtained applying classical Full-Reference Image Quality Assessment (FR-IQA) algorithms to judge the efficiency of the proposed method. DOI: http://dx.doi.org/10.11591/telkomnika.v11i6.2716

Machine learning to design full-reference image quality assessment algorithm

A crucial step in image compression is the evaluation of its performance, and more precisely, available ways to measure the quality of compressed images. In this paper, a machine learning expert, providing a quality score is proposed. This quality measure is based on a learned classification process in order to respect human observers. The proposed method namely Machine Learning-based Image Quality Measure (MLIQM) first classifies the quality using multi-Support Vector Machine (SVM) classification according to the quality scale recommended by the ITU. This quality scale contains 5 ranks ordered from 1 (the worst quality) to 5 (the best quality). To evaluate the quality of images, a feature vector containing visual attributes describing images content is constructed. Then, a classification process is performed to provide the final quality class of the considered image. Finally, once a quality class is associated to the considered image, a specific SVM regression is performed to score its quality. Obtained results are compared to the one obtained applying classical Full-Reference Image Quality Assessment (FR-IQA) algorithms to judge the efficiency of the proposed method.

A Statistical Evaluation of Recent Full Reference Image Quality Assessment Algorithms

Measurement of visual quality is of fundamental importance for numerous image and video processing applications, where the goal of quality assessment (QA) algorithms is to automatically assess the quality of images or videos in agreement with human quality judgments. Over the years, many researchers have taken different approaches to the problem and have contributed significant research in this area and claim to have made progress in their respective domains. It is important to evaluate the performance of these algorithms in a comparative setting and analyze the strengths and weaknesses of these methods. In this paper, we present results of an extensive subjective quality assessment study in which a total of 779 distorted images were evaluated by about two dozen human subjects. The "ground truth" image quality data obtained from about 25,000 individual human quality judgments is used to evaluate the performance of several prominent full-reference image quality assessment algorithms. To the best of our knowledge, apart from video quality studies conducted by the Video Quality Experts Group, the study presented in this paper is the largest subjective image quality study in the literature in terms of number of images, distortion types, and number of human judgments per image. Moreover, we have made the data from the study freely available to the research community. This would allow other researchers to easily report comparative results in the future.