Image Quality Assessment Methods Research Articles

No-reference blurred image quality assessment method based on structure of structure features

The deep structure in the image contains certain information of the image, which is helpful to perceive the quality of the image. Inspired by deep level image features extracted via deep learning methods, we propose a no-reference blurred image quality evaluation model based on the structure of structure features. In spatial domain, the novel weighted local binary patterns are proposed which leverage maximum local variation maps to extract structural features from multi-resolution images. In spectral domain, gradient information of multi-scale Log-Gabor filtered images is extracted as the structure of structure features, and combined with entropy features. Then, the features extracted from both domains are fused to form a quality perception feature vector and mapped into the quality score via support vector regression (SVR). Experiments are conducted to evaluate the performance of the proposed method on various IQA databases, including the LIVE, CSIQ, TID2008, TID2013, CID2013, CLIVE, and BID. The experimental results show that compared with some state-of-the-art methods, our proposed method achieves better evaluation results and is more in line with the human visual system. The source code will be released at https://github.com/JamesC0321/s2s_features/.

Unifying Dual-Attention and Siamese Transformer Network for Full-Reference Image Quality Assessment

Image Quality Assessment (IQA) is a critical task of computer vision. Most Full-Reference (FR) IQA methods have limitation in the accurate prediction of perceptual qualities of the traditional distorted images and the Generative Adversarial Networks (GANs) based distorted images. To address this issue, we propose a novel method by Unifying Dual-Attention and Siamese Transformer Network (UniDASTN) for FR-IQA. An important contribution is the spatial attention module composed of a Siamese Transformer Network and a feature fusion block. It can focus on significant regions and effectively maps the perceptual differences between the reference and distorted images to a latent distance for distortion evaluation. Another contribution is the dual-attention strategy that exploits channel attention and spatial attention to aggregate features for enhancing distortion sensitivity. In addition, a novel loss function is designed by jointly exploiting Mean Square Error (MSE), bidirectional Kullback–Leibler divergence, and rank order of quality scores. The designed loss function can offer stable training and thus enables the proposed UniDASTN to effectively learn visual perceptual image quality. Extensive experiments on standard IQA databases are conducted to validate the effectiveness of the proposed UniDASTN. The IQA results demonstrate that the proposed UniDASTN outperforms some state-of-the-art FR-IQA methods on the LIVE, CSIQ, TID2013, and PIPAL databases.

Super-resolution image visual quality assessment based on structure–texture features

Assessing the visual quality of super-resolution images (SRIs) is crucial for advancing algorithm development, but it remains an unsolved problem. In this paper, we present a novel reduced-reference image quality assessment (RR-IQA) method specifically suited for evaluating SRIs. Our approach leverages information from the input low-resolution (LR) image as a reference signal to extract features that are most relevant to modeling the visual quality of SRIs. We analyze the artifact characteristics of SRIs and demonstrate that features describing edge orientations, high frequency components, and textures are the most important for this task. To extract these features, we first perform structure–texture decompositions (STD) on both the SRI and its LR input, then obtain the edge orientation feature through a traditional hand-crafted approach, and deep neural networks to extract features related to high frequency components and textures. We employ a shallow multilayer perceptron (MLP) to predict an image quality score based on these quality-relevant features. To improve feature representation ability and prevent overfitting, we pretrain the feature extraction module using a large number of unlabeled samples, which accounts for over 99.8% of the total model parameters. We use precious samples with mean opinion score (MOS) labels to train a high-quality shallow MLP predictor. Our experimental results show that the proposed method outperforms classical and state-of-the-art models.

Unsupervised blind image quality assessment via joint spatial and transform features

A novel unsupervised blind image quality assessment (BIQA) method, which requires no mean opinion scores for model training is presented in this paper. The method employs joint spatial and transform features as quality degradation metrics, specifically, phase congruency, gradient magnitude (GM), and GM and Laplacian of Gaussian response and local normalized coefficient are extracted as spatial features, and Karhunen–Loéve transform coefficient and discrete cosine transform coefficient are modeled as transform features. Both spatial and transform features are well analyzed to remove the redundancy, and then fitted to the multivariate Gaussian model for no-reference image quality assessment. Extensive experiments conducted on seven IQA databases demonstrate the superiority of the proposed method over the state-of-the-art both supervised and unsupervised BIQA methods.

Underwater Ranker: Learn Which Is Better and How to Be Better

In this paper, we present a ranking-based underwater image quality assessment (UIQA) method, abbreviated as URanker. The URanker is built on the efficient conv-attentional image Transformer. In terms of underwater images, we specially devise (1) the histogram prior that embeds the color distribution of an underwater image as histogram token to attend global degradation and (2) the dynamic cross-scale correspondence to model local degradation. The final prediction depends on the class tokens from different scales, which comprehensively considers multi-scale dependencies. With the margin ranking loss, our URanker can accurately rank the order of underwater images of the same scene enhanced by different underwater image enhancement (UIE) algorithms according to their visual quality. To achieve that, we also contribute a dataset, URankerSet, containing sufficient results enhanced by different UIE algorithms and the corresponding perceptual rankings, to train our URanker. Apart from the good performance of URanker, we found that a simple U-shape UIE network can obtain promising performance when it is coupled with our pre-trained URanker as additional supervision. In addition, we also propose a normalization tail that can significantly improve the performance of UIE networks. Extensive experiments demonstrate the state-of-the-art performance of our method. The key designs of our method are discussed. Our code and dataset are available at https://li-chongyi.github.io/URanker_files/.

Underwater image quality assessment.

To obtain high-visual-quality underwater images by image post-processing, many underwater image restoration and enhancement methods have been proposed. Underwater image quality assessment (UIQA) methods have been developed to compare these restoration and enhancement methods. This paper comprehensively summarizes the subjective and objective UIQA methods, metrics, and datasets. Experiments are conducted on two underwater image datasets to analyze the performance of several typical UIQA metrics. Suggestions for further research directions are put forward as well.

Deep learning model for automatic image quality assessment in PET

BackgroundA variety of external factors might seriously degrade PET image quality and lead to inconsistent results. The aim of this study is to explore a potential PET image quality assessment (QA) method with deep learning (DL).MethodsA total of 89 PET images were acquired from Peking Union Medical College Hospital (PUMCH) in China in this study. Ground-truth quality for images was assessed by two senior radiologists and classified into five grades (grade 1, grade 2, grade 3, grade 4, and grade 5). Grade 5 is the best image quality. After preprocessing, the Dense Convolutional Network (DenseNet) was trained to automatically recognize optimal- and poor-quality PET images. Accuracy (ACC), sensitivity, specificity, receiver operating characteristic curve (ROC), and area under the ROC Curve (AUC) were used to evaluate the diagnostic properties of all models. All indicators of models were assessed using fivefold cross-validation. An image quality QA tool was developed based on our deep learning model. A PET QA report can be automatically obtained after inputting PET images.ResultsFour tasks were generated. Task2 showed worst performance in AUC,ACC, specificity and sensitivity among 4 tasks, and task1 showed unstable performance between training and testing and task3 showed low specificity in both training and testing. Task 4 showed the best diagnostic properties and discriminative performance between poor image quality (grade 1, grade 2) and good quality (grade 3, grade 4, grade 5) images. The automated quality assessment of task 4 showed ACC = 0.77, specificity = 0.71, and sensitivity = 0.83, in the train set; ACC = 0.85, specificity = 0.79, and sensitivity = 0.91, in the test set, respectively. The ROC measuring performance of task 4 had an AUC of 0.86 in the train set and 0.91 in the test set. The image QA tool could output basic information of images, scan and reconstruction parameters, typical instances of PET images, and deep learning score.ConclusionsThis study highlights the feasibility of the assessment of image quality in PET images using a deep learning model, which may assist with accelerating clinical research by reliably assessing image quality.

No-Reference Image Quality Assessment Based on a Multitask Image Restoration Network

When image quality is evaluated, the human visual system (HVS) infers the details in the image through its internal generative mechanism. In this process, the HVS integrates both local and global information about the image, utilizes contextual information to restore the original image information, and compares it with the distorted image information for image quality evaluation. Inspired by this mechanism, a no-reference image quality assessment method is proposed based on a multitask image restoration network. The multitask image restoration network generates a pseudo-reference image as the main task and produces a structural similarity index measure map as an auxiliary task. By mutually promoting the two tasks, a higher-quality pseudo-reference image is generated. In addition, when predicting the image quality score, both the quality restoration features and the difference features between the distorted and reference images are used, thereby fully utilizing the information from the pseudo-reference image. In order to facilitate the model’s ability to extract both global and local features, we introduce a multi-scale feature fusion module. Experimental results demonstrate that the proposed method achieves excellent performance on both synthetically and authentically distorted databases.

An End-to-End Blind Image Quality Assessment Method Using a Recurrent Network and Self-Attention

In this paper, we propose a blind image quality assessment (BIQA) method using self-attention and a recurrent neural network (RNN); this approach can effectively capture both local and global information from an input image. The implementation of our constructed deep no-reference (NR) assessment framework does not rely on any convolutional operations. First, the capture step for obtaining locally significant information is performed by a self-attention operation inside a divided window. Second, we design a serialized feature input memory subnetwork to fuse the global features of the image. Finally, all the integrated features are uniformly mapped to the target score. The experimental results obtained on publicly available benchmark IQA databases show that our approach outperforms other state-of-the-art algorithms.

Deep Neural Network for Blind Visual Quality Assessment of 4K Content

The 4K content can deliver a more immersive visual experience to consumers due to the huge improvement in spatial resolution. However, the high spatial resolution brings a great challenge for video transmission and storage. Therefore, it is necessary to compress or downscale the 4K content before transmitting it to end-users. Existing blind image quality assessment (BIQA) methods are not suitable for 4K contents due to the high spatial resolution and specific distortions caused by upscaling methods. In this paper, we propose a deep learning-based BIQA model for 4K content, aiming to recognize true and pseudo 4K content and meanwhile evaluate their visual quality. Considering the characteristic that high spatial resolution can represent more abundant high-frequency information, we first propose a Grey-level Co-occurrence Matrix (GLCM) based texture complexity measure to select three representative image patches from a 4K image, which can reduce the computational complexity and is proven to be very effective for the overall quality prediction. Then, we extract various visual features from the intermediate layers of the convolutional neural network (CNN) and integrate them into the quality-aware feature representation. Finally, two multilayer perception (MLP) networks are utilized to map the quality-aware features into the class probability and the quality score of each patch respectively. The overall quality index is obtained through averaging the results of all patches. The proposed model is trained via the multi-task learning manner and the uncertainty principle is introduced to balance the losses of the classification and regression tasks. The experimental results show that the proposed model outperforms all compared BIQA metrics on four 4K content quality assessment databases.

Multi-layer and Multi-scale feature aggregation for DIBR-Synthesized image quality assessment

Depth-Image-Based Rendering (DIBR) is one of the main fundamental techniques for generating new viewpoints in 3D video applications such as multi-viewpoint video (MVV), free viewpoint video (FVV) and virtual reality (VR). Due to the imperfections of color images, depth maps or texture restoration techniques, several types of distortions occur in synthesized views. However, most of related works evaluated the quality of DIBR-synthesized views by only detecting a specific type of distortion, such as stretching, black holes, blurring, etc., which were unable to accurately evaluate the quality of DIBR-synthesized views. In this paper, a new no-reference image quality assessment method is proposed to evaluate the quality of DIBR-synthesized images by combining multi-layer and multi-scale features of images. To be specific, the distortions introduced by different stages of virtual viewpoint synthesis are first analyzed, and then multi-layer and multi-scale features are extracted to estimate the degree of texture and structure distortions. As a result, individual quality scores associated with two types of distortions (e.g., structural distortion and texture distortion) are aggregated to an overall image quality. Experimental results on two publicly available DIBR datasets show that the method has better performance than the state-of-the-art models.Index Terms: image quality assessment, DIBR-synthesized image, distortion correction, BIQA.

No-Reference Image Quality Assessment of Magnetic Resonance images with multi-level and multi-model representations based on fusion of deep architectures

Accurate quality assessment of Magnetic Resonance (MR) images is essential for effective medical diagnostics, as it impacts the time spent on image acquisition and image interpretation by radiologists. Therefore, this study aims to provide a novel deep learning-based No-Reference (NR) MR Image Quality Assessment (IQA) method for the quality prediction of MR images. Therefore, in this work, an internal fusion of two complementary deep learning architectures is introduced that offers MR-specific quality-aware features. Apart from obtaining multi-level image representations of joint networks, to further enhance the quality prediction performance, features from re-trained single deep learning architectures are also used. Finally, as one of the main findings of this study, the quality assessment is performed by a high-level quality model trained on scores of quality models obtained for layers of the networks. The superiority of the method against the state-of-the-art techniques is verified by using two publicly available MRIQA benchmarks containing MR images and subjective scores provided by a large number of radiologists. As reported, the method offers superior IQA of MR images as the obtained scores are highly correlated with subjective opinions of medical specialists. It is characterized by the weighted average values of the Spearman Rank-order Correlation Coefficient, Kendall Rank-order Correlation Coefficient, and Pearson Linear Correlation Coefficient of 0.8754, 0.7185, and 0.9062, respectively.

Blind quality assessment of multi-exposure fused images considering the detail, structure and color characteristics.

In the process of multi-exposure image fusion (MEF), the appearance of various distortions will inevitably cause the deterioration of visual quality. It is essential to predict the visual quality of MEF images. In this work, a novel blind image quality assessment (IQA) method is proposed for MEF images considering the detail, structure, and color characteristics. Specifically, to better perceive the detail and structure distortion, based on the joint bilateral filtering, the MEF image is decomposed into two layers (i.e., the energy layer and the structure layer). Obviously, this is a symmetric process that the two decomposition results can independently and almost completely describe the information of MEF images. As the former layer contains rich intensity information and the latter captures some image structures, some energy-related and structure-related features are extracted from these two layers to perceive the detail and structure distortion phenomena. Besides, some color-related features are also obtained to present the color degradation which are combined with the above energy-related and structure-related features for quality regression. Experimental results on the public MEF image database demonstrate that the proposed method achieves higher performance than the state-of-the-art quality assessment ones.

АНАЛІЗ СУБ’ЄКТИВНИХ МЕТРИК І ЕКСПЕРТНИХ МЕТОДІВ ОЦІНЮВАННЯ ЯКОСТІ ЗОБРАЖЕНЬ

An analysis of subjective metrics and expert methods of image quality assessment is provided

A versatile blind JPEG image quality assessment method

A versatile blind JPEG image quality assessment method

No-Reference Image Quality Assessment Using the Statistics of Global and Local Image Features

Methods of image quality assessment are widely used for ranking computer vision algorithms or controlling the perceptual quality of video and streaming applications. The ever-increasing number of digital images has encouraged the research in this field at an accelerated pace in recent decades. After the appearance of convolutional neural networks, many researchers have paid attention to different deep architectures to devise no-reference image quality assessment algorithms. However, many systems still rely on handcrafted features to ensure interpretability and restrict the consumption of resources. In this study, our efforts are focused on creating a quality-aware feature vector containing information about both global and local image features. Specifically, the research results of visual physiology indicate that the human visual system first quickly and automatically creates a global perception before gradually focusing on certain local areas to judge the quality of an image. Specifically, a broad spectrum of statistics extracted from global and local image features is utilized to represent the quality-aware aspects of a digital image from various points of view. The experimental results demonstrate that our method’s predicted quality ratings relate strongly with the subjective quality ratings. In particular, the introduced algorithm was compared with 16 other well-known advanced methods and outperformed them by a large margin on 9 accepted benchmark datasets in the literature: CLIVE, KonIQ-10k, SPAQ, BIQ2021, TID2008, TID2013, MDID, KADID-10k, and GFIQA-20k, which are considered de facto standards and generally accepted in image quality assessment.

An Innovative Possibilistic Fingerprint Quality Assessment (PFQA) Filter to Improve the Recognition Rate of a Level-2 AFIS.

In this paper, we propose an innovative approach to improve the performance of an Automatic Fingerprint Identification System (AFIS). The method is based on the design of a Possibilistic Fingerprint Quality Assessment (PFQA) filter where ground truths of fingerprint images of effective and ineffective quality are built by learning. The first approach, QS_I, is based on the AFIS decision for the image without considering its paired image to decide its effectiveness or ineffectiveness. The second approach, QS_PI, is based on the AFIS decision when considering the pair (effective image, ineffective image). The two ground truths (effective/ineffective) are used to design the PFQA filter. PFQA discards the images for which the AFIS does not generate a correct decision. The proposed intervention does not affect how the AFIS works but ensures a selection of the input images, recognizing the most suitable ones to reach the AFIS's highest recognition rate (RR). The performance of PFQA is evaluated on two experimental databases using two conventional AFIS, and a comparison is made with four current fingerprint image quality assessment (IQA) methods. The results show that an AFIS using PFQA can improve its RR by roughly 10% over an AFIS not using an IQA method. However, compared to other fingerprint IQA methods using the same AFIS, the RR improvement is more modest, in a 5-6% range.

Generation-Based Joint Luminance-Chrominance Learning for Underwater Image Quality Assessment

Underwater enhanced images (UEIs) are affected by not only the color cast and haze effect due to light attenuation and scattering, but also the over-enhancement and texture distortion caused by enhancement algorithms. However, existing underwater image quality assessment (UIQA) methods mainly focus on the inherent distortion caused by underwater optical imaging, and ignore the widespread artificial distortion, which leads to poor performance in evaluating UEIs. In this paper, a novel mapping-based underwater image quality representation is proposed. We divide underwater enhanced images into different domains and utilize a feature vector to measure the distance from the raw image domain to each enhanced image domain. The length and direction of the vector are defined as the enhancement degree and enhancement direction of the image. We construct a best enhancement direction and map other vectors to this direction to obtain the corresponding quality representation. Based on this, a novel network, called generation-based joint luminance-chrominance underwater image quality evaluation (GLCQE), is proposed, which is mainly divided into three parts: bi-directional reference generation module (BRGM), chromatic distortion evaluation network (CDEN), and sharpness distortion evaluation network (SDEN). BRGM is designed to generate two reference images about the unenhanced and the optimal enhanced versions of input UEI. In addition, the distortions in the luminance and chrominance domains of the UEI are analyzed. The luminance and chrominance channels of images are separated and input to SDEN and CDEN respectively to detect different distortions. A multi-scale feature mapping module is proposed in CDEN and SDEN to extract the feature representation of quality in chrominance and luminance of these images respectively. Moreover, a parallel spatial attention module is designed to focus on distortions in structural space by utilizing the different receptive fields of the convolution layer, due to the diverse manifestations of structural loss in the image. Finally, the mapped features extracted by two collaborative networks help the model evaluate the quality of underwater images more accurately. Extensive experiments demonstrate the superiority of our model against other representative state-of-the-art models.

Subjective Assessment of Objective Image Quality Metrics Range Guaranteeing Visually Lossless Compression.

The usage of media such as images and videos has been extensively increased in recent years. It has become impractical to store images and videos acquired by camera sensors in their raw form due to their huge storage size. Generally, image data is compressed with a compression algorithm and then stored or transmitted to another platform. Thus, image compression helps to reduce the storage size and transmission cost of the images and videos. However, image compression might cause visual artifacts, depending on the compression level. In this regard, performance evaluation of the compression algorithms is an essential task needed to reconstruct images with visually or near-visually lossless quality in case of lossy compression. The performance of the compression algorithms is assessed by both subjective and objective image quality assessment (IQA) methodologies. In this paper, subjective and objective IQA methods are integrated to evaluate the range of the image quality metrics (IQMs) values that guarantee the visually or near-visually lossless compression performed by the JPEG 1 standard (ISO/IEC 10918). A novel "Flicker Test Software" is developed for conducting the proposed subjective and objective evaluation study. In the flicker test, the selected test images are subjectively analyzed by subjects at different compression levels. The IQMs are calculated at the previous compression level, when the images were visually lossless for each subject. The results analysis shows that the objective IQMs with more closely packed values having the least standard deviation that guaranteed the visually lossless compression of the images with JPEG 1 are the feature similarity index measure (FSIM), the multiscale structural similarity index measure (MS-SSIM), and the information content weighted SSIM (IW-SSIM), with average values of 0.9997, 0.9970, and 0.9970 respectively.

Noise quality estimation on portraits in realistic controlled scenarios

The wide use of cameras by the public has raised the interest of image quality evaluation and ranking. Current cameras embed complex processing pipelines that adapt strongly to the scene content by implementing, for instance, advanced noise reduction or local adjustment on faces. However, current methods of Image Quality assessment are based on static geometric charts which are not representative of the common camera usage that targets mostly portraits. Moreover, on non-synthetic content most relevant features such as detail preservation or noisiness are often un-tractable. To overcome this situation, we propose to mix classical measurements and Machine learning based methods: we reproduce realistic content triggering this complex processing pipelines in controlled conditions in the lab which allows for rigorous quality assessment. Then, ML based methods can reproduce perceptual quality annotated previously. In this paper, we focus on noise quality evaluation and test on two different set ups: closeup and distant portraits. These setups provide scene capture conditions flexibility, but most of all, allow the evaluation of all quality camera ranges from high quality DSLR to poor quality video conference. Our numerical results show the relevance of our solution compared to geometric charts and the importance of adapting to realistic content.