Image Quality Assessment Approach Research Articles

Omnidirectional image quality assessment with local–global vision transformers

With the rising popularity of omnidirectional images (ODIs) in virtual reality applications, the need for specialized image quality assessment (IQA) methods becomes increasingly critical. Traditional IQA approaches, designed for rectilinear images, often fail to evaluate ODIs accurately due to their 360-degree scene representation. Addressing this, we introduce the Local–Global Transformer for 360-degree Image Quality Assessment (LGT360IQ). This novel framework features dual branches tailored to mimic top-down and bottom-up visual attention mechanisms, adapted for the spherical characteristics of ODIs. The local branch processes tangent viewports from salient regions within the equirectangular projection image, extracting detailed features for granular quality assessment. In parallel, the global branch utilizes a task-dependent token sampling strategy for holistic image feature processing and quality score prediction. This integrated approach combines local and global information, offering an effective IQA method for ODIs. Our extensive evaluation across three benchmark ODI datasets, CVIQ, OIQA, and ODI, demonstrates LGT360IQ superior performance and establishes its role in advancing the field of IQA for omnidirectional images.

A full reference quality assessment method with fused monocular and binocular features for stereo images.

Aiming to automatically monitor and improve stereoscopic image and video processing systems, stereoscopic image quality assessment approaches are becoming more and more important as 3D technology gains popularity. We propose a full-reference stereoscopic image quality assessment method that incorporate monocular and binocular features based on binocular competition and binocular integration. To start, we create a three-channel RGB fused view by fusing Gabor filter bank responses and disparity maps. Then, using the monocular view and the RGB fusion view, respectively, we extract monocular and binocular features. To alter the local features in the binocular features, we simultaneously estimate the saliency of the RGB fusion image. Finally, the monocular and binocular quality scores are calculated based on the monocular and binocular features, and the quality scores of the stereo image prediction are obtained by fusion. Performance testing in the LIVE 3D IQA database Phase I and Phase II. The results of the proposed method are compared with newer methods. The experimental results show good consistency and robustness.

Blind image quality assessment by pairwise ranking image series

Image quality assessment (IQA) is constantly innovating, but there are still three types of stickers that have not been resolved: the "content sticker" - limitation of training set, the "annotation sticker" - subjective instability in opinion scores and the "distortion sticker" - disordered distortion settings. In this paper, a No-Reference Image Quality Assessment (NR IQA) approach is proposed to deal with the problems. For "content sticker", we introduce the idea of pairwise comparison and generate a largescale ranking set to pre-train the network; For "annotation sticker", the absolute noise-containing subjective scores are transformed into ranking comparison results, and we design an indirect unsupervised regression based on Eigenvalue Decomposition (EVD); For "distortion sticker", we propose a perception-based distortion classification method, which makes the distortion types clear and refined. Experiments have proved that our NR IQA approach Experiments show that the algorithm performs well and has good generalization ability. Furthermore, the proposed perception based distortion classification method would be able to provide insights on how the visual related studies may be developed and to broaden our understanding of human visual system.

Two image quality assessment methods based on evidential modeling and uncertainty: application to automatic iris identification systems

The performance of an Automatic iris Identification System is impacted by both the poor quality of iris images and the uncertainty of information. Assessing image quality and rejecting poor-quality images can substantially improve the performances of the current biometric systems. The main idea behind our proposed Image Quality Assessment approaches is to take advantage, firstly, of the texture of iris images and, secondly, of the uncertainty of these information. This is achieved by defining a set of Contextual Quality Indicators extracted from the image texture and transforming them into Quality Assessment Criteria in the evidential framework, taking into account the information uncertainty degree. The Contextual Quality Indicators are defined based on a priori analysis of the context of the application. We use ‘iris’ as the context of application. Generally, only the normalized iris image is saved, i.e. the acquired iris image is not always available. So, the main advantage of our approaches over other related methods is that it can act in the normalization level of the processing chain to reject poor-quality images. So that, the subsequent Automatic iris Identification System can process only good-quality images, which result in better recognition rate performance. The functioning of our evidential approaches is illustrated using image samples from CASIA 1.0 database. The performance of over the proposed image quality assessment approaches is compared with the standard iris identification system without an image quality assessment step. A statistical test, based on 95% confidence interval, is used to assess if there is a statistically significant difference between the performances of the proposed approaches. The CASIA 1.0 has been used to make the comparison. The comparison results highlight the effectiveness of the proposed approaches for iris domain of applications. The source code of our paper is available at https://github.com/Sonda09/IIQA

LIQA: Lifelong Blind Image Quality Assessment

The image distortions are complex and dynamically changing in the real-world scenario, due to the fast development of the image processing system. The blind image quality assessment (BIQA) models may encounter the challenge of processing images with distortion types never seen before deployment. However, existing BIQA models generally cannot evolve with unseen distortion types adaptively, which greatly limits the deployment and application of BIQA models in real-world scenarios. To address this problem, we propose a novel Lifelong blind Image Quality Assessment (LIQA) approach, targeting to achieve the lifelong learning of BIQA. Without accessing to previous training data, our proposed LIQA can not only learn new knowledge, but also mitigate the catastrophic forgetting of learned knowledge. Specifically, we adopt the Split-and-Merge distillation strategy to train a single-head network that makes task-agnostic predictions. In the split stage, we first employ a distortion-specific generator to generate pseudo features of each previously seen distortion. Then, we utilize an auxiliary multi-head regression network to keep the response of each distortion. In the merge stage, we replay the pseudo features and use the pseudo labels generated by the auxiliary multi-head network to distill the knowledge of the multiple heads, which can build the final regression single head. Extensive experiments demonstrate that LIQA can perform well in handling both inner-dataset distortion shift and cross-dataset distortion shift. More importantly, our model can achieve stable performance even if the task sequences are long.

CSPP-IQA: a multi-scale spatial pyramid pooling-based approach for blind image quality assessment.

The traditional image quality assessment (IQA) methods are usually based on convolutional neural networks (CNNs). For these IQA methods using CNNs, limited by the feature size of the fully connected layer, the input image needs be tailored to a pre-defined size, which usually results in destroying the original structure and content of the input image and thus reduces the accuracy of the quality assessment. In this paper, a blind image quality assessment method (named CSPP-IQA), which is based on multi-scale spatial pyramid pooling, is proposed. CSPP-IQA allows inputting the original image when assessing the image quality without any image adjustment. Moreover, by facilitating the convolutional block attention module and image understanding module, CSPP-IQA achieved better accuracy, generalization and efficiency than traditional IQA methods. The result of experiments running on real-scene IQA datasets in this study verified the effectiveness and efficiency of CSPP-IQA.

From Distortion Manifold to Perceptual Quality: a Data Efficient Blind Image Quality Assessment Approach

Though current no-reference image quality assessment (NR-IQA) approaches have achieved impressive performance gain thanks to deep learning techniques, it is claimed that the risk of over-fitting exists. To improve model generalization ability, most of the current researches incorporate mass data to train or tune the data-driven models. However, the process of image data collection and quality label annotation is quite time-consuming and labour-intensive. Therefore, in this paper, we explore an alternative solution to promote model generalizability but with relatively small fractions of training data. Compared with previous approaches which make effort to approximate the whole complex image distribution, we propose to explicitly learn an image distortion manifold first, which lies in a much lower dimension space and also representative in capturing general degradation patterns. We then project the images to their perceived quality from the learned manifold to obtain quality predictions. Since the manifold embeds general distortion features despite of varying image contents, it can be learned with relatively small amount of samples. In order to learn the manifold and quality projection, we introduce a two-branched network to learn both low level distortions and high level semantics. We also propose a simple but efficient training framework, composing of a masked labelling strategy and a gradual weighting curriculum to fulfill the task. Thanks to the learned distortion manifold, the proposed model achieves superior generalizability compared with previous models. Extensive experiments demonstrate its effectiveness in terms of training with limited data, testing on large scale images, and with unseen types of distorted images.

BIQ2021: a large-scale blind image quality assessment database

The assessment of the perceptual quality of digital images is becoming increasingly important as a result of the widespread use of digital multimedia devices. Smartphones and high-speed internet are just two examples of technologies that have multiplied the amount of multimedia content available. Thus, obtaining a representative dataset, which is required for objective quality assessment training, is a significant challenge. The Blind Image Quality Assessment Database, BIQ2021, is presented in this article. By selecting images with naturally occurring distortions and reliable labeling, the dataset addresses the challenge of obtaining representative images for no-reference image quality assessment. The dataset consists of three sets of images: those taken without the intention of using them for image quality assessment, those taken with intentionally introduced natural distortions, and those taken from an open-source image-sharing platform. It is attempted to maintain a diverse collection of images from various devices, containing a variety of different types of objects and varying degrees of foreground and background information. To obtain reliable scores, these images are subjectively scored in a laboratory environment using a single stimulus method. The database contains information about subjective scoring, human subject statistics, and the standard deviation of each image. The dataset's Mean Opinion Scores (MOS) make it useful for assessing visual quality. Additionally, the proposed database is used to evaluate existing blind image quality assessment approaches, and the scores are analyzed using Pearson and Spearman's correlation coefficients. The image database and MOS are freely available for use and benchmarking.

Blind image quality assessment of magnetic resonance images with statistics of local intensity extrema

Magnetic resonance (MR) imaging provides a large amount of data that requires a visual inspection before a diagnosis can be made. Since the exclusion of low-quality image sequences is performed manually and image processing methods are evaluated using techniques developed for natural images, automatic and reliable MR image quality assessment (IQA) approaches are desirable. Therefore, in this work, a new no-reference (NR) MR-IQA technique is proposed. The method uses introduced quality-aware features addressing characteristics of MR images. Specifically, in the method, an MR image is scaled, filtered with two gradient operators, and subjected to identification of the local intensity extrema. Then, the entropy and κ curvature are calculated to characterize extrema sequences and used as perceptual features to train a quality model with the Support Vector Regression (SVR) technique. In this paper, an extensive comparative evaluation of the method against recent NR approaches, including deep learning-based models, is conducted on two representative MR-IQA benchmarks. The results reveal the superiority of the introduced approach over competing methods as it obtained better overall Spearman and Pearson correlation coefficients by 5% and 3%, respectively.

Image quality assessment for machine learning tasks using meta-reinforcement learning.

In this paper, we consider image quality assessment (IQA) as a measure of how images are amenable with respect to a given downstream task, or task amenability. When the task is performed using machine learning algorithms, such as a neural-network-based task predictor for image classification or segmentation, the performance of the task predictor provides an objective estimate of task amenability. In this work, we use an IQA controller to predict the task amenability which, itself being parameterised by neural networks, can be trained simultaneously with the task predictor. We further develop a meta-reinforcement learning framework to improve the adaptability for both IQA controllers and task predictors, such that they can be fine-tuned efficiently on new datasets or meta-tasks. We demonstrate the efficacy of the proposed task-specific, adaptable IQA approach, using two clinical applications for ultrasound-guided prostate intervention and pneumonia detection on X-ray images.

Image quality assessment by overlapping task-specific and task-agnostic measures: application to prostate multiparametric MR images for cancer segmentation

Image quality assessment (IQA) in medical imaging can be used to ensure that downstream clinical tasks can be reliably performed. Quantifying the impact of an image on the specific target tasks, also named as task amenability, is needed. A <em>task-specific</em> IQA has recently been proposed to learn an image-amenability-predicting controller simultaneously with a target task predictor. This allows for the trained IQA controller to measure the impact an image has on the target task performance, when this task is performed using the predictor, e.g. segmentation and classification neural networks in modern clinical applications. In this work, we propose an extension to this task-specific IQA approach, by adding a <em>task-agnostic</em> IQA based on auto-encoding as the target task. Analysing the intersection between low-quality images, deemed by both the task-specific and task-agnostic IQA, may help to differentiate the underpinning factors that caused the poor target task performance. For example, common imaging artefacts may not adversely affect the target task, which would lead to a low task-agnostic quality and a high task-specific quality, whilst individual cases considered clinically challenging, which can not be improved by better imaging equipment or protocols, is likely to result in a high task-agnostic quality but a low task-specific quality. We first describe a flexible reward shaping strategy which allows for the adjustment of weighting between task-agnostic and task-specific quality scoring. Furthermore, we evaluate the proposed reinforcement learning algorithm, using a clinically challenging target task of prostate tumour segmentation on multiparametric magnetic resonance (mpMR) images. Based on experimental results using mpMR images from 850 patients, it was found that <em>a</em>) The task-agnostic IQA may identify artefacts, but with limited impact on the accuracy of cancer segmentation networks. A Dice score of 0.367±0.017 was obtained after rejecting 10% of low quality images, compared to 0.354±0.016 from a non-selective baseline; <em>b</em>} The task-specific IQA alone improved the performance to 0.415±0.020, at the same rejection ratio. However, this system indeed rejected both images that impact task performance due to imaging defects and due to being clinically challenging; and <em>c</em>) The proposed reward shaping strategy, when the task-agnostic and task-specific IQA are weighted appropriately, successfully identified samples that need re-acquisition due to defected imaging process, as opposed to clinically challenging cases due to low contrast in pathological tissues or other equivocacy in radiological presentation.<br>Our code is available at https://github.com/s-sd/task-amenability/tree/v1

Cross-Domain Feature Similarity Guided Blind Image Quality Assessment.

This work proposes an end-to-end cross-domain feature similarity guided deep neural network for perceptual quality assessment. Our proposed blind image quality assessment approach is based on the observation that features similarity across different domains (e.g., Semantic Recognition and Quality Prediction) is well correlated with the subjective quality annotations. Such phenomenon is validated by thoroughly analyze the intrinsic interaction between an object recognition task and a quality prediction task in terms of characteristics of the human visual system. Based on the observation, we designed an explicable and self-contained cross-domain feature similarity guided BIQA framework. Experimental results on both authentical and synthetic image quality databases demonstrate the superiority of our approach, as compared to the state-of-the-art models.

No Reference Pansharpened Image Quality Assessment Through Deep Feature Similarity

Pansharpening refers to the process of enhancing the spatial resolution of a multispectral image with the help of a high spatial resolution panchromatic image. Quality assessment (QA) of pansharpened images helps provide a formal framework for the analysis and design of pansharpening methods and is thus extremely important. However, lack of availability of a reference multispectral image makes QA of pansharpening algorithms a challenging task. Given the popular use of QA algorithms that use a reference, this paper focuses on predicting the quality under a &#x2018;no-reference&#x0027; (NR) setting. Specifically, a learning based NR pansharpened image quality assessment (IQA) approach is adopted to predict state of the art reference based measures such as <inline-formula><tex-math notation="LaTeX">$Q2^{n}$</tex-math></inline-formula> and SAM without the need of a reference. We design an end-to-end deep pansharpening IQA network to compute the similarity of deep features fused from the panchromatic and input low resolution multispectral with similar features extracted from the given pansharpened image. To train and test our learning based approach, we create a large corpus of pansharpened images belonging to different satellites and thematic scenes by applying different pansharpening algorithms. Our experiments demonstrate that our NR pansharpened IQA algorithm achieves excellent performance and generalizes well across different satellites and resolutions. The source code and the dataset will be made available at <uri>https://github.com/neerajbadal/DPIQA</uri>.

Neural evidence for image quality perception based on algebraic topology.

In this paper, the algebraic topological characteristics of brain networks composed of electroencephalogram(EEG) signals induced by different quality images were studied, and on that basis, a neurophysiological image quality assessment approach was proposed. Our approach acquired quality perception-related neural information via integrating the EEG collection with conventional image assessment procedures, and the physiologically meaningful brain responses to different distortion-level images were obtained by topological data analysis. According to the validation experiment results, statistically significant discrepancies of the algebraic topological characteristics of EEG data evoked by a clear image compared to that of an unclear image are observed in several frequency bands, especially in the beta band. Furthermore, the phase transition difference of brain network caused by JPEG compression is more significant, indicating that humans are more sensitive to JPEG compression other than Gaussian blur. In general, the algebraic topological characteristics of EEG signals evoked by distorted images were investigated in this paper, which contributes to the study of neurophysiological assessment of image quality.

Multi-gene Genetic Programming based Predictive Models for Full-reference Image Quality Assessment

Abstract Many objective quality metrics for assessing the visual quality of images have been developed during the last decade. A simple way to fine tune the efficiency of assessment is through permutation and combination of these metrics. The goal of this fusion approach is to take advantage of the metrics utilized and minimize the influence of their drawbacks. In this paper, a symbolic regression technique using an evolutionary algorithm known as multi-gene genetic programming (MGGP) is applied for predicting subject scores of images in datasets using a combination of objective scores of a set of image quality metrics (IQM). By learning from image datasets, the MGGP algorithm can determine appropriate image quality metrics, from 21 metrics utilized, whose objective scores employed as predictors in the symbolic regression model, by optimizing simultaneously two competing objectives of model ‘goodness of fit’ to data and model ‘complexity’. Six large image databases (namely LIVE, CSIQ, TID2008, TID2013, IVC and MDID) that are available in public domain are used for learning and testing the predictive models, according the k-fold-cross-validation and the cross dataset strategies. The proposed approach is compared against state-of-the-art objective image quality assessment approaches. Results of comparison reveal that the proposed approach outperforms other state-of-the-art recently developed fusion approaches.

Blind image quality assessment with channel attention based deep residual network and extended LargeVis dimensionality reduction

Image Quality Assessment (IQA) is one of the fundamental problems in the fields of image processing, image/video coding and transmission, and so on. In this paper, a Blind Image Quality Assessment (BIQA) approach with channel attention based deep Residual Network (ResNet)and extended LargeVis dimensionality reduction is proposed. Firstly, ResNet50 with channel attention mechanism is used as the backbone network to extract the deep features from the image. In order to reduce the dimensionality of the deep features, LargeVis, which is originally designed for the visualization of large scale high-dimensional data, is extended by using Support Vector Regression (SVR) to perform on a single feature vector data. The extended LargeVis can remove the redundant information of the deep features so as to obtain a low-dimensional and discriminative feature representation. Finally, the quality prediction model is established by using SVR as the fitting method. The low-dimensional feature representation and quality score of the image form the pair-wise data samples to train the fitting model. Experimental results on authentic distortions datasets and synthetic distortions datasets show that our proposed method can achieve superior performance compared with the state-of-the-art methods.

Quantifying Visual Image Quality: A Bayesian View.

Image quality assessment (IQA) models aim to establish a quantitative relationship between visual images and their quality as perceived by human observers. IQA modeling plays a special bridging role between vision science and engineering practice, both as a test-bed for vision theories and computational biovision models and as a powerful tool that could potentially have a profound impact on a broad range of image processing, computer vision, and computer graphics applications for design, optimization, and evaluation purposes. The growth of IQA research has accelerated over the past two decades. In this review, we present an overview of IQA methods from a Bayesian perspective, with the goals of unifying a wide spectrum of IQA approaches under a common framework and providing useful references to fundamental concepts accessible to vision scientists and image processing practitioners. We discuss the implications of the successes and limitations of modern IQA methods for biological vision and the prospect for vision science to inform the design of future artificial vision systems. (The detailed model taxonomy can be found at http://ivc.uwaterloo.ca/research/bayesianIQA/.).

An efficient approach for no-reference image quality assessment based on statistical texture and structural features

This paper presents a rotation-invariant and computationally efficient no-reference image quality assessment (NR-IQA) model. It estimates the image quality based on texture and structural information associated with the images. The human visual system (HVS) uses perceptual features such as texture and structure as primary information to understand the visual scene and the image content. Moreover, the texture and structural information capture the loss of naturalness due to distortions in the image. Therefore, in this work, the important texture features are extracted using the local binary patterns (LBP).The modified LBP, also known as the hyper-smoothing LBP (H-LBP) and Laplacian of H-LBP (LH-LBP), represents the image structure. Further, the image quality prediction model computes the quality of the image based on the statistical feature measures of the texture and structural information. In the proposed approach, the image quality prediction model uses support vector regression (SVR) to measure image quality. Various experimentations are carried out on the LIVE and TID2013 database to test the effectiveness of the proposed NR-IQA model. The performance metrics such as Spearman rank-ordered correlation coefficient, Pearson linear correlation coefficient, and root mean square error is computed to show the efficiency of the presented approach. The experimental results illustrate a high correlation between the predicted quality score and the human visual perceptions. It is also found to be competitive with the best-performing full-reference and no-reference IQA models.

DLNR-SIQA: Deep Learning-Based No-Reference Stitched Image Quality Assessment

Due to recent advancements in virtual reality (VR) and augmented reality (AR), the demand for high quality immersive contents is a primary concern for production companies and consumers. Similarly, the topical record-breaking performance of deep learning in various domains of artificial intelligence has extended the attention of researchers to contribute to different fields of computer vision. To ensure the quality of immersive media contents using these advanced deep learning technologies, several learning based Stitched Image Quality Assessment methods have been proposed with reasonable performances. However, these methods are unable to localize, segment, and extract the stitching errors in panoramic images. Further, these methods used computationally complex procedures for quality assessment of panoramic images. With these motivations, in this paper, we propose a novel three-fold Deep Learning based No-Reference Stitched Image Quality Assessment (DLNR-SIQA) approach to evaluate the quality of immersive contents. In the first fold, we fined-tuned the state-of-the-art Mask R-CNN (Regional Convolutional Neural Network) on manually annotated various stitching error-based cropped images from the two publicly available datasets. In the second fold, we segment and localize various stitching errors present in the immersive contents. Finally, based on the distorted regions present in the immersive contents, we measured the overall quality of the stitched images. Unlike existing methods that only measure the quality of the images using deep features, our proposed method can efficiently segment and localize stitching errors and estimate the image quality by investigating segmented regions. We also carried out extensive qualitative and quantitative comparison with full reference image quality assessment (FR-IQA) and no reference image quality assessment (NR-IQA) on two publicly available datasets, where the proposed system outperformed the existing state-of-the-art techniques.

Image Quality Assessment Using Kernel Sparse Coding

One key in image quality assessment (IQA) is the design of image representations that can capture the changes of image structures caused by distortions. Recent studies show that sparse coding has emerged as a promising approach to analyzing image structures for IQA. However, existing sparse-coding-based IQA approaches use linear coding models, which ignore the nonlinearities of manifolds of image patches and thus cannot analyze complex image structures well. To overcome such a weakness, in this paper, we introduce nonlinear sparse coding to IQA. A kernel dictionary construction scheme is proposed, which combines analytic dictionaries and learnable dictionaries to guarantee both the stability and effectiveness of kernel sparse coding in the context of IQA. Built upon the kernel dictionary construction, an effective full-reference IQA metric is developed. Benefiting from the considerations on nonlinearities during sparse coding, the proposed IQA metric not only characterizes image distortions better, but also achieves improvement on the consistency with subjective perception, when compared to the metrics built upon linear sparse coding. Such benefits are demonstrated with the experimental results on eight benchmark datasets in terms of common criteria.