Image Quality Assessment Task Research Articles

Shift-insensitive perceptual feature of quadratic sum of gradient magnitude and LoG signals for image quality assessment and image classification

Most existing full-reference (FR) Image quality assessment (IQA) models work in the premise of that the two images should be well registered. Shifting an image would lead to an inaccurate evaluation of image quality, because small spatial shifts are far less noticeable than structural distortion for human observers. To this regard, we propose to study an IQA feature that is shift-insensitive to the basic primitive structure of images, i.e., image edge. According to previous studies, the image gradient magnitude (GM) and the Laplacian of Gaussian (LoG) operator that depict the edge profiles of natural images are highly efficient structural features in IQA tasks. In this paper, we find that the Quadratic sum of the normalized GM and the LoG signals (QGL) has excellent shift-insensitive property in representing image edges after theoretically solving the selection problem of a ratio parameter to balance the GM and LoG signals. Based on the proposed QGL feature, two FR-IQA models can be built directly by measuring the similarity map with mean and standard deviation pooling strategies, named mQGL and sQGL, respectively. Experimental results show that the proposed sQGL and mQGL work robustly on four benchmark IQA databases, and QGL-based models show great shift-insensitive property to spatial translation and image rotation while judging the image quality. In addition, we explore the feasibility of combining QGL feature with deep neural networks, and verify that it can help to promote image pattern recognition in texture classification tasks.

McmIQA: Multi-Module Collaborative Model for No-Reference Image Quality Assessment

No reference image quality assessment is a technique that uses computers to simulate the human visual system and automatically evaluate the perceived quality of images. In recent years, with the widespread success of deep learning in the field of computer vision, many end-to-end image quality assessment algorithms based on deep learning have emerged. However, unlike other computer vision tasks that focus on image content, an excellent image quality assessment model should simultaneously consider distortions in the image and comprehensively evaluate their relationships. Motivated by this, we propose a Multi-module Collaborative Model for Image Quality Assessment (McmIQA). The image quality assessment is divided into three subtasks: distortion perception, content recognition, and correlation mapping. And specific modules are constructed for each subtask: the distortion perception module, the content recognition module, and the correlation mapping module. Specifically, we apply two contrastive learning frameworks on two constructed datasets to train the distortion perception module and the content recognition module to extract two types of features from the image. Subsequently, using these extracted features as input, we employ a ranking loss to train the correlation mapping module to predict image quality on image quality assessment datasets. Extensive experiments conducted on seven relevant datasets demonstrated that the proposed method achieves state-of-the-art performance in both synthetic distortion and natural distortion image quality assessment tasks.

Auxiliary Information Guided Self-attention for Image Quality Assessment

Image quality assessment (IQA) is an important problem in computer vision with many applications. We propose a transformer-based multi-task learning framework for the IQA task. Two subtasks: constructing an auxiliary information error map and completing image quality prediction, are jointly optimized using a shared feature extractor. We use visual transformers (ViT) as a feature extractor for feature extraction and guide ViT to focus on image quality-related features by building auxiliary information error map subtask. In particular, we propose a fusion network that includes a channel focus module. Unlike the fusion methods commonly used in previous IQA methods, we use the fusion network, including the channel attention module, to fuse the auxiliary information error map features with the image features, which facilitates the model to mine the image quality features for more accurate image quality assessment. And by jointly optimizing the two subtasks, ViT focuses more on extracting image quality features and building a more precise mapping from feature representation to quality score. With slight adjustments to the model, our approach can be used in both no-reference (NR) and full-reference (FR) IQA environments. We evaluate the proposed method in multiple IQA databases, showing better performance than state-of-the-art FR and NR IQA methods.

A brain-inspired quality assessment model for sonar image super-resolution

Sonar sensors are vital in the marine industry for detecting underwater targets in challenging conditions. The imaging distance and image resolution are negatively correlated due to the propagation characteristics of sound waves in water. Although Super-Resolution (SR) techniques alleviate this limitation, they introduce complex distortions that may not fit the desired utility of reconstructed sonar images. Quantifying image quality is essential, yet existing Image Quality Assessment (IQA) algorithms fail to simultaneously consider reconstruction distortions and the sonar image task background. Furthermore, the scarcity of sonar images poses challenges for deep-learning-based algorithms. To address these issues, we propose a brain-inspired model for Super-Resolution reconstructed Sonar Image Quality Assessment (SRSIQA) based on transfer learning. On the one hand, we adopt an effective feature extractor to extract multi-level features that align with the human brain recognition process. On the other hand, we develop an SEA-Block to implement feature weight adjustment and multi-level feature scale matching, given that some transferred features do not fit the IQA task well. Experimental results demonstrate the superiority of the proposed method.

A Soft-Reference Breast Ultrasound Image Quality Assessment Method That Considers the Local Lesion Area.

The quality of breast ultrasound images has a significant impact on the accuracy of disease diagnosis. Existing image quality assessment (IQA) methods usually use pixel-level feature statistical methods or end-to-end deep learning methods, which focus on the global image quality but ignore the image quality of the lesion region. However, in clinical practice, doctors' evaluation of ultrasound image quality relies more on the local area of the lesion, which determines the diagnostic value of ultrasound images. In this study, a global-local integrated IQA framework for breast ultrasound images was proposed to learn doctors' clinical evaluation standards. In this study, 1285 breast ultrasound images were collected and scored by experienced doctors. After being classified as either images with lesions or images without lesions, they were evaluated using soft-reference IQA or bilinear CNN IQA, respectively. Experiments showed that for ultrasound images with lesions, our proposed soft-reference IQA achieved PLCC 0.8418 with doctors' annotation, while the existing end-to-end deep learning method that did not consider the local lesion features only achieved PLCC 0.6606. Due to the accuracy improvement for the images with lesions, our proposed global-local integrated IQA framework had better performance in the IQA task than the existing end-to-end deep learning method, with PLCC improving from 0.8306 to 0.8851.

A Novel Blind Reference Method for Multi-Purpose Image Quality Assessment Tasks

A Novel Blind Reference Method for Multi-Purpose Image Quality Assessment Tasks

MMMNet: An End-to-End Multi-Task Deep Convolution Neural Network With Multi-Scale and Multi-Hierarchy Fusion for Blind Image Quality Assessment

As the evaluation of image quality depends on the human visual system (HVS), many existing image quality assessment (IQA) methods focus on modeling the HVS to account for subjective perception. The visual attention of the HVS makes humans more sensitive to distortion on the attended regions than on regions which are not the focus of attention. Therefore, we propose an end-to-end multi-task deep convolution neural network with multi-scale and multi-hierarchy fusion (MMMNet), in which the IQA and saliency subtasks are jointly optimized to improve saliency-guided IQA performance. Particularly, the incorporation of saliency information is achieved by fusing saliency features with IQA features hierarchically to progressively improve the IQA features over network depth. A multi-scale feature extraction module (MSFE) is proposed to provide effective saliency features for the IQA network. Based on the saliency fusion, MMMNet introduces an auxiliary saliency task, achieving the multi-task learning to improve the generalization of the IQA task. Experimental results show that MMMNet achieves state-of-the-art performance and strong generalization ability on IQA databases.

TTL-IQA: Transitive Transfer Learning Based No-Reference Image Quality Assessment

Image quality assessment (IQA) based on deep learning faces the overfitting problem due to limited training samples available in existing IQA databases. Transfer learning is a plausible solution to the problem, in which the shared features derived from the large-scale Imagenet source domain could be transferred from the original recognition task to the intended IQA task. However, the Imagenet source domain and the IQA target domain as well as their corresponding tasks are not directly related. In this paper, we propose a new transitive transfer learning method for no-reference image quality assessment (TTL-IQA). First, the architecture of the multi-domain transitive transfer learning for IQA is developed to transfer the Imagenet source domain to the auxiliary domain, and then to the IQA target domain. Second, the auxiliary domain and the auxiliary task are constructed by a new generative adversarial network based on distortion translation (DT-GAN). Furthermore, a TTL network of the semantic features transfer (SFTnet) is proposed to optimize the shared features for the TTL-IQA. Experiments are conducted to evaluate the performance of the proposed method on various IQA databases, including the LIVE, TID2013, CSIQ, LIVE multiply distorted and LIVE challenge. The results show that the proposed method significantly outperforms the state-of-the-art methods. In addition, our proposed method demonstrates a strong generalization ability.

Deep feature importance awareness based no-reference image quality prediction

Deep-learning based image quality assessment (IQA) algorithms usually use the transfer learning method that transfers a pre-trained network for classification task to handle IQA task. Although it can overcome the problem of having insufficient IQA databases to some extent, it cannot distinguish between the important and unimportant deep features for the IQA task, which potentially leads to inaccurate prediction performance. In this paper, we propose a no-reference IQA method based on modelling of deep feature importance. A SE-VGG network is developed by using adaptive transfer learning method. It can suppress the features of local parts of salient objects of images that are not important to the IQA task, and emphasize the features of image distortion and salient objects that are important to IQA task. Moreover, the structure of the SE-VGG is investigated to improve the accuracy of the image quality assessment on a small IQA database. Experiments are conducted to evaluate the performance of the proposed method on various databases, including the LIVE, TID2013, CSIQ, LIVE multiply distorted and LIVE challenge. The results show the proposed method significantly outperforms the state-of-the-art methods. In addition, our method demonstrates a strong generalization ability.

Image quality assessment via spatial‐transformed domains multi‐feature fusion

The basis of image processing is to evaluate and monitor image quality using algorithms rather than subjective methods. Conventional gradient operators have been popularly used in previous image quality assessment tasks to reflect the edge contour of an image, while there are some obvious defects in terms of the selection of mask scale and direction. Some improved versions are also less than ideal since they fail to consider the gradient information of the same pixel in different directions at the same time. The authors adopt a powerful gradient operator that can simultaneously capture edge information in all four directions at the same pixel point with more relevant values being considered instead of selecting the maximum in these four directions. Furthermore, four complementary types of features extracted from the spatial and transform domains are considered. A set of 12-dimensional feature vectors is generated for each image by multi-feature fusion. Ultimately, random forest regression technique is employed to train their model and then map the distortion effects to the prediction scores. The experimental results show that the proposed FVC-G has better overall performance, more powerful cross-database operation capability, and higher visual consistency than other advanced methods.

End-to-End Blind Image Quality Prediction With Cascaded Deep Neural Network

The deep convolutional neural network (CNN) has achieved great success in image recognition. Many image quality assessment (IQA) methods directly use recognition-oriented CNN for quality prediction. However, the properties of IQA task is different from image recognition task. Image recognition should be sensitive to visual content and robust to distortion, while IQA should be sensitive to both distortion and visual content. In this paper, an IQA-oriented CNN method is developed for blind IQA (BIQA), which can efficiently represent the quality degradation. CNN is large-data driven, while the sizes of existing IQA databases are too small for CNN optimization. Thus, a large IQA dataset is firstly established, which includes more than one million distorted images (each image is assigned with a quality score as its substitute of Mean Opinion Score (MOS), abbreviated as pseudo-MOS). Next, inspired by the hierarchical perception mechanism (from local structure to global semantics) in human visual system, a novel IQA-orientated CNN method is designed, in which the hierarchical degradation is considered. Finally, by jointly optimizing the multilevel feature extraction, hierarchical degradation concatenation (HDC) and quality prediction in an end-to-end framework, the Cascaded CNN with HDC (named as CaHDC) is introduced. Experiments on the benchmark IQA databases demonstrate the superiority of CaHDC compared with existing BIQA methods. Meanwhile, the CaHDC (with about 0.73M parameters) is lightweight comparing to other CNN-based BIQA models, which can be easily realized in the microprocessing system. The dataset and source code of the proposed method are available at https://web.xidian.edu.cn/wjj/paper.html .

Entropy Based Data Expansion Method for Blind Image Quality Assessment.

Image quality assessment (IQA) is a fundamental technology for image applications that can help correct low-quality images during the capture process. The ability to expand distorted images and create human visual system (HVS)-aware labels for training is the key to performing IQA tasks using deep neural networks (DNNs), and image quality is highly sensitive to changes in entropy. Therefore, a new data expansion method based on entropy and guided by saliency and distortion is proposed in this paper. We introduce saliency into a large-scale expansion strategy for the first time. We regionally add distortion to a set of original images to obtain a distorted image database and label the distorted images using entropy. The careful design of the distorted images and the entropy-based labels fully reflects the influences of both saliency and distortion on quality. The expanded database plays an important role in the application of a DNN for IQA. Experimental results on IQA databases demonstrate the effectiveness of the expansion method, and the network’s prediction effect on the IQA databases is found to be improved compared with its predecessor algorithm. Therefore, we conclude that a data expansion approach that fully reflects HVS-aware quality factors is beneficial for IQA. This study presents a novel method for incorporating saliency into IQA, namely, representing it as regional distortion.

Image quality assessment based on adaptive multiple Skyline query

Non-reference image quality assessment has attracted great emphasis in recent years. Traditional image quality assessment algorithms based on structural similarity cannot make full use of the image gradient features, and the contrast similarity features often ignore the consistency of continuous color blocks within the image, which leads to large discrepancy between the evaluation result and the subjective judgment of human vision system. In this paper, we propose a deep model for image quality assessment where the spatial and visual features of image are both considered. For better feature fusion, we design an adaptive multiple Skyline query algorithm named MSFF, which takes as input multiple features of images, and learns the feature weights through end-to-end training. Extensive experiments on image quality assessment tasks prove that the proposed model exhibits superior performance compared with existing solutions.

Siamese-Network-Based Learning to Rank for No-Reference 2D and 3D Image Quality Assessment

2D image quality assessment (IQA) and stereoscopic 3D IQA are considered as two different tasks in the literature. In this paper, we present an index for both no-reference 2D and 3D IQA. We propose to transform the IQA task into a task of quality comparison between images. By generating image pairs, the amount of training data reaches the square of the original amount of data, effectively solving the lacking of training samples. We also propose a learning to rank model using Siamese convolutional neural networks (LRSN) for quality comparison. The presented LRSN has two branches that have the same structure, share weights with each other, and take two image patches as inputs. The goal of LRSN is learning to rank the quality scores between the two input image patches. The relative quality score of a test image is obtained by first comparing its image patches with many image patches of other images and counts the number of times that its image patches are ranked superior to other patches. The experimental results on three 2D (LIVE, CSIQ, and TID2013) and three 3D (LIVE 3D Phase-I, LIVE 3D Phase-II, and NBU) IQA databases demonstrate that the proposed LRSN model works well for both 2D and 3D IQA and outperforms the state-of-the-art no-reference 2D and 3D IQA metrics.

Naturalization Module in Neural Networks for Screen Content Image Quality Assessment

Deep learning approaches have demonstrated success in no-reference image quality assessment tasks. However, due to the specific properties of screen content images (SCIs), deep neural networks for SCI quality assessment are not as optimal as those designed for images depicting natural scenes. In order to tackle this discrepancy, a “naturalization” module composed of an upsampling layer and a convolutional layer is proposed to transform SCIs to have characteristics more similar to that of natural images. In addition, a new deep learning model architecture along with data augmentation techniques tailored to SCIs are implemented. The performance of the proposed approach is evaluated on the Screen Image Quality Assessment Database and Screen Content Image Database, and has shown to have superior performance to state-of-the-art methods in predicting the perceptual quality of SCIs.

A model observer study using acquired mammographic images of an anthropomorphic breast phantom.

To study the feasibility of a task-based framework composed of an anthropomorphic breast phantom and mathematical model observers (MOs) for the evaluation of system-processed mammographic images. A prototype anthropomorphic breast phantom with inserted gold discs of 0.1 mm and 0.25 mm diameter was imaged with two digital mammography systems (system A and B) at four different dose levels. From the acquired processed and unprocessed images, signal-present and signal-absent regions of interest (ROIs) were extracted. The ROIs were evaluated by a non-pre-whitening MO with eye filter (NPWE) and by three human observers in a two-alternative forced-choice experiment. We compared the human and the MO performance on a simple detection task of the calcification-like discs in ROIs with and without postprocessing. Proportion of correct responses of the human (PCH ) and NPWE (PCNPWE ) experiments was calculated and the correlation between the two was analyzed using a mixed-effect regression model. Correlation results including the goodness of fit (r2 ) of PCH and PCNPWE for all different parameters investigated were evaluated to determine whether NPWE MO can be used to predict human observer performance. PCH and PCNPWE increased with dose for all conditions investigated (signal size, processing status, and different system). In case of the 0.1 mm discs, for system A, r2 between PCH with PCNPWE was 0.81. For system B, r2 was 0.93. In case of the 0.25 mm discs, r2 in system A was 0.79 and for system B, r2 was 0.82. For the combined parameters investigated, and after excluding the 0.1 mm discs on system A because the results were influenced by aliasing, the overall r2 was 0.81. Image processing did not affect the detectability of calcification-like signals. No significant difference (P > 0.05) was found between the predicted PCH(pred) by the MO and the PCH for all different conditions. The framework seems promising to be used in objective image quality assessment. It was found to be relatively robust for the range of parameters investigated. However, further optimization of the anthropomorphic breast phantom and investigation of other MOs for a broader range of image quality assessment tasks is needed.

Blind image quality assessment via content-invariant statistical feature

A large number of blind image quality assessment (BIQA) methods take advantage of statistical features. Those statistical features-based methods are under the assumption that when the distortion damages the structures of images the statistical distributions of images will change. The traditional statistical features focus on the ‘changes’ caused by distortions, but few researches focus on that the image contents could disturb the statistical property for BIQA task. Here we aim to develop a robust image quality statistical feature named content-invariant statistical feature (CISF), which is sensitive to image quality but insensitive to image content, and hence could effectively reduce the disturbance caused by image content in image quality assessment task. To this end, we first convolute images with a set of multi-scales Laplacian-of-Gaussian (LOG) kernels to obtain a set of response maps and extract the curves of pixel sequence (CPS) from the response maps. Then, we calculate the fitting parameters of the CPSs with asymmetric Generalized Gaussian distribution and compute the statistical histograms of the fitting parameters as the CISF features. Finally, we use the developed CISF features and a simple support vector regression (SVR) model for BIQA. In the experiments, we evaluate our method on three large-scale benchmark databases and highly competitive performance is achieved compared with state-of-the-art BIQA models.

3D Visual Attention for Stereoscopic Image Quality Assessment

Depth perception is one of the most important characteristic in three-dimensional (3D) images different from traditional two-dimensional (2D) images. Therefore, 3D visual attention will be advantageous to improve 3D visual experience and particularly depth perception. In this paper, we propose a 3D visual attention model for stereoscopic image quality assessment task. The proposed model is constructed based on 2D saliency model, center bias, depth cue (foreground cue and background). Different combination and modulation means of the 3D visual attention model for quality assessment are investigated. The experimental results show that compared with other schemes, the proposed 3D visual attention-based pooling scheme can achieve higher consistency with the subjective assessment of stereoscopic images.

Learning to Predict Localized Distortions in Rendered Images

AbstractIn this work, we present an analysis of feature descriptors for objective image quality assessment. We explore a large space of possible features including components of existing image quality metrics as well as many traditional computer vision and statistical features. Additionally, we propose new features motivated by human perception and we analyze visual saliency maps acquired using an eye tracker in our user experiments. The discriminative power of the features is assessed by means of a machine learning framework revealing the importance of each feature for image quality assessment task. Furthermore, we propose a new data‐driven full‐reference image quality metric which outperforms current state‐of‐theart metrics. The metric was trained on subjective ground truth data combining two publicly available datasets. For the sake of completeness we create a new testing synthetic dataset including experimentally measured subjective distortion maps. Finally, using the same machine‐learning framework we optimize the parameters of popular existing metrics.

WE‐C‐110‐05: The Influence of Beam Filtration and Collimation on CT Number Accuracy in the ACR CT Accreditation Process

Purpose: The ACR accreditation process requires the submission of images of the 200 mm ACR phantom to evaluate CT number accuracy using the institutionˈs adult abdomen protocol. Difficulties in obtaining CT numbers that fall within the allowed limits may be due to the use of a protocol designed for a body significantly larger than the phantom. We show that adjusting the sFOV to a size appropriate for the phantom significantly shifts the CT numbers toward the expected values. We also show that consistent with the hypothesis that an over‐correction for scatter results in the increased CT numbers, a reduced collimation, where less scatter correction is required for reconstruction, also shifts the CT numbers in the right direction. Methods: The ACR Gammex phantom was scanned on a 64‐slice GE LightSpeed‐VCT scanner with various bowtie filters, beam collimation and sFOV using our institution`s routine adult abdomen protocol. Results: Our results show that the CT numbers for water and polyethylene fall well outside the required range if we rigidly adhere to scanning the ACR phantom with the standard protocol. The observed inaccuracy in reported CT numbers is believed to originate from over‐corrections due to scatter contributions and beam hardening effects as the phantom is significantly smaller than a typical adult abdomen. CT number accuracy was improved when a small bowtie filter was used instead of the large one. The CT number dropped to 4.02 HU for water and to −87.32 for polyethylene. CT number accuracy further improved when scanning the phantom with a 20 mm beam collimation instead of 40 mm. Conclusions: In performing the ACR image quality assessment task, the employment of a bowtie filter consistent with the size of the phantom resulted in a significant improvement in achieving the CT number accuracy requirements of the ACR.