Image Quality Assessment Framework Research Articles

A Soft-Reference Breast Ultrasound Image Quality Assessment Method That Considers the Local Lesion Area.

The quality of breast ultrasound images has a significant impact on the accuracy of disease diagnosis. Existing image quality assessment (IQA) methods usually use pixel-level feature statistical methods or end-to-end deep learning methods, which focus on the global image quality but ignore the image quality of the lesion region. However, in clinical practice, doctors' evaluation of ultrasound image quality relies more on the local area of the lesion, which determines the diagnostic value of ultrasound images. In this study, a global-local integrated IQA framework for breast ultrasound images was proposed to learn doctors' clinical evaluation standards. In this study, 1285 breast ultrasound images were collected and scored by experienced doctors. After being classified as either images with lesions or images without lesions, they were evaluated using soft-reference IQA or bilinear CNN IQA, respectively. Experiments showed that for ultrasound images with lesions, our proposed soft-reference IQA achieved PLCC 0.8418 with doctors' annotation, while the existing end-to-end deep learning method that did not consider the local lesion features only achieved PLCC 0.6606. Due to the accuracy improvement for the images with lesions, our proposed global-local integrated IQA framework had better performance in the IQA task than the existing end-to-end deep learning method, with PLCC improving from 0.8306 to 0.8851.

Automatic No-Reference kidney tissue whole slide image quality assessment based on composite fusion models

The visual quality of digitized whole slide images (WSIs) for kidney tissue not only affects the diagnosis and subsequent treatment, but also has a decisive impact on the accuracy of multiclass segmentation, classification and object detection during computer intelligent analysis. Currently, pathologists usually assess image quality through eye screening, which greatly relies on the pathologist experience and brings about subjectivity and non-repeatability issues. In this paper, we develop a no-reference image quality assessment framework including a fused CNN classification module, a quality score conversion module and a comprehensive quality prediction module, which automatically classifies WSIs of kidney tissue into four quality levels: excellent, good, average, and poor, and calculates a rough quality score. The original image and the regions of interest are combined and fused to comprehensively evaluate the quality of a WSI through multiple factors instead of a simple deep learning network. Extensive experiments conducted on our in-house dataset confirm that our proposed framework obtains satisfactory results with an accuracy of 90.05%, surpassing the performance of the typical image quality assessment models, and achieves the level of junior pathologist. Therefore, our proposed method can be embedded into a computer assisted diagnosis system to help pathologists in analysis of histopathological images and judgment of reliability of the images. The source code and trained models will be available at https://github.com/kidneyPathology/WSIQA.

No-Reference Framework for Image Quality Assessment

The technological advances in computer and communication devices have been an important factor in the appearance for the applications of visual communication. Image quality assessment is one of the pillars for these applications. It is a measure to assess the quality of images. There are many degradations in image quality that occur during the reproduction and transmission of the image. This paper aims to introduce a framework of No-Reference Image Quality Assessment. The framework provides a general estimation for three types of image distortions which are sharpness, blackness, and noisiness. Based on this framework, experiments have been conducted by the use of two datasets (LIVE database, CSIQ database). The results of the experiments have shown that we have introduced a more precise framework of No-Reference Image Quality Assessment.

Blind image quality assessment based on progressive multi-task learning

Due to the lack of adequate training data and sufficient mining of prior knowledge related to perceived quality, most existing image quality assessment (IQA) methods show limited generalization performance. In this paper, we study the prior knowledge from the factors affecting perceived quality, and introduce a novel progressive multi-task learning based blind IQA method. Inspired by the definition of IQA: human comprehensive perception for degradation of image content, we firstly deconstruct IQA into three elements, i.e., image content, pattern of degradation, and intensity of degradation. Based on these elements, we design the corresponding auxiliary tasks for instructing the network to learn IQA. By statistical analysis on a great deal of data, we find that there is progressive relevance among the four tasks. Furthermore, we mathematically derive that introducing the progressive relevance into a multi-task learning network can effectively constrain the hypothesis space of the main task. Under the guidance of the derivation, we propose an end-to-end IQA framework based on progressive multi-task learning. Experimental results demonstrate the excellent generalization capability of the proposed method, which achieves state-of-the-art performance against these existing IQA methods.

Performance Assessment Framework for Neural Network Denoising.

The proliferation of deep learning image processing calls for a quantitative image quality assessment framework that is suitable for nonlinear, data-dependent algorithms. In this work, we propose a method to systematically evaluate the system and noise responses such that the nonlinear transfer properties can be mapped out. The method involves sampling of lesion perturbations as a function of size, contrast, as well as clinically relevant features such as shape and texture that may be important for diagnosis. We embed the perturbations in backgrounds of varying attenuation levels, noise magnitude and correlation that are associated with different patient anatomies and imaging protocols. The range of system and noise response are further used to evaluate performance for clinical tasks such as signal detection and classification. We performed the assessment for an example CNN-denoising algorithm for low does lung CT screening. The system response of the CNN-denoising algorithm exhibits highly nonlinear behavior where both contrast and higher order lesion features such as spiculated boundaries are not reliably represented for lesions perturbations with small size and low contrast. The noise properties are potentially highly nonstationary, and should be assumed to be the same between the signal-present and signal-absent images. Furthermore, we observer a high degree dependency of both system and noise response on the background attenuation levels. Inputs around zeros are effectively imposed a non-negativity constraint; transfer properties for higher background levels are highly variable. For a detection task, CNN-denoised images improved detectability index by 16-18% compared to low dose CT inputs. For classification task between spiculated and smooth lesions, CNN-denoised images result in a much larger improvement up to 50%. The performance assessment framework propose in this work can systematically map out the nonlinear transfer functions for deep learning algorithms and can potentially enable robust deployment of such algorithms in medical imaging applications.

RM-IQA: A new no-reference image quality assessment framework based on range mapping method

Significant progress has been made in recent years in image quality assessment (IQA). In particular, the development of deep learning has provided no-reference (NR)-IQA with more impressive solutions. However, improving the generalization of NR-IQA models is still an urgent necessity. In this study, we propose a new framework that uses the range mapping method to map an existing full-reference (FR)-IQA dataset to an NR-IQA dataset, thereby further enhancing the accuracy and generalization of the NR-IQA model. First, an NR-IQA model is employed to score an FR-IQA dataset to obtain the corresponding mean opinion score (MOS) values. Then, the correlation coefficients between these MOS values and the original differential mean opinion score (DMOS) values marked by the FR-IQA dataset itself is calculated. Subsequently, the matching sequence pair is obtained according to these correlation coefficients. Then, a range mapping function is selected based on this sequence pair, and this function is used to map the entire FR-IQA dataset to the existing NR-IQA dataset, and a new NR-IQA dataset is generated. Finally, the new and the existing NR-IQA datasets are merged into a new dataset, which can train an end-to-end multi-task network to obtain the final model RM-IQA. This model exhibits better performance as it exploits more prior information. Based on the largest available NR-IQA dataset KonIQ-10k and FR-IQA dataset KADID-10K, the experimental results proved the effectiveness of the proposed framework.

Perceptual Image Quality Assessment for Various Viewing Conditions and Display Systems

From complete darkness to direct sunlight, real-world displays operate in various viewing conditions often resulting in a non-optimal viewing experience. Most existing Image Quality Assessment (IQA) methods, however, assume ideal environments and displays, and thus cannot be used when viewing conditions differ from the standard. In this paper, we investigate the influence of ambient illumination level and display luminance on human perception of image quality. We conduct a psychophysical study to collect a novel dataset of over 10000 image quality preference judgments performed in illumination conditions ranging from 0 lux to 20000 lux. We also propose a perceptual IQA framework that allows most existing image quality metrics (IQM) to accurately predict image quality for a wide range of illumination conditions and display parameters. Our analysis demonstrates strong correlation between human IQA and the predictions of our proposed framework combined with multiple prominent IQMs and across a wide range of luminance values.

Research on the parallelization of image quality analysis algorithm based on deep learning

Image quality assessment is an indispensable in computer vision applications, such as image classification, image parsing. With the development of Internet, image data acquisition becomes more conveniently. However, image distortion is inevitable due to imperfect image acquisition system, image transmission medium and image recording equipment. Traditional image quality assessment algorithms only focus on low-level visual features such as color or texture, which could not encode high-level features effectively. CNN-based methods have shown satisfactory results in image quality assessment. However, existing methods have problems such as incomplete feature extraction, partial image block distortion, and inability to determine scores. So in this paper, we propose a novel framework for image quality assessment based on deep learning. We incorporate both low-level visual features and high-level semantic features to better describe images. And image quality is analyzed in a parallel processing mode. Experiments are conducted on LIVE and TID2008 datasets demonstrate the proposed model can predict the quality of the distorted image well, and both SROCC and PLCC can reach 0.92 or higher.

Free- Reference Image Quality Assessment Framework Using Metrics Fusion and Dimensionality Reduction

Free- Reference Image Quality Assessment Framework Using Metrics Fusion and Dimensionality Reduction

Free- Reference Image Quality Assessment Framework Using Metrics Fusion and Dimensionality Reduction

This paper focuses on no-reference image quality assessment(NR-IQA)metrics. In the literature, a wide range of algorithms are proposed to automatically estimate the perceived quality of visual data. However, most of them are not able to effectively quantify the various degradations and artifacts that the image may undergo. Thus, merging of diverse metrics operating in different information domains is hoped to yield better performances, which is the main theme of the proposed work. In particular, the metric proposed in this paper is based on three well-known NR-IQA objective metrics that depend on natural scene statistical attributes from three different domains to extract a vector of image features. Then, Singular Value Decomposition (SVD) based dominant eigenvectors method is used to select the most relevant image quality attributes. These latter are used as input to Relevance Vector Machine (RVM) to derive the overall quality index. Validation experiments are divided into two groups; in the first group, learning process (training and test phases) is applied on one single image quality database whereas in the second group of simulations, training and test phases are separated on two distinct datasets. Obtained results demonstrate that the proposed metric performs very well in terms of correlation, monotonicity and accuracy in both the two scenarios.

Multimedia image quality assessment based on deep feature extraction

Measurement of visual quality is of significant importance to many image processing tasks. The target of image quality assessment (IQA) is to design effective computational models in order to automatically predict the quality of images in a perceptual consistent manner. We propose a full reference (FR) IQA metric based on deep convolutional neural networks and information-theoretic IQA framework. The previous proposed PAVIF is incorporated into the powerful convolutional network VGG19. Both the reference and distorted image are fed into the VGG19, and the output of each channels in the first 35 layers are utilized to measure the perceptual quality difference. The final objective score is obtained by averaging all the channel-wise quality scores. Experimental results on TID2013 and LIVE image database demonstrate that our proposed metric is competitive with many state-of-the-art IQA metrics.

Blind Quality Assessment Based on Pseudo-Reference Image

Traditional full-reference image quality assessment (IQA) metrics generally predict the quality of the distorted image by measuring its deviation from a perfect quality image called reference image. When the reference image is not fully available, the reduced-reference and no-reference IQA metrics may still be able to derive some characteristics of the perfect quality images, and then measure the distorted image's deviation from these characteristics. In this paper, contrary to the conventional IQA metrics, we utilize a new “reference” called pseudo-reference image (PRI) and a PRI-based blind IQA (BIQA) framework. Different from a traditional reference image, which is assumed to have a perfect quality, PRI is generated from the distorted image and is assumed to suffer from the severest distortion for a given application. Based on the PRI-based BIQA framework, we develop distortion-specific metrics to estimate blockiness, sharpness, and noisiness. The PRI-based metrics calculate the similarity between the distorted image's and the PRI's structures. An image suffering from severer distortion has a higher degree of similarity with the corresponding PRI. Through a two-stage quality regression after a distortion identification framework, we then integrate the PRI-based distortion-specific metrics into a general-purpose BIQA method named blind PRI-based (BPRI) metric. The BPRI metric is opinion-unaware (OU) and almost training-free except for the distortion identification process. Comparative studies on five large IQA databases show that the proposed BPRI model is comparable to the state-of-the-art opinion-aware- and OU-BIQA models. Furthermore, BPRI not only performs well on natural scene images, but also is applicable to screen content images. The MATLAB source code of BPRI and other PRI-based distortion-specific metrics will be publicly available.

A machine-learning framework for automatic reference-free quality assessment in MRI

Magnetic resonance (MR) imaging offers a wide variety of imaging techniques. A large amount of data is created per examination which needs to be checked for sufficient quality in order to derive a meaningful diagnosis. This is a manual process and therefore time- and cost-intensive. Any imaging artifacts originating from scanner hardware, signal processing or induced by the patient may reduce the image quality and complicate the diagnosis or any image post-processing. Therefore, the assessment or the ensurance of sufficient image quality in an automated manner is of high interest. Usually no reference image is available or difficult to define. Therefore, classical reference-based approaches are not applicable. Model observers mimicking the human observers (HO) can assist in this task. Thus, we propose a new machine-learning-based reference-free MR image quality assessment framework which is trained on HO-derived labels to assess MR image quality immediately after each acquisition. We include the concept of active learning and present an efficient blinded reading platform to reduce the effort in the HO labeling procedure. Derived image features and the applied classifiers (support-vector-machine, deep neural network) are investigated for a cohort of 250 patients. The MR image quality assessment framework can achieve a high test accuracy of 93.7% for estimating quality classes on a 5-point Likert-scale. The proposed MR image quality assessment framework is able to provide an accurate and efficient quality estimation which can be used as a prospective quality assurance including automatic acquisition adaptation or guided MR scanner operation, and/or as a retrospective quality assessment including support of diagnostic decisions or quality control in cohort studies.

NO-REFERENCE IMAGE QUALITY MEASURE FOR IMAGES WITH MULTIPLE DISTORTIONS USING RANDOM FORESTS FOR MULTI METHOD FUSION

Over the years image quality assessment is one of the active area of research in image processing. Distortion in images can be caused by various sources like noise, blur, transmission channel errors, compression artifacts etc. Image distortions can occur during the image acquisition process (blur/noise), image compression (ringing and blocking artifacts) or during the transmission process. A single image can be distorted by multiple sources and assessing quality of such images is an extremely challenging task. The human visual system can easily identify image quality in such cases, but for a computer algorithm performing the task of quality assessment is a very difficult. In this paper, we propose a new no-reference image quality assessment for images corrupted by more than one type of distortions. The proposed technique is compared with the best-known framework for image quality assessment for multiply distorted images and standard state of the art Full reference and No-reference image quality assessment techniques available.

Blind Image Quality Estimation via Distortion Aggravation

Traditional blind image quality assessment (IQA) measures generally predict quality from a sole distorted image directly. In this paper, we first introduce multiple pseudo reference images (MPRIs) by further degrading the distorted image in several ways and to certain degrees, and then compare the similarities between the distorted image and the MPRIs. Via such distortion aggravation, we can have some references to compare with, i.e., the MPRIs, and utilize the full-reference IQA framework to compute the quality. Specifically, we apply four types and five levels of distortion aggravation to deal with the commonly encountered distortions. Local binary pattern features are extracted to describe the similarities between the distorted image and the MPRIs. The similarity scores are then utilized to estimate the overall quality. More similar to a specific pseudo reference image (PRI) indicates closer quality to this PRI. Owning to the availability of the created multiple PRIs, we can reduce the influence of image content, and infer the image quality more accurately and consistently. Validation is conducted on four mainstream natural scene image and screen content image quality assessment databases, and the proposed method is comparable to or outperforms the state-of-the-art blind IQA measures. The MATLAB source code of the proposed measure will be publicly available.

Visual Importance and Distortion Guided Deep Image Quality Assessment Framework

In this paper, we tackle the problem of no-reference image quality assessment (IQA). A learning-based IQA framework “VIDGIQA” is proposed, which extracts quality features from the input image and regresses the visual quality on these features. Since different distortions lead to different visual perceptions in the human visual system, distortion information is adopted to guide the feature learning process together with the human quality scores. Besides, a regression method is proposed to model and estimate the visual importance weights of all local regions, which can effectively improve the performance. More importantly, all these operations are integrated into one deep neural network, so that they can be jointly optimized and well cooperate with each other. Experiments were conducted to demonstrate the power of the proposed method on several datasets, including the LIVE dataset [1], the TID 2013 dataset [2], the LIVE multiply distorted IQA dataset [3], CSIQ [4] , and the LIVE in the wild image quality database [5]. The proposed method achieves 0.969 and 0.973 on the LIVE dataset [1] in terms of the spearman rank-order correlation coefficient and the Pearson linear correlation coefficient, respectively, which outperforms the state-of-the-art methods.

DeepSim: Deep similarity for image quality assessment

This paper studies one interesting problem: how does the deep neural network (DNN) architecture affect the image quality assessment (IQA) performance? In order to find the answer, we propose a novel full-reference IQA framework, codenamed deep similarity (DeepSim). In DeepSim, we first measure the local similarities between the features (produced by a DNN model) of the test image and those of the reference image; Afterwards, the local quality indices are gradually pooled together to estimate the overall quality score. In addition, various factors that may affect the IQA performance are investigated. Thorough experiments conducted on standard databases show that: (1) DeepSim can accurately predict human perceived image quality and outperforms previous state-of-the-art; (2) mid-level representations are most effective for quality prediction; and (3) preprocessing, the restricted linear units and max-pooling operations are beneficial for the IQA performance.

Generalized referenceless image quality assessment framework using texture energy measures and pattern strength features

Referenceless image quality assessment is a challenging and critical problem in today's multimedia applica\-tions. Texture patterns in images are normally at high frequencies compared to lower ones. Due to the effect of distortions during acquisition, compression, and transmission, texture deviation artifacts are generated that cause a granular effect in the image. Other artifacts, such as blocking, affect high frequencies in an image, causing distorted edges. Combining the analysis of texture deviation and other artifacts helps in determining the quality of an image. The proposed approach uses variation in the energy of pixels to quantify the quality of an image. These variations are calculated using texture energy measures and pattern strength-based statistical features. In the proposed approach, machine learning-based classifiers are used to predict the quality score for an image. The performance of the proposed method is tested for all images ranging from pristine to poor quality from LIVE and TID2008 databases. For different distortions, results are shown to have good correlation if they lie between the predicted score and the differential mean opinion score. Results obtained with this approach are compared with other widely used referenceless approaches. It is observed that the proposed approach shows better performance in the quantification of the quality of an image.

Similar Reference Image Quality Assessment: A New Database and A Trial with Local Feature Matching

Conventionally, the reference image for image quality assessment (IQA) is completely available (full-reference IQA) or unavailable (no-reference IQA). Even for reduced-reference IQA, the features that are used to predict image quality are still extracted from the pristine reference image. However, the pristine reference image is always unavailable in many real scenarios. In contrast, it is convenient to obtain a number of similar reference images via retrieval from the Internet. These similar reference images may share similar contents and scenes with the image to be assessed. In this paper, we attempt to discuss the image quality assessment problem from the view of similar images, i.e. similar reference IQA. Although the similar reference images share similar contents with the degraded image, the difference between them still cannot be ignored. Therefore, we propose an IQA framework based on local feature matching, which can help to identify the similar regions and structures. Then the IQA features are computed only from these similar regions to predict the final image quality score. Besides, since there is no IQA databases for the similar reference IQA problem, we establish a novel IQA database that consists of 272 images from four scenes. The experiments demonstrate that the performance of our scheme goes beyond state-of-the-art no-reference IQA methods and some full-reference IQA algorithms.

Quantifying blur in colour images using higher order singular values

This Letter introduces a novel framework for blind blur assessment in colour images using higher order singular values. The RGB colour image is seen as a third-order tensor to exploit the spatial and inter-channel correlations, so that blurring effects are captured more robustly. The tensor is decomposed into different two-dimensional matrices, also called unfoldings. The conventional singular value decomposition is carried out for these unfoldings instead of computing it for the luminance component alone. The experiments were performed on several publicly available databases and the results validate the superiority of the proposed metric among different state-of-the-art blind blur assessment metrics. The proposed framework for image quality assessment (IQA) from colour images fits well with the current trends and research efforts put in enhancing the quality of experience for different multimedia applications and in benchmarking new imaging and sensing technologies including camera and other vision systems with IQA capabilities.