Image Quality Assessment Algorithms Research Articles

A large-scale image database for benchmarking mobile camera quality and NR-IQA algorithms

Camera-oriented quality assessment (CQA) differs from traditional image quality assessment (IQA) in that “distorted” images are straight out of real devices instead of various types and levels of artificial operations. However, despite its value for both customers and manufacturers, academic and industrial fields, there are few CQA databases created years ago. To reflect recent mobile imaging advancements, we present a new massive Phone Camera Benchmarking database (PCB2021) in this paper. In PCB2021, 40 modern phone units featuring photography are simultaneously compared in 182 scenes for a total of 7280 images, which can be classified into six categories (sub-datasets) based on different focal lengths and user cases: main camera (107), ultra-wide (20), 2×, 3×, 5× zoom lenses (26, 8, 10) and night-mode (11). The shooting process begins from 7:00 am to 11:00 pm and lasts more than a month. In the subjective study, to overcome the high-resolution-induced overall quality evaluation difficulty, five image attributes: exposure/contrast, color, sharpness, graininess, artifacts are assessed separately on each dataset. To reduce ranking complexity for large-scale cameras, a dynamic anchor ruler method is proposed to obtain quality orders efficiently. With the constructed PCB2021, we further evaluate 15 mainstream no-reference (NR) IQA algorithms. The finding is that for zoomed images, sharpness metrics can achieve Spearman correlation coefficients above 0.8, while for the subtle main camera and night-mode images, performances of all fifteen algorithms drop down quickly, i.e. 0.1-0.2 for the former and ∼0.5 for the latter. The entire database, expert rankings and algorithm performance reports will be freely available on request.

Image Quality Assessment for Magnetic Resonance Imaging

Image quality assessment (IQA) algorithms aim to reproduce the human's perception of the image quality. The growing popularity of image enhancement, generation, and recovery models instigated the development of many methods to assess their performance. However, most IQA solutions are designed to predict image quality in the general domain, with the applicability to specific areas, such as medical imaging, remaining questionable. Moreover, the selection of these IQA metrics for a specific task typically involves intentionally induced distortions, such as manually added noise or artificial blurring; yet, the chosen metrics are then used to judge the output of real-life computer vision models. In this work, we aspire to fill these gaps by carrying out the most extensive IQA evaluation study for Magnetic Resonance Imaging (MRI) to date (14,700 subjective scores). We use outputs of neural network models trained to solve problems relevant to MRI, including image reconstruction in the scan acceleration, motion correction, and denoising. Our emphasis is on reflecting the radiologist's perception of the reconstructed images, gauging the most diagnostically influential criteria for the quality of MRI scans: signal-to-noise ratio, contrast-to-noise ratio, and the presence of artifacts. Seven trained radiologists assess these distorted images, with their verdicts then correlated with 35 different image quality metrics (full-reference, no-reference, and distribution-based metrics considered). The top performers -- DISTS, HaarPSI, VSI, and FID-VGG16 -- are found to be efficient across three proposed quality criteria, for all considered anatomies and the target tasks.

No-Reference Image Quality Assessment Algorithm Based on Semi-Supervised Learning

No-Reference Image Quality Assessment Algorithm Based on Semi-Supervised Learning

A no-reference perceptual image quality assessment database for learned image codecs

The drastic growth of research in image compression, especially deep learning-based image compression techniques, poses new challenges to objective image quality assessment (IQA). Typical artifacts encountered in the emerging image codecs are significantly different from that produced by traditional block-based codecs, leading to inapplicability of the existing objective IQA algorithms. Towards advancing the development of objective IQA algorithms for recent compression artifacts, we built a learning-based compressed image quality assessment (LCIQA) database involving traditional block-based image codecs, hybrid neural network based image codecs, convolutional neural network based and generative adversarial network (GAN) based end-to-end optimized image coding approaches. Our study confirms the statistical difference and human perception difference between reconstructions of learned compression and traditional block-based compression. We propose a two-step deep learning model for learning-based compressed image quality assessment. Extensive experiments on LCIQA database demonstrate that our proposed model performs better than other counterparts on learning-based compressed images, especially on GAN compressed images, and achieves competitive performance to the state-of-the-art IQA metrics on traditional compressed images.

Jomodevi: A joint motion and depth visibility prediction algorithm for perceived stereoscopic 3D quality

In this paper, we introduce a completely blind and unsupervised no-reference model that performs the quality assessment of stereoscopic 3D videos by using joint motion and depth visibility. In order to achieve such objective metric, first we compute the correlation map between the motion vector map and the depth map, and then we study the natural scene statistics of the estimated correlation map. We empirically model these statistics with Univariate Generalized Gaussian Distribution (UGGD) and compute the UGGD parameters at multi-scale and multi-orient steerable subband decomposition. It is shown that the estimated UGGD features are capable of discriminating distortions. We then compute the mean vector and the covariance matrix from the estimated UGGD features of the correlation maps of motion vector and the depth maps of undistorted and distorted S3D videos. This is followed by the computation of the Wave Hedges distance between the mean vectors of the undistorted and distorted contents, and the Bhattacharyya distance is measured between the corresponding covariance matrices. We pool these distance measures to achieve the overall joint motion and depth quality of S3D videos. The efficiency of the proposed algorithm is evaluated on the well-known IRCCYN, LFOVIAS3DPh1 and LFOVIAS3DPh2 S3D video datasets. Our model demonstrates robust and consistent performance across all distortion types and shows competitive performance against other 2D and 3D image and video quality assessment algorithms. Furthermore, since the proposed algorithm is completely blind, it does not require any training and testing analysis on the content features and the subjective scores.

A novel algorithm for comprehensive quality assessment of clinical magnetic resonance images based on natural scene statistics in spatial domain

BackgroundA magnetic resonance imaging (MRI)-specific objective image quality assessment (IQA) algorithm, the quality evaluation using multidirectional filters for MRI (QEMDIM), was previously reported. QEMDIM requires a set of reference images to calculate the quality score (SQ) for an assessed image. SQ may be affected by the quality of the reference set owing to the calculation procedure. PurposeTo propose a modified version of the IQA algorithm and compare the IQA performance of the original and modified algorithms. AssessmentBrain axial T1- and T2-weighted spin-echo images of varying quality levels (noise and blurring) were acquired from seven healthy men. Subjective IQA (paired comparisons) was conducted on the images, and subjective quality scores were obtained. With reference sets of various quality levels, QEMDIM and modified IQA were applied to the same images that underwent the subjective IQA. The correlation of each SQ and modified score (Smod) with the subjective scores was evaluated for content-related subsets of assessed images and for each reference set. The effect of the reference-set quality on the distribution of the correlation coefficients (CCs) was statistically evaluated for SQ and Smod using a one-way analysis of variance test with a significance level of 0.05. We also evaluated the variation in Smod for images with almost the same qualities using the standard deviation (SD). ResultsThe CCs of SQ varied significantly with the quality of the reference set, whereas that of Smod did not. The SD of Smod for almost-same-quality images was less than that corresponding to the confidence interval of the subjective scores. ConclusionOur modified algorithm was superior to QEMDIM in terms of IQA performance in clinical practice, especially in terms of accuracy, robustness, and reproducibility.

Saliency-Guided Local Full-Reference Image Quality Assessment

Research and development of image quality assessment (IQA) algorithms have been in the focus of the computer vision and image processing community for decades. The intent of IQA methods is to estimate the perceptual quality of digital images correlating as high as possible with human judgements. Full-reference image quality assessment algorithms, which have full access to the distortion-free images, usually contain two phases: local image quality estimation and pooling. Previous works have utilized visual saliency in the final pooling stage. In addition to this, visual saliency was utilized as weights in the weighted averaging of local image quality scores, emphasizing image regions that are salient to human observers. In contrast to this common practice, visual saliency is applied in the computation of local image quality in this study, based on the observation that local image quality is determined both by local image degradation and visual saliency simultaneously. Experimental results on KADID-10k, TID2013, TID2008, and CSIQ have shown that the proposed method was able to improve the state-of-the-art’s performance at low computational costs.

Super Resolution Image Visual Quality Assessment Based on Feature Optimization.

Most existing no-referenced image quality assessment (NR-IQA) algorithms need to extract features first and then predict image quality. However, only a small number of features work in the model, and the rest will degrade the model performance. Consequently, an NR-IQA framework based on feature optimization is proposed to solve this problem and apply to the SR-IQA field. In this study, we designed a feature engineering method to solve this problem. Specifically, the features associate with the SR images were first collected and aggregated. Furthermore, several advanced feature selection algorithms were used to sort the feature sets according to their importance, and the importance matrix of features is obtained. Then, we examined the linear relationship between the number of features and Pearson linear correlation coefficient (PLCC) to determine the optimal number of features and the optimal feature selection algorithm, so as to obtain the optimal model. The results showed that the image quality scores predicted by the optimal model are in good agreement with the human subjective scores. Adopting the proposed feature optimization framework, we can effectively reduce the number of features in the model and obtain better performance. The experimental results indicated that SR image quality can be accurately predicted using only a small part of image features. In summary, we proposed a feature optimization framework to solve the current problem of irrelevant features in SR-IQA, and an SR image quality assessment model was proposed consequently.

No-Reference Quality Assessment of Authentically Distorted Images Based on Local and Global Features.

With the development of digital imaging techniques, image quality assessment methods are receiving more attention in the literature. Since distortion-free versions of camera images in many practical, everyday applications are not available, the need for effective no-reference image quality assessment algorithms is growing. Therefore, this paper introduces a novel no-reference image quality assessment algorithm for the objective evaluation of authentically distorted images. Specifically, we apply a broad spectrum of local and global feature vectors to characterize the variety of authentic distortions. Among the employed local features, the statistics of popular local feature descriptors, such as SURF, FAST, BRISK, or KAZE, are proposed for NR-IQA; other features are also introduced to boost the performances of local features. The proposed method was compared to 12 other state-of-the-art algorithms on popular and accepted benchmark datasets containing RGB images with authentic distortions (CLIVE, KonIQ-10k, and SPAQ). The introduced algorithm significantly outperforms the state-of-the-art in terms of correlation with human perceptual quality ratings.

A Underwater Sequence Image Dataset for Sharpness and Color Analysis.

The complex underwater environment usually leads to the problem of quality degradation in underwater images, and the distortion of sharpness and color are the main factors to the quality of underwater images. The paper discloses an underwater sequence image dataset called TankImage-I with gradually changing sharpness and color distortion collected in a pool. TankImage-I contains two plane targets, a total of 78 images. It includes two lighting conditions and three different water transparency. The imaging distance is also changed during the photographing process. The paper introduces the relevant details of the photographing process, and provides the measurement results of the sharpness and color distortion of the sequence images. In addition, we verify the performance of 14 image quality assessment methods on TankImage-I, and analyze the results of 14 image quality assessment methods from the aspects of sharpness and color, which provides a reference for the design and improvement of underwater image quality assessment algorithm and underwater imaging system design.

Evaluation of Virtual Grid Processed Clinical Chest Radiographs.

We evaluated the different Virtual Grid software ratios (Fujifilm, Tokyo, Japan) on gridless clinical chest radiographs with visual grading analysis (VGA). In addition, we investigated the 2 image quality assessment algorithms (IQAAs). Gridless chest radiographs of 50 different intensive care unit patients were collected and afterward processed with Virtual Grid software. Different software (SW) grid ratios-6:1, 10:1, 13:1, 17:1, and 20:1-were applied to investigate the image quality (IQ) improvement. Image quality improvement was assessed by 4 radiologists in a relative VGA study where the reference image was processed with SW grid ratio of 10:1. One of the IQAAs used to analyze the radiographs was implemented in our department but was originally developed by the research group of the Duke University Medical Center. A general IQ score (IQS) was calculated based on contrast, detail, and noise. Another IQAA-NIQE (naturalness image quality evaluator)-available in Matlab (MATLAB Research R2019b; the MathWorks, Inc) was evaluated. Both methods were compared with VGA. Visual grading analysis scores of gridless radiographs are significantly lower ( P < 0.001). Image quality increases with increasing SW grid ratios, up to grid ratio of 17:1. However, some anatomical structures-spine and ribs-are negatively affected by the higher grid ratios. A correlation coefficient of 0.99 between the VGA and the IQS was observed. The correlation coefficient between VGA and NIQE was 1.00. Virtual Grid with SW grid ratio of 6:1 improves the IQ of gridless chest bedside radiographs. The grid ratios 17:1 and 20:1 should be considered carefully as the SW negatively affects parts of the ribs and spine. Therefore, grid ratios up to 13:1 can be advised. The IQAAs are promising and could be used to detect differences in IQ when different scatter correction SW settings are used.

No-Reference Quality Assessment for 360-Degree Images by Analysis of Multifrequency Information and Local-Global Naturalness

360-degree/omnidirectional images (OIs) have received remarkable attention due to the increasing applications of virtual reality (VR). Compared to conventional 2D images, OIs can provide more immersive experiences to consumers, benefiting from the higher resolution and plentiful field of views (FoVs). Moreover, observing OIs is usually in a head-mounted display (HMD) without references. Therefore, an efficient blind quality assessment method, which is specifically designed for 360-degree images, is urgently desired. In this paper, motivated by the characteristics of the human visual system (HVS) and the viewing process of VR visual content, we propose a novel and effective no-reference omnidirectional image quality assessment (NR OIQA) algorithm by MultiFrequency Information and Local-Global Naturalness (MFILGN). Specifically, inspired by the frequency-dependent property of the visual cortex, we first decompose the projected equirectangular projection (ERP) maps into wavelet subbands by using discrete Haar wavelet transform (DHWT). Then, the entropy intensities of low-frequency and high-frequency subbands are exploited to measure the multifrequency information of OIs. In addition to considering the global naturalness of ERP maps, owing to the browsed FoVs, we extract the natural scene statistics (NSS) features from each viewport image as the measure of local naturalness. With the proposed multifrequency information measurement and local-global naturalness measurement, we utilize support vector regression (SVR) as the final image quality regressor to train the quality evaluation model from visual quality-related features to human ratings. To our knowledge, the proposed model is the first no-reference quality assessment method for 360-degree images that combines multifrequency information and image naturalness. Experimental results on two publicly available OIQA databases demonstrate that our proposed MFILGN outperforms state-of-the-art full-reference (FR) and NR approaches.

No-reference panoramic image quality assessment based on multi-region adjacent pixels correlation.

The distortion measurement plays an important role in panoramic image processing. Most measurement algorithms judge the panoramic image quality by means of weighting the quality of the local areas. However, such a calculation fails to globally reflect the quality of the panoramic image. Therefore, the multi-region adjacent pixels correlation (MRAPC) is proposed as the efficient feature for no-reference panoramic images quality assessment in this paper. Specifically, from the perspective of the statistical characteristics, the differences of the adjacent pixels in panoramic image are proved to be highly related to the degree of distortion and independent of image content. Besides, the difference map has limited pixel value range, which can improve the efficiency of quality assessment. Based on these advantages, the proposed MRAPC feature collaborates with the support vector regression to globally predict the quality of panoramic images. Extensive experimental results show that the proposed no-reference panoramic image quality assessment algorithm achieves higher evaluation performance than the existing algorithms.

The CTSimU software toolbox for CT-related image processing and quality assessment

A software toolbox is introduced that addresses several needs common to computed tomography (CT). Built for the WIPANO CTSimU project [1] to serve as the reference implementation for its image processing and evaluation tasks, it provides a Python 3 interface that is adaptable to many conceivable applications. Foremost, the toolbox features a pipeline architecture for sequential 2D image processing tasks, such as flat field corrections and image binning, and enables the user to create their own processing modules. Beyond that, it provides means to measure line profiles and image quality assessment algorithms to calculate modulation transfer functions (MTF) or to determine the interpolated basic spatial resolution (iSRb) using a duplex wire image [2]. It can also be used to calculate projection matrices for the reconstruction of scans with arbitrary industrial CT geometries and trajectories. The CTSimU project defined a framework of projection- and volume-based test scenarios [3] for the qualification of radiographic simulation software towards its use in dimensional metrology. The toolbox implements the necessary evaluation routines and generates reports for all projection-based tests.

Characterizing Scatter Correction Software of 5 Mobile Radiography Units

Bedside radiographs are usually obtained gridless, without a physical scatter correction grid because of several limitations. Therefore, multiple manufacturers of mobile radiography systems provide the possibility to apply scatter correction software (SC SW) on those images. The purpose of this study was to characterize different series of radiographs-gridless, SC SW, and physical grid-with an image quality assessment algorithm (IQAA). Furthermore, we investigated the potential dose reduction and the correlation between the output of the IQAA and the human observers. We obtained different series of radiographs with an anthropomorphic phantom (multipurpose chest phantom N1 "Lungman," Kyoto Kagaku, Kyoto, Japan). All radiographs were obtained with flat-panel detectors of 5 different manufacturers in a wall bucky system. An IQAA to analyze the radiographs was implemented in our department but was originally developed by the research group of the Duke University Medical Center. Seven physical quantities were calculated by the IQAA: rib-lung contrast (RLcontrast), subdiaphragm-lung contrast (SLcontrast), lung detail (Ldetail), mediastinum detail (Mdetail), lung noise (Lnoise), mediastinum noise (Mnoise), and rib-lung sharpness (RLsharpness). In a proof of concept, the results of the IQAA were validated by 3 experienced radiologists. Regression coefficients (b) of the linear regression model indicate that the human observer results correlate well with the IQAA (b ≥ 0.89, R2 ≥ 0.83). All manufacturers have SC SW that increases the 7 physical quantities of the gridless images. However, several manufacturers have SC SW that increases the physical metrics to the same level as the physical grid images. The SC SW radiographs obtained with a reduced tube load have an increased level of contrast, detail, sharpness, and noise compared with the gridless images obtained with the higher tube load. We have proven in a proof of concept that the originally developed IQAA can be used to characterize different series of images of different manufacturers. Based on the physical quantities, SC SW increases the contrast, detail, sharpness, and noise. The experimental results in this study assume a patient dose reduction could be possible when SC SW is applied.

An improved blind/referenceless image spatial quality evaluator algorithm for image quality assessment

An improved blind/referenceless image spatial quality evaluator algorithm for image quality assessment

No Reference Pansharpened Image Quality Assessment Through Deep Feature Similarity

Pansharpening refers to the process of enhancing the spatial resolution of a multispectral image with the help of a high spatial resolution panchromatic image. Quality assessment (QA) of pansharpened images helps provide a formal framework for the analysis and design of pansharpening methods and is thus extremely important. However, lack of availability of a reference multispectral image makes QA of pansharpening algorithms a challenging task. Given the popular use of QA algorithms that use a reference, this paper focuses on predicting the quality under a &#x2018;no-reference&#x0027; (NR) setting. Specifically, a learning based NR pansharpened image quality assessment (IQA) approach is adopted to predict state of the art reference based measures such as <inline-formula><tex-math notation="LaTeX">$Q2^{n}$</tex-math></inline-formula> and SAM without the need of a reference. We design an end-to-end deep pansharpening IQA network to compute the similarity of deep features fused from the panchromatic and input low resolution multispectral with similar features extracted from the given pansharpened image. To train and test our learning based approach, we create a large corpus of pansharpened images belonging to different satellites and thematic scenes by applying different pansharpening algorithms. Our experiments demonstrate that our NR pansharpened IQA algorithm achieves excellent performance and generalizes well across different satellites and resolutions. The source code and the dataset will be made available at <uri>https://github.com/neerajbadal/DPIQA</uri>.

A Display-Independent Quality Assessment for HDR Images

High Dynamic Range (HDR) images improve visual quality by representing a wide range of luminance in real world. HDR Image Quality Assessment (IQA) is essential in HDR processing technologies. This work focus on the IQA for the HDR applications represented by HDR compression. Towards designing IQA for these tasks, user-end display information should not be considered, given that these tasks only focus on the media intrinsic features. However, most existing HDR metrics are associated with display luminance, hence they can not handle IQA during HDR processing. In this work, we find that it is possible to evaluate HDR image quality without prior knowledge of luminance for display. Based on this, we proposed a low complexity Display-Independent Gradient Magnitude Similarity (DIGMS) metric. Experimental results demonstrate that the proposed DIGMS outperforms the state-of-the-arts in HDR IQA. This work opens up a new perspective for designing IQA algorithms for HDR images.

Algorithm improvement of neural network in endoscopic image recognition of upper digestive tract system

AbstractThe development of effective gastrointestinal diseases computer‐aided diagnosis tools and automatic image quality assessment algorithms is very important to improve the effectiveness of diagnosis and treatment. In order to further study the application of neural network algorithm in the endoscopic image of upper digestive tract, improve the efficiency of neural network algorithm in the field of endoscopic image. In this study, neural network algorithms were used to identify endoscopic images of the upper digestive tract. 1335 cases with upper gastrointestinal endoscopic images were collected. After the data was enlarged, it was randomly divided into training set and test set according to the proportion, and the obtained training set was input. After convolutional neural network training, an algorithm model was established in the institute. 1653 test set data samples were input into the neural network to verify the accuracy. Finally, the accuracy of the neural root network model constructed in this study reached 0.0942. Through horizontal comparison, it can be concluded that the neural network model proposed in this study not only has a higher accuracy rate, but also is better than the current existing related neural network algorithms. Based on the above experimental verification, it can be concluded that the upper gastrointestinal endoscopic image recognition algorithm based on neural network proposed in this study can more accurately and effectively identify the lesions in the upper gastrointestinal endoscopic images.

A technical review of no-reference image quality assessment algorithms for contrast distorted images

Automatic image quality assessment similar to human vision perception is an essential process for real-time image processing applications to perform perceptual image assessments for effectively achieving their goals. As no-reference image quality assessment (NR-IQA) schemes perform perceptual assessments of images without any information about their original version, these algorithms suit real-time computer vision techniques because of the non-availability of reference images. Contrast and colorfulness play important roles in determining the quality of color images. By combining many IQA metrics, a number of combined metrics had been devised. This study provides an insight into major NR-IQA methods and their effectiveness in assessing contrast, colorfulness, and overall quality of contrast-degraded images with technical analysis. The effectiveness of top-ranking NR-IQA methods is experimentally assessed with benchmark assessment methods on images from benchmarked databases. The study provides insight into open research challenges in the area of NR-IQA for developing new promising methods by clearly demarcating the difficulties of top-ranking NR-IQA methods.