Differential Mean Opinion Score Research Articles

Subjective Panoramic Video Quality Assessment Database for Coding Applications

With the development of virtual reality, higher quality panoramic videos are in great demand to guarantee the immersive viewing experience. Therefore, quality assessment attaches much importance to correlated technologies. Considering the geometric transformation in projection and the limited resolution of head-mounted device (HMD), a modified display protocol of the high resolution sequences for the subjective rating test is proposed, in which an optimal display resolution is determined based on the geometry constraints between screen and human eyes. By sampling the videos to the optimal resolution before coding, the proposed method significantly alleviates the interference of HMD sampling while displaying, thus ensuring the reliability of subjective quality opinion in terms of video coding. Using the proposed display protocol, a subjective quality database for panoramic videos is established for video coding applications. The proposed database contains 50 distorted sequences obtained from ten raw panoramic video sequences. Distortions are introduced with the High Efficiency Video Coding compression. Each sequence is evaluated by 30 subjects on video quality, following the absolute category rating with hidden reference method. The rating scores and differential mean opinion scores (DMOSs) are recorded and included in the database. With the proposed database, several state-of-the-art objective quality assessment methods are further evaluated with correlation analysis. The database, including the video sequences, subjective rating scores and DMOS, can be used to facilitate future researches on coding applications.

Noise Estimation and Quality Assessment of Gaussian Noise Corrupted Images

Evaluating the exact quantity of noise present in an image and quality of an image in the absence of reference image is a challenging task. We propose a near perfect noise estimation method and a no reference image quality assessment method for images corrupted by Gaussian noise. The proposed methods obtain initial estimate of noise standard deviation present in an image using the median of wavelet transform coefficients and then obtains a near to exact estimate using curve fitting. The proposed noise estimation method provides the estimate of noise within average error of +/-4%. For quality assessment, this noise estimate is mapped to fit the Differential Mean Opinion Score (DMOS) using a nonlinear function. The proposed methods require minimum training and yields the noise estimate and image quality score. Images from Laboratory for image and Video Processing (LIVE) database and Computational Perception and Image Quality (CSIQ) database are used for validation of the proposed quality assessment method. Experimental results show that the performance of proposed quality assessment method is at par with the existing no reference image quality assessment metric for Gaussian noise corrupted images.

Arrow’s Impossibility Theorem inspired subjective image quality assessment approach

A large number of subjective image quality assessment databases have been constructed in the last decade, in which the Mean Opinion Score (MOS) (with single or double stimulus), and Paired Comparison (PC) are two dominant approaches for collecting the ground truth quality ratings and usually up to 15 or more subjects are needed for each image. In this paper, we show the fact that there is a potential “dictatorship” risk of using such averaging-over-multiple-rating type of method. Using Arrow’s Impossibility Theorem (AIT), we prove that meeting of the unanimity and independence of irrelevant alternatives (IIA) will generate a “pivotal subject”, who in fact determines the final rank of image quality. We also prove that no an ideal democratic approach to synthesize the whole opinions of subjects. Therefore, we advocate to recruit a small number of experts (a.k.a the “golden eyes”) for subjective viewing tests. In order to verify the reliability of our proposal, experiments on two different databases conducting on the general distorted images and professional images (here is Terahertz security image) are performed. In each experiment, the raw scores of images are subjectively assigned by at least 15 inexperienced viewers and 3 experts, and meanwhile the MOS or difference mean opinion score (DMOS) are obtained. Afterwards, the correlation of the scores rated by naive subjects and experts is analyzed. For general image experiment, it is revealed that DMOS of inexperience viewers are highly related to DMOS of experts based on six effective evaluation metrics. In professional image experiment, the preferences of experts also maintain favourable relevance with the opinions of inexperienced viewers in overall quality of THz image. Moreover, considering the quality assessments of illegal substance regions in THz images, the experts have higher accuracy than the inexperienced observers. In conclusion, the results of two validation experiments verify that a small number of experts are more suitable for assessing the perceptual quality of images, which can reduce cost and simplify procedure of creating databases.

A Detail-Based Method for Linear Full Reference Image Quality Prediction.

In this paper, a novel full Reference method is proposed for image quality assessment, using the combination of two separate metrics to measure the perceptually distinct impact of detail losses and of spurious details. To this purpose, the gradient of the impaired image is locally decomposed as a predicted version of the original gradient, plus a gradient residual. It is assumed that the detail attenuation identifies the detail loss, whereas the gradient residuals describe the spurious details. It turns out that the perceptual impact of detail losses is roughly linear with the loss of the positional Fisher information, while the perceptual impact of the spurious details is roughly proportional to a logarithmic measure of the signal to residual ratio. The affine combination of these two metrics forms a new index strongly correlated with the empirical differential mean opinion score (DMOS) for a significant class of image impairments, as verified for three independent popular databases. The method allowed alignment and merging of DMOS data coming from these different databases to a common DMOS scale by affine transformations. Unexpectedly, the DMOS scale setting is possible by the analysis of a single image affected by additive noise.

Modified-BRISQUE as no reference image quality assessment for structural MR images

An effective and practical Image Quality Assessment (IQA) model is needed to assess the image quality produced from any new hardware or software in MRI. A highly competitive No Reference – IQA (NR – IQA) model called Blind/Referenceless Image Spatial Quality Evaluator (BRISQUE) initially designed for natural images were modified to evaluate structural MR images. The BRISQUE model measures the image quality by using the locally normalized luminance coefficients, which were used to calculate the image features. The modified-BRISQUE model trained a new regression model using MR image features and Difference Mean Opinion Score (DMOS) from 775 MR images. Two types of benchmarks: objective and subjective assessments were used as performance evaluators for both original and modified-BRISQUE models. There was a high correlation between the modified-BRISQUE with both benchmarks, and they were higher than those for the original BRISQUE. There was a significant percentage improvement in their correlation values. The modified-BRISQUE was statistically better than the original BRISQUE. The modified-BRISQUE model can accurately measure the image quality of MR images. It is a practical NR-IQA model for MR images without using reference images.

A visual quality evaluation method for telemedicine applications

We present a new approach for image and video quality evaluation in telemedicine applications, relying on the quality assessment of a pre-known reduced size logo embedded in an unused part of the medical ultrasound frame. The method is tested using two different objective metrics, namely: the Peak Signal to Noise Ratio (PSNR) and the Structural SIMilarity index (SSIM). Comparisons with subjective results in terms of Differential Mean Opinion Scores (DMOS) are also presented. We show that the presented method, not needing the original frame to predict the quality, achieves a high correlation with subjective results (more than 0.9) for the different quality metrics used. We also present relationships between the quality derived via the logo and via the original frame and we assess the overhead in data transmission resulting from the compressed logo data and its protection overhead.

No-reference stereoscopic 3D image quality assessment via combined model

Currently, stereoscopic 3D image has been widely applied in many fields. However, it may suffer from various quality degradations during the acquisition and transmission. Therefore, an effective 3D image quality assessment (IQA) method has great significance for 3D multimedia applications. Since 3D image pair has two images, it is easily distorted asymmetrically. In this paper, we have designed a no-reference quality assessment algorithm for asymmetrically distorted 3D images by utilizing combined model. First, in order to extract the distorted information in different frequency, the Gabor filter bank is employed to decompose the 3D image pair. Second, the “Cyclopean” and difference maps, representing for binocular characteristic and asymmetric information, are generated from the Gabor filter results. Then, the statistical characteristics of “Cyclopean” and difference maps are estimated by utilizing the generalized Gaussian distribution (GGD) fitting. Finally, a SVR regression is learned to map the feature vector to the recorded subjective difference mean opinion scores (DMOS). Besides, we also make an attempt to utilize structural similarity index (SSIM) to measure the asymmetric information of 3D image pair. The performance of our algorithm is evaluated on the popular 3D IQA databases. Extensive results show that the proposed algorithm outperforms state-of-the-art no-reference 3D IQA algorithms and is comparable to some full-reference 3D IQA algorithms.

Generalized referenceless image quality assessment framework using texture energy measures and pattern strength features

Referenceless image quality assessment is a challenging and critical problem in today's multimedia applica\-tions. Texture patterns in images are normally at high frequencies compared to lower ones. Due to the effect of distortions during acquisition, compression, and transmission, texture deviation artifacts are generated that cause a granular effect in the image. Other artifacts, such as blocking, affect high frequencies in an image, causing distorted edges. Combining the analysis of texture deviation and other artifacts helps in determining the quality of an image. The proposed approach uses variation in the energy of pixels to quantify the quality of an image. These variations are calculated using texture energy measures and pattern strength-based statistical features. In the proposed approach, machine learning-based classifiers are used to predict the quality score for an image. The performance of the proposed method is tested for all images ranging from pristine to poor quality from LIVE and TID2008 databases. For different distortions, results are shown to have good correlation if they lie between the predicted score and the differential mean opinion score. Results obtained with this approach are compared with other widely used referenceless approaches. It is observed that the proposed approach shows better performance in the quantification of the quality of an image.

Quality Monitor for 3-D Video Over Hybrid Broadcast Networks

Hybrid broadcast networks are particularly envisaged to merge broadcast TV with broadband Internet and to act as a key enabler for new and better video services in the near future. This is expected to contribute for the evolution of 3-D and multiview video services due to the inherent diversity of its coded data, comprising several complementary streams. Using the multiview video-plus-depth format, at least two independent streams may be delivered through different channels over a hybrid network, that is, broadcasting backward compatible 2-D video in one channel and delivering its corresponding depth stream through complementary channels like LTE-based broadband Internet accesses. This article addresses the problem of monitoring the quality of 3-D video (color plus depth) delivered in such hybrid networking environments, proposing a novel scheme to estimate the impact of visual quality degradation resulting from packet losses in the broadband Internet carrying only the depth stream, without relying on the texture component of the video or any other reference data. A novel non-reference (NR) approach is described, operating as a cascade of two estimators, using only header information of the packets carrying the depth stream through IP broadband. The two-stage cascaded estimator comprises an NR packet-layer model based on an artificial neural network followed by a logistic model, with each stage outputting a separate quality estimate. Performance evaluations, done by comparing the actual and estimated scores for the structural similarity index and subjective differential mean-opinion score, reveals high accuracy for both of these estimates, with Pearson linear correlation coefficient values greater than 0.89. Since only packet-layer information is used, the algorithmic complexity of this monitoring tool is low, making it suitable for standalone implementation at arbitrary network nodes.

Reduced-reference video quality assessment using a static video pattern

A reduced-reference video quality assessment (VQA) method was proposed by using structural similarity (SSIM) index as a tool to extract features from both the original and the target video sequences, using a reference video pattern. The method is suitable for monitoring the video quality in real time and across the service provision chain. The performance of the proposed method was evaluated using a large experimental set of reference and nonreference video sequences and achieves an accuracy higher than 2.56% in comparison to SSIM. Additionally, comparison to subjectively evaluated scores of Laboratory for Image and Video Engineering video quality dataset, based on difference mean opinion scores, shows that the performance of the proposed method is within the range of the full reference VQA methods.

Correlation between subjective and objective assessment of magnetic resonance (MR) images

Medical Image Quality Assessment (IQA) plays an important role in assisting and evaluating the development of any new hardware, imaging sequences, pre-processing or post-processing algorithms. We have performed a quantitative analysis of the correlation between subjective and objective Full Reference - IQA (FR-IQA) on Magnetic Resonance (MR) images of the human brain, spine, knee and abdomen. We have created a MR image database that consists of 25 original reference images and 750 distorted images. The reference images were distorted with six types of distortions: Rician Noise, Gaussian White Noise, Gaussian Blur, DCT compression, JPEG compression and JPEG2000 compression, at various levels of distortion. Twenty eight subjects were chosen to evaluate the images resulting in a total of 21,700 human evaluations. The raw scores were then converted to Difference Mean Opinion Score (DMOS). Thirteen objective FR-IQA metrics were used to determine the validity of the subjective DMOS. The results indicate a high correlation between the subjective and objective assessment of the MR images. The Noise Quality Measurement (NQM) has the highest correlation with DMOS, where the mean Pearson Linear Correlation Coefficient (PLCC) and Spearman Rank Order Correlation Coefficient (SROCC) are 0.936 and 0.938 respectively. The Universal Quality Index (UQI) has the lowest correlation with DMOS, where the mean PLCC and SROCC are 0.807 and 0.815 respectively. Student's T-test was used to find the difference in performance of FR-IQA across different types of distortion. The superior IQAs tested statistically are UQI for Rician noise images, Visual Information Fidelity (VIF) for Gaussian blur images, NQM for both DCT and JPEG compressed images, Peak Signal-to-Noise Ratio (PSNR) for JPEG2000 compressed images.

Blind Image Quality Assessment Based on Multichannel Feature Fusion and Label Transfer

In this paper, we propose an efficient blind image quality assessment (BIQA) algorithm, which is characterized by a new feature fusion scheme and a ${k}$ -nearest-neighbor (KNN)-based quality prediction model. Our goal is to predict the perceptual quality of an image without any prior information of its reference image and distortion type. Since the reference image is inaccessible in many applications, the BIQA is quite desirable in this context. In our method, a new feature fusion scheme is first introduced by combining an image’s statistical information from multiple domains (i.e., discrete cosine transform, wavelet, and spatial domains) and multiple color channels (i.e., Y, Cb, and Cr). Then, the predicted image quality is generated from a nonparametric model, which is referred to as the label transfer (LT). Based on the assumption that similar images share similar perceptual qualities, we implement the LT with an image retrieval procedure, where a query image’s KNNs are searched for from some annotated images. The weighted average of the KNN labels (e.g., difference mean opinion score or mean opinion score) is used as the predicted quality score. The proposed method is straightforward and computationally appealing. Experimental results on three publicly available databases (i.e., LIVE II, TID2008, and CSIQ) show that the proposed method is highly consistent with human perception and outperforms many representative BIQA metrics.

No-reference Stereoscopic Image Quality Assessment Using Natural Scene Statistics

We present two contributions in this work: (i) a bivariate generalized Gaussian distribution (BGGD) model for the joint distribution of luminance and disparity subband coefficients of natural stereoscopic scenes and (ii) a no-reference (NR) stereo image quality assessment algorithm based on the BGGD model. We first empirically show that a BGGD accurately models the joint distribution of luminance and disparity subband coefficients. We then show that the model parameters form good discriminatory features for NR quality assessment. Additionally, we rely on the previously established result that luminance and disparity subband coefficients of natural stereo scenes are correlated, and show that correlation also forms a good feature for NR quality assessment. These features are computed for both the left and right luminance-disparity pairs in the stereo image and consolidated into one feature vector per stereo pair. This feature set and the stereo pair׳s difference mean opinion score (DMOS) (labels) are used for supervised learning with a support vector machine (SVM). Support vector regression is used to estimate the perceptual quality of a test stereo image pair. The performance of the algorithm is evaluated over popular databases and shown to be competitive with the state-of-the-art no-reference quality assessment algorithms. Further, the strength of the proposed algorithm is demonstrated by its consistently good performance over both symmetric and asymmetric distortion types. Our algorithm is called Stereo QUality Evaluator (StereoQUE).

3D video subjective quality: a new database and grade comparison study

This paper presents a research study on the subjective assessment of 3D video quality using a newly constructed 3D video database ([email protected]). This database consists of 8 original 3D video sequences, each degraded with 22 different degradation types, including degradations specific to stereoscopic systems. The subjective assessment was done with the support of a purpose-built easily customizable grade collection platform and conducted in two research laboratories, in Croatia and Portugal. Subjective scores for quality, depth and comfort were collected and DMOS (Difference Mean Opinion Score) values were calculated. Different objective measures (for image, 3D image, 2D video and 3D video) were separately compared with DMOS values for quality, depth and comfort. The 3D video grade-annotated database described is publicly accessible and can be used in research-related activities like assessment of existing objective measures, using the entire database or parts of it, and construction of new objective measures specific to 3D video degradations. The system presented can also be used to collect and compare subjective quality grades originating from different sites to study the effect of different observation conditions and observer/graders populations on the DMOS quality values for 3D video depth and comfort.

No reference image quality assessment using sparse feature representation in two dimensions spatial correlation

Universal no reference image quality assessment is attracting significant attention in the fields of image processing, visual and machine learning. This study presents a novel method to evaluate the image quality from human subjective scores of the training samples by regressing. The primary characteristic of this novel method is the learning dictionary derived from extracting features of two dimensional spatial correlations of the sample images. Each atom in the dictionary includes 10 elements. They are DMOS (differential mean opinion score), three extracted features and their corresponding image structure patches and PCMs (pixel correlative matrix). The three extracted features are the standard deviation, the gray scale deviation and the distribution width. During the quality assessment, a distorted image is transformed into an image with structural information and partitioned into patches. The patch with the largest feature value is selected and represented sparsely in the learning dictionary. Afterwards the image quality index is obtained by quantifying the sparse representation coefficients, DMOS values and feature values. Comparing with other image quality assessment models, the proposed NSRCIQ method is simple and effective. The resulted IQA scores have not only comparable accuracy, but also high linearity to human perception of image quality. Moreover, the algorithm can be implemented in real-time.

Reduced reference blockiness and blurriness meter for image quality assessment

This research work demonstrates an image quality assessment meter (IQAM) in reduced reference (RR) mode for discrete cosine transform (DCT) based compressed images. The DCT based image compression technique is lossy. As a result, it introduces blockiness and blurriness artefacts in the coded image. To send the image in image transmission systems, the system bandwidth is limited, and image data need to be adapted accordingly. To determine the quality of image at the recipient end, fewer image parameters are sent rather than the complete reference image. The aim of this paper is to suggest a method for the receiver to evaluate the objective received image quality. The presented RR quality meter operates in the frequency domain for the estimation of artefacts in the compressed image. The quality meter takes the image, performs edge detection, transforms it into the frequency domain and estimates the harmonics. The estimated strength of harmonics in the compressed image is compared with the received RR parameter for blockiness and blurriness distortions. For improved quality judgment, the combined effect of both these artefacts is measured as well. Peak signal to noise ratio (PSNR) is another most familiar method of image quality assessment. The received image quality is also observed by measuring the PSNR. The objective values of the artefacts obtained by the designed RR IQAM algorithm are correlated with the subjective differential mean opinion scores (DMOS) blockiness and blurriness values of Laboratory for Image and Video Engineering (LIVE) image database2. The results obtained are well matched and give 93–94% correlation with the LIVE database values, whereas the correlation obtained by PSNR values with LIVE DMOS values is only around 70%. This proves the practical application of the designed IQAM in RR.

No-reference image quality assessment based on noise, blurring and blocking effect

Since existing no-reference image quality assessment (IQA) algorithms are not consistent with subjective assessment, a novel no-reference image quality assessment method is proposed by introducing three types of image distortion, including noise, blur degree and blocking effects. Firstly, the standard deviation of image noise is estimated by modified wavelet medium estimation. Secondly, the blur degree of image is obtained by counting edge pixel points. Thirdly, blocking effect is represented by characteristics of image pixel blocks. Finally, the assessment model is established by combining these three distortion types. Combining the differential mean opinion scores (DMOS) provided in the LIVE IQA database, the weighting coefficients are obtained. The experimental results indicate that these evaluation values of this algorithm not only agree with PSNR in objective assessment, but also are consistent with the DMOS in subjective assessment.

Novel full-reference image quality assessment metric based on entropy fusion

In this paper, by analyzing the performances of existing image quality assessment functions in different differential mean opinion scores (DMOS) ranges, the best method in each range is selected. These methods are further integrated with each other based on entropy fusion strategy. A novel full-reference image quality assessment function called Multi-Metric Fusion (MMF) is constructed. Experimental results showed that the new developed image quality assessment function can obtain better performance than other popular methods and the statistical significances verified its novelty.

Video Quality Assessment on Mobile Devices: Subjective, Behavioral and Objective Studies

We introduce a new video quality database that models video distortions in heavily-trafficked wireless networks and that contains measurements of human subjective impressions of the quality of videos. The new LIVE Mobile Video Quality Assessment (VQA) database consists of 200 distorted videos created from 10 RAW HD reference videos, obtained using a RED ONE digital cinematographic camera. While the LIVE Mobile VQA database includes distortions that have been previously studied such as compression and wireless packet-loss, it also incorporates dynamically varying distortions that change as a function of time, such as frame-freezes and temporally varying compression rates. In this article, we describe the construction of the database and detail the human study that was performed on mobile phones and tablets in order to gauge the human perception of quality on mobile devices. The subjective study portion of the database includes both the differential mean opinion scores (DMOS) computed from the ratings that the subjects provided at the end of each video clip, as well as the continuous temporal scores that the subjects recorded as they viewed the video. The study involved over 50 subjects and resulted in 5,300 summary subjective scores and time-sampled subjective traces of quality. In the behavioral portion of the article we analyze human opinion using statistical techniques, and also study a variety of models of temporal pooling that may reflect strategies that the subjects used to make the final decision on video quality. Further, we compare the quality ratings obtained from the tablet and the mobile phone studies in order to study the impact of these different display modes on quality. We also evaluate several objective image and video quality assessment (IQA/VQA) algorithms with regards to their efficacy in predicting visual quality. A detailed correlation analysis and statistical hypothesis testing is carried out. Our general conclusion is that existing VQA algorithms are not well-equipped to handle distortions that vary over time. The LIVE Mobile VQA database, along with the subject DMOS and the continuous temporal scores is being made available to researchers in the field of VQA at no cost in order to further research in the area of video quality assessment.

Blind Image Quality Assessment Without Human Training Using Latent Quality Factors

We propose a highly unsupervised, training free, no reference image quality assessment (IQA) model that is based on the hypothesis that distorted images have certain latent characteristics that differ from those of “natural” or “pristine” images. These latent characteristics are uncovered by applying a “topic model” to visual words extracted from an assortment of pristine and distorted images. For the latent characteristics to be discriminatory between pristine and distorted images, the choice of the visual words is important. We extract quality-aware visual words that are based on natural scene statistic features [1]. We show that the similarity between the probability of occurrence of the different topics in an unseen image and the distribution of latent topics averaged over a large number of pristine natural images yields a quality measure. This measure correlates well with human difference mean opinion scores on the LIVE IQA database [2].