Differential Mean Opinion Score Research Articles

Smart algorithm in wireless networks for video streaming based on adaptive quantization

SummaryNowadays, smart multimedia network services have become crucial in the healthcare system. The network parameters of Quality of Service (QoS) are widely affecting the efficiency and accuracy of multimedia streaming in wireless environments. This paper proposes an adaptation framework model, which makes a relation between the QP (quantization parameter) in H.264 and H.265 codecs and the QoS of 5G wireless technology. Besides, the effect of QP and packet loss have been studied because of their impact on video streaming. Packet loss of 5G wireless network characteristic is emulated to determine the impact of QP on the received video quality using objective and subjective quality metrics such as PSNR (peak signal to noise ratio), SSIM (structure similarity), and DMOS (differential mean opinion score). In this research, a Testbed is implemented to stream the encoded video from the server to the end users. The application model framework has automatically evaluated the QoE (Quality of Experience). Accordingly, the model detects the defect of network packet loss and selects the optimum QP value to enhance the QoE by the end‐users. The application has been tested on low and high video motions with full high definition (HD) resolution (1920 × 1080) which were taken from ( https://www.xiph.org/downloads/). Test results based on the objective and subjective quality measurements indicate that an optimal QP = 35 and QP = 30 have been chosen for low and high motion respectively to satisfy user QoE requirements.

Quadratic fitting model in no-reference image quality assessment

The perceptual quality of image is affected by distortions during compression, delivery and storage. Distortions also impact automatic image quality assessment (IQA) that needs to be highly correlated with subjective scores. In the absence of reference, which is a typical scenario in practice, no-reference (NR) metrics are necessary for quality measurements. Recently such methods are proposed, and they employ natural scene statistics (NSS). The experimental analysis performed in this paper takes into consideration two fitting or regression models of several NR-IQA metrics relying on different distortion types. The results show quadratic model as promising for making relations in terms of difference mean opinion score and Shannon entropy.

RM-IQA: A new no-reference image quality assessment framework based on range mapping method

Significant progress has been made in recent years in image quality assessment (IQA). In particular, the development of deep learning has provided no-reference (NR)-IQA with more impressive solutions. However, improving the generalization of NR-IQA models is still an urgent necessity. In this study, we propose a new framework that uses the range mapping method to map an existing full-reference (FR)-IQA dataset to an NR-IQA dataset, thereby further enhancing the accuracy and generalization of the NR-IQA model. First, an NR-IQA model is employed to score an FR-IQA dataset to obtain the corresponding mean opinion score (MOS) values. Then, the correlation coefficients between these MOS values and the original differential mean opinion score (DMOS) values marked by the FR-IQA dataset itself is calculated. Subsequently, the matching sequence pair is obtained according to these correlation coefficients. Then, a range mapping function is selected based on this sequence pair, and this function is used to map the entire FR-IQA dataset to the existing NR-IQA dataset, and a new NR-IQA dataset is generated. Finally, the new and the existing NR-IQA datasets are merged into a new dataset, which can train an end-to-end multi-task network to obtain the final model RM-IQA. This model exhibits better performance as it exploits more prior information. Based on the largest available NR-IQA dataset KonIQ-10k and FR-IQA dataset KADID-10K, the experimental results proved the effectiveness of the proposed framework.

An Improved SPSIM Index for Image Quality Assessment

Objective image quality assessment (IQA) measures are playing an increasingly important role in the evaluation of digital image quality. New IQA indices are expected to be strongly correlated with subjective observer evaluations expressed by Mean Opinion Score (MOS) or Difference Mean Opinion Score (DMOS). One such recently proposed index is the SuperPixel-based SIMilarity (SPSIM) index, which uses superpixel patches instead of a rectangular pixel grid. The authors of this paper have proposed three modifications to the SPSIM index. For this purpose, the color space used by SPSIM was changed and the way SPSIM determines similarity maps was modified using methods derived from an algorithm for computing the Mean Deviation Similarity Index (MDSI). The third modification was a combination of the first two. These three new quality indices were used in the assessment process. The experimental results obtained for many color images from five image databases demonstrated the advantages of the proposed SPSIM modifications.

An optimized CNN-based quality assessment model for screen content image

Most existing convolutional neural network (CNN) based models designed for natural image quality assessment (IQA) employ image patches as training samples for data augmentation, and obtain final quality score by averaging all predicted scores of image patches. This brings two problems when applying these methods for screen content image (SCI) quality assessment. Firstly, SCI contains more complex content compared to natural image. As a result, qualities of SCI patches are different, and the subjective differential mean opinion score (DMOS) is not appropriate as qualities of all image patches. Secondly, the average score of image patches does not represent the quality of entire SCI since the human visual system (HVS) is sensitive to image patches containing texture and edge information. In this paper, we propose a novel quadratic optimized model based on the deep convolutional neural network (QODCNN) for full-reference (FR) and no-reference (NR) SCI quality assessment to overcome these two problems. The contribution of our algorithm can be concluded as follows: 1) Considering the characteristics of SCIs, a valid network architecture is designed for both NR and FR visual quality evaluation of SCIs, which makes the networks learn the feature differences for FR-IQA; 2) with the consideration of correlation between local quality and DMOS, a training data selection method is proposed to fine-tune the pre-trained model with valid SCI patches; 3) an adaptive pooling approach is employed to fuse patch quality to obtain image quality, owns strong noise robust and effects on both FR and NR IQA. Experimental results verify that our model outperforms both current no-reference and full-reference image quality assessment methods on the benchmark screen content image quality assessment database (SIQAD). Cross-database evaluation shows high generalization ability and high effectiveness of our model.

A No-Reference Image Quality Comprehensive Assessment Method

On the basis of the research status of image quality comprehensive assessment, a no-reference image quality comprehensive assessment function model is proposed in this paper. First, the image quality is classified as contrast, sharpness, and signal-to-noise ratio (SNR), and the interrelation of each assessment index is researched and analyzed; second, the weights in the comprehensive assessment model are studied when only contrast, sharpness, and SNR are changed. Finally, on the basis of studying each kind of distortion separately, and considering the different types of image distortion, we studied how to determine the weights of each index in the comprehensive image quality assessment. The results show that the no-reference image quality comprehensive assessment function model proposed in this paper can better fit human visual perception, and it has a good correlation with Difference Mean Opinion Score (DMOS). Correlation Coefficient (CC) reached 0.8331, Spearman Rank Order Correlation Coefficient (SROCC) reached 0.8206, Mean Absolute Error (MAE) was only 0.0920, Root Mean Square Error (RMSE) was only 0.1122, Outlier Ratio (OR) was only 0.0365. The method proposed in this paper can be applied to photoelectric measurement equipment television system and give an accurate and reliable quality assessment to no reference television images.

HEVC compatible perceptual multiple description video coding for reliable video transmission over packet networks

Multiple description coding (MDC) has been an effective scheme for reliable transmission of videos over error prone networks but requires higher data rates. Recently, perceptual video coding schemes are able to encode videos at a lower data rate, but with the same perceived decoding quality. Such schemes exploit human visual system (HVS) properties to encode videos. Although HVS characteristics have been used in MDC, but only for residual information and are not fully standard compatible. To this end, we propose a high efficiency video coding (HEVC) standard compatible perceptual multiple description video coding (PMDVC) framework. Our proposed framework temporally sub-samples the input video into even and odd frame sub-sequences, which are individually encoded (using HEVC) in a perceptual manner. We encode each description using a context based visual saliency model to obtain visual saliency mask, which is optimally thresholded into salient and non-salient regions. Coding tree unit (CTU) level perceptual relevance mask of each frame is generated by dividing binary saliency mask into five different groups. The quantization parameter at CTU level of each perceptual relevant group is adjusted in such a manner that data rate is minimized while maintaining the perceived quality. Our proposed HEVC based PMDVC scheme is evaluated under lossless and lossy channel conditions. Our results show better performance in data rate reduction, perceptual peak signal-to-noise-ratio, multi- scale structural similarity index, and difference mean opinion score, when compared with HEVC based temporally sub-sampled multiple description video coding scheme.

Subjective Evaluation of Ultra-high Definition (UHD) Videos

This paper presents a detailed subjective quality assessment for the ultra-high definition (UHD) videos having frame rates of 30fps and 60 fps. The subjective assessment is based on the ITU-R BT-500 recommendations, where double stimulus continuous quality scale (DSCQS-type II) test is performed for the evaluation of the perceived quality of the user’s in terms of differential mean opinion score (DMOS). Encoding of the UHD videos by opting encoders i.e. H.264/AVC, H.265/HEVC, and VP9 at five different quantization parameter (QP) levels is done to investigate the perceived user’s quality of experience (QoE) given as DMOS. Moreover, the encoding efficiency as the encoding time for each encoder and qualitative performance by employing full-reference (FR) quality metrics are presented in this work.

Three Different Features Based Metric To Assess Image Quality Blindly

Abstract When creating image quality assessment metric (IQA) no confirmation all distortion types are available. Non-specific distortion blind/no-reference (NR) IQA algorithms mostly need prior knowledge about anticipated distortions. This paper introduce a generic and distortion unaware (DU) approach for IQA with No Reference (NR). The approach uses three different measuring features which are initiated from the gist of natural scenes (NS) using Log-derivatives of the parameters; a general Gaussian distribution model, two sharpness functions, and Weibull distribution. All features were analyzed and co mpared together to examine their performance. When calibrating the proposed features performance on LIVE database, experiments show they have good contribution to the state of the art IQA and they outperform the popular full-reference peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) methods. Also they show sharpness features are the best when assess both prediction monotonicity and predict accuracy evaluation among the three features categories. Besides they show asymmetric generalized Gaussian distribution (AGGD) based features have the best correlation with differential mean opinion score.

Perceptual intra-frame coding for HEVC still picture profile based on invisible signal suppression

In this paper, a perceptual intra-frame coding for high-efficiency video coding under the still picture profile is proposed to improve coding efficiency in terms of subjective quality. The proposed method suppresses invisible signals based on the just notable distortion (JND) model by maintaining subjective quality with a lower bitrate. For quantized transform coefficients of residual signals, the magnitudes of nonzero coefficients iteratively reduce by 10% and the maximum reduction is decided by the non-noticeable quantized coefficients based on the JND measure with a given quantization parameter. The performance of the proposed algorithm was verified on the HM reference encoder. In the all-intra-configuration of the common test condition, the proposed perceptual encoder achieves a bit saving of 13.87%, on average, and up to 39.52% for several test sequences. The mean difference mean opinion score for subjective quality evaluation is observed approximately 0.64–0.15 with 15 subjects.

Screen content image quality assessment based on convolutional neural networks

Screen content image (SCI) is a composite image including textual and pictorial regions resulting in many difficulties in image quality assessment (IQA). Large SCIs are divided into image patches to increase training samples for CNN training of IQA model, and this brings two problems: (1) local quality of each image patch is not equal to subjective differential mean opinion score (DMOS) of an entire image; (2) importance of different image patches is not same for quality assessment. In this paper, we propose a novel no-reference (NR) IQA model based on the convolutional neural network (CNN) for assessing the perceptual quality of SCIs. Our model conducts two designs solving problems which benefits from two strategies. For the first strategy, to imitate full-reference (FR) CNN-based model behavior, a CNN-based model is designed for both FR and NR IQA, and performance of NR-IQA part improves when the image patch scores predicted by FR-IQA part are adopted as the ground-truth to train NR-IQA part. For the second strategy, image patch qualities of one entire SCI are fused to obtain the SCI quality with an adaptive weighting method taking account the effect of the different image patch contents. Experimental results verify that our model outperforms all test NR IQA methods and most FR IQA methods on the screen content image quality assessment database (SIQAD). On the cross-database evaluation, the proposed method outperforms the existing NR IQA method in terms of at least 2.4 percent in PLCC and 2.8 percent in SRCC, which shows high generalization ability and high effectiveness of our model.

No-reference screen content image quality assessment based on multi-region features

Unlike natural image captured with cameras, screen content image (SCI) is a composite image including textual and pictorial regions. The different characteristics lead to many difficulties in image quality assessment (IQA). Most existing models based on the convolutional neural network (CNN) divide large SCIs into image patches to increase training samples for CNN training. This brings two problems: (1) a single image patch can not represent the quality of the entire image, especially in IQA of SCI; (2) SCI patches of an entire image degraded by the same distortion type and strength may have drastically different quality. In addition, these models adopt the mean square error (MSE) between the predicted quality and the subjective differential mean opinion score (DMOS) to train the CNN, without considering quality ranking between different SCIs. In this paper, we propose a novel no-reference (NR) IQA model based on the convolutional neural network (CNN). The contributions of our algorithm can be concluded as follows: (1) considering a large difference exists in different regions in a SCI, the pseudo global features generated with multi-region local features are utilized for quality evaluation, which better reflect image quality than local features of each image patch; (2) the noise classification task is used as an auxiliary task which aids the quality score prediction task to improve the representation ability; (3) the Siamese networks are used to predict the quality scores of two different SCIs, and a new ranking loss is proposed to rank the predicted scores, aiming to enhance the ability of the model to rank image in terms of quality. Experimental results verify that our model outperforms all test NR IQA methods and full-reference (FR) IQA methods on the screen content image quality assessment database (SIQAD).

An ensemble image quality assessment algorithm based on deep feature clustering

To effective handle image quality assessment (IQA) where the images might be with sophisticated characteristics, we proposed a deep clustering-based ensemble approach for image quality assessment toward diverse images. Our approach is based on a convolutional DAE-aware deep architecture. By leveraging a layer-by-layer pre-training, our proposed deep feature clustering architecture extracted a fixed number of high-level features at first. Then, it optimally splits image samples into different clusters by using the fuzzy C-means algorithm based on the engineered deep features. For each cluster, we simulated a particular fitting function of differential mean opinion scores with each assessed image’s PSNR, SIMM, and VIF scores. Comprehensive experimental results on TID2008, TID2013 and LIVE databases have demonstrated that compared to the state-of-the-art counterparts, our proposed IQA method can reflect the subjective quality of images more accurately by seamlessly integrating the advantages of three existed IQA methods.

PeQASO: Perceptual Quality Assessment of Streamed Videos Using Optical Flow Features

In this paper, we introduce PeQASO, a perceptual quality assessment framework for streamed videos using optical flow features. This approach is a reduced-reference pixel-based and relies only on the deviation of the optical flow of the corrupted frames. This technique compares an optical flow descriptor from the received frame against the descriptor obtained from the anchor frame. This approach is suitable for videos with complex motion patterns. Our technique does not make any assumptions on the coding conditions, network loss patterns or error concealment techniques. In this paper, we consider both sources of artifacts and distortions in streaming, including compression artifacts. We validate our proposed metric by testing it on a variety of distorted sequences from three proposed and commonly utilized video quality assessment databases. Our results show that our metric estimates the perceptual quality at the sequence level accurately. We report the correlation coefficients with the differential mean opinion scores reported in the database. For compression artifacts, the results show Spearman’s and Pearson’s correlations of 0.96 and 0.94 for all the tested sequences, respectively. For channel-induced distortion, the results show Spearman’s and Pearson’s correlations of 0.88 and 0.89, respectively. For all other distortions, the average Spearman’s and Pearson’s correlations are 0.82 and 0.79, respectively.

Assessing Visual Quality of Omnidirectional Videos

In contrast with traditional video, omnidirectional video enables spherical viewing direction with support for head-mounted displays, providing an interactive and immersive experience. Unfortunately, to the best of our knowledge, there are few visual quality assessment (VQA) methods, either subjective or objective, for omnidirectional video coding. This paper proposes both subjective and objective methods for assessing quality loss in encoding omnidirectional video. Specifically, we first present a new database, which includes the viewing direction data from several subjects watching omnidirectional video sequences. Then, from our database, we find a high consistency in viewing directions across different subjects. The viewing directions are normally distributed in the center of the front regions, but they sometimes fall into other regions, related to video content. Given this finding, we present a subjective VQA method for measuring difference mean opinion score (DMOS) of the whole and regional omnidirectional video, in terms of overall DMOS (O-DMOS) and vectorized DMOS (V-DMOS), respectively. Moreover, we propose two objective VQA methods for encoded omnidirectional video, in light of human perception characteristics of omnidirectional video. One method weighs the distortion of pixels with regard to their distances to the center of front regions, which considers human preference in a panorama. The other method predicts viewing directions according to video content, and then the predicted viewing directions are leveraged to allocate weights to the distortion of each pixel in our objective VQA method. Finally, our experimental results verify that both the subjective and objective methods proposed in this paper advance state-of-the-art VQA for omnidirectional video.

Crowdsourced subjective 3D video quality assessment

This article proposes a new method for subjective 3D video quality assessment based on crowdsourced workers—Crowd3D. The limitations of traditional laboratory-based grade collection procedures are outlined, and their solution through the use of a crowd-based approach is described. Several conceptual and technical requirements of crowd-based 3D video quality assessment methods are identified and the solutions adopted described in detail. The system built takes the form of a web-based platform that supports 3D video monitors, and orchestrates the entire process of observer validation, content presentation and quality, depth, and comfort grade recording in a remote database. The crowdsourced subjective 3D quality assessment system uses as source contents a set of 3D video and grades database assembled earlier in a laboratory setting. To evaluate the validity of the crowd-based approach, the grades gathered using the crowdsourced system were analysed and compared to a set of grades obtained in laboratory settings using the same data set. Results show that it is possible to obtain Pearson’s and Spearman’s correlation up to 0.95 for quality Difference Mean Opinion Score and 0.96 for quality Mean Opinion Score, when comparing with laboratory grades. Apart from the present study, the 3D video quality assessment platform proposed can be used with advantage for further related research activities, reducing the time and cost compared to the traditional laboratory-based quality assessments.

NO-REFERENCE IMAGE QUALITY ASSESSMENT BASED ON DISTORTION SPECIFIC AND NATURAL SCENE STATISTICS BASED PARAMETERS: A HYBRID APPROACH

In web applications, for efficient use of bandwidth and storage requirement, images are compressed with lossy techniques. Quality of images degrades due to the compression artifacts. Transmission channels also contribute in degradation by adding noise. Image quality assessment plays a vital role to address this issue. In majority of cases, reference image remains unavailable for the assessment. Thus, No-Reference quality assessment techniques are widely used. In this paper, machine learning based hybrid approach for NR quality assessment is proposed. Blockiness based parameters, other statistical parameters and NSS based features are provided as input to the feed forward neural network. The back propagation training algorithm predicts a quality score. This score is correlated with differential mean opinion score (DMOS). Here, input can be images of any type from best to worst quality, and the approach used exploits the nonlinearity in the behaviors of parameters. It has been noticed that the predicted score correlates well with the DMOS with 93% accuracy. The above parameters are also used as input to Support Vector Machine for classification. The observed accuracy of this classifier is 89%.

No-Reference Image Quality Assessment Using Independent Component Analysis and Convolutional Neural Network

As digital images have become a significantly primary medium in a broad area, there is a growing interest in the development of automatic objective image quality assessment (IQA) methods. In this paper, a novel no-reference IQA (NRIQA) algorithm is proposed based on independent component analysis and convolutional neural network. The proposed NRIQA algorithm consists of the following three steps: selection of some representative patches, extraction of the features of the selected image patches, and prediction of the image quality by exploiting the features. Initially, an image is divided into non-overlapping patches and then some patches are selected with the suitable property for assessing the overall image quality. In this paper, we refer to the selected patches as image quality patches. The largest infinity norm of the gradient of each image quality patch is employed as a basis when the image quality patches being selected. Second, we employ independent component analysis (ICA) to extract the features of image quality patches. At the last moment, a convolutional neural network (CNN) is applied to the independent component coefficients of image quality patches to predict the corresponding differential mean opinion score (DMOS). We compared the performance of the proposed NQIRM with other IQMs in terms of PCC, SROCC, and RMSE on the database LIVE2, CSIQ and TID2008/2013. The PCC, SROCC and RMSE values achieve respectively to 0.996, 0.999 and 6.011 on the database TID2013. The performance comparison results show the proposed NRIQM is superior to commonly used IQMs.

A 5-Step Framework for Measuring the Quality of Experience (QoE) of Internet of Things (IoT) Services

The ultimate benefit of the Internet of Things (IoT) is to improve quality of life by supporting the automation and interconnection of various services; this interconnection includes public facilities in platforms such as smart cities. IoT implementation is believed to improve services and yield additional benefits. However, assessment of how the IoT brings such an experience to users is still a challenge that is not yet standardized. Given that little research has discussed this issue, this paper aims to formulate a framework for measuring the quality of experience (QoE) regarding IoT services on the basis of the Absolute Category Rating with Hidden Reference (ACR-HR) scale, which measures the perception of experience by having the IoT users provide scores for their experiences before and after IoT implementation. The proposed framework consists of 5 steps: setting up the focus of the IoT services and the QoE parameters being assessed, defining users of IoT services, conducting a mean opinion score (MOS) survey of the users, calculating the DMOS (differential MOS) on the basis of the ACR-HR quantitative scale, and finally providing the strategic implications to those who implement the service. To test the proposed framework, we have conducted a proof of concept by measuring the QoE for the IoT services in the Jakarta Smart City platform. Six institutional users operate the IoT for their public services, ranging from fire and rescue brigades to public bus transportation. The results imply that the IoT-based services provide a good experience to institutional users when compared to the state of services without IoT. A strategic implication is that some features of Jakarta Smart City must be evaluated and improved to increase the level of satisfaction of users. To conclude, our framework is sufficiently reliable in measuring the QoE for IoT-based services and can be recommended for use for any other IoT-type service.

On Hypothesis Testing for Comparing Image Quality Assessment Metrics [Tips & Tricks

In developing novel image quality assessment (IQA) metrics, researchers should compare their proposed metrics with state-of-the-art metrics. A commonly adopted approach is by comparing two residuals between the nonlinearly mapped scores of two IQA metrics and the difference mean opinion score, which are assumed from Gaussian distributions with zero means. An F-test is then used to test the equality of variances of the two sets of residuals. If the variances are significantly different, then we conclude that the residuals are from different Gaussian distributions and that the two IQA metrics are significantly different. The F-test assumes that the two sets of residuals are independent. However, given that the IQA metrics are calculated on the same database, the two sets of residuals are paired and may be correlated. We note this improper usage of the F-test by practitioners, which can result in misleading comparison results of two IQA metrics. To solve this practical problem, we introduce the Pitman test to investigate the equality of variances for two sets of correlated residuals. Experiments on the Laboratory for Image and Video Engineering (LIVE) database show that the two tests can provide different conclusions.