Perceptual Quality Score Research Articles

Local Indicators of Spatial Autocorrelation (LISA): Application to Blind Noise-Based Perceptual Quality Metric Index for Magnetic Resonance Images.

Noise-based quality evaluation of MRI images is highly desired in noise-dominant environments. Current noise-based MRI quality evaluation methods have drawbacks which limit their effective performance. Traditional full-reference methods such as SNR and most of the model-based techniques cannot provide perceptual quality metrics required for accurate diagnosis, treatment and monitoring of diseases. Although techniques based on the Moran coefficients are perceptual quality metrics, they are full-reference methods and will be ineffective in applications where the reference image is not available. Furthermore, the predicted quality scores are difficult to interpret because their quality indices are not standardized. In this paper, we propose a new no-reference perceptual quality evaluation method for grayscale images such as MRI images. Our approach is formulated to mimic how humans perceive an image. It transforms noise level into a standardized perceptual quality score. Global Moran statistics is combined with local indicators of spatial autocorrelation in the form of local Moran statistics. Quality score is predicted from perceptually weighted combination of clustered and random pixels. Performance evaluation, comparative performance evaluation and validation by human observers, shows that the proposed method will be a useful tool in the evaluation of retrospectively acquired MRI images and the evaluation of noise reduction algorithms.

Blind Noisy Image Quality Assessment Using Sub-Band Kurtosis.

Noise that afflicts natural images, regardless of the source, generally disturbs the perception of image quality by introducing a high-frequency random element that, when severe, can mask image content. Except at very low levels, where it may play a purpose, it is annoying. There exist significant statistical differences between distortion-free natural images and noisy images that become evident upon comparing the empirical probability distribution histograms of their discrete wavelet transform (DWT) coefficients. The DWT coefficients of low- or no-noise natural images have leptokurtic, peaky distributions with heavy tails; while noisy images tend to be platykurtic with less peaky distributions and shallower tails. The sample kurtosis is a natural measure of the peakedness and tail weight of the distributions of random variables. Here, we study the efficacy of the sample kurtosis of image wavelet coefficients as a feature driving, an extreme learning machine which learns to map kurtosis values into perceptual quality scores. The model is trained and tested on five types of noisy images, including additive white Gaussian noise, additive Gaussian color noise, impulse noise, masked noise, and high-frequency noise from the LIVE, CSIQ, TID2008, and TID2013 image quality databases. The experimental results show that the trained model has better quality evaluation performance on noisy images than existing blind noise assessment models, while also outperforming general-purpose blind and full-reference image quality assessment methods.

Hearing Aid Speech Enhancement Using Phase Difference-Controlled Dual-Microphone Generalized Sidelobe Canceller

This paper proposes a new technique for improving a generalized sidelobe canceller (GSC) for dual-microphone speech enhancement to be applied in an auditory device such as a hearing aid. Here, the GSC is implemented on a 32-channel uniform polyphase discrete Fourier transform filter bank, where the overall algorithm processing delay is 8 ms to meet hearing aid requirements. The proposed method can improve the fixed beamformer (FBF) and control the adaptive algorithm in the noise canceller (NC) using the phase difference obtained from dual-microphone signals. For this, spatial cues such as the phase differences are used to estimate the target-to-non-target directional signal ratio (TNR). A target-directional speech enhancing spectral gain-attenuator is calculated based on the estimated TNR, which is then incorporated to improve the FBF in the GSC. Furthermore, the weight update of the adaptive NC in the GSC is formulated using the phase difference-based TNR. The experimental results show that the auditory speech enhancement system that employs the proposed dual-microphone GSC algorithm provides better perceptual quality and intelligibility scores than conventional methods such as a beamformer, phase-error-based filter (PEF), GSC, or PEF-controlled GSC under multiple noise conditions of signal-to-noise ratio range 0-20 dB.

Predicting perceptual quality of images in realistic scenario using deep filter banks

Classical image perceptual quality assessment models usually resort to natural scene statistic methods, which are based on an assumption that certain reliable statistical regularities hold on undistorted images and will be corrupted by introduced distortions. However, these models usually fail to accurately predict degradation severity of images in realistic scenarios since complex, multiple, and interactive authentic distortions usually appear on them. We propose a quality prediction model based on convolutional neural network. Quality-aware features extracted from filter banks of multiple convolutional layers are aggregated into the image representation. Furthermore, an easy-to-implement and effective feature selection strategy is used to further refine the image representation and finally a linear support vector regression model is trained to map image representation into images’ subjective perceptual quality scores. The experimental results on benchmark databases present the effectiveness and generalizability of the proposed model.

From Sparse Coding Significance to Perceptual Quality: A New Approach for Image Quality Assessment.

An increasing number of image processing applications require an automated quality prediction of the visual content as perceived by humans. Since, sparse coding is suggested to be an underlying strategy of the brain's neural system, it would be logical to assume that specific tasks like quality assessment also attempt to adhere to this strategy. However, existing perceptual quality predictors, often mimicking the different stages of the human visual system and deploying machine learning strategies, such as neural networks, rarely integrate the concept of sparse coding in their design. In this paper, we first investigate the validity of such assumption by performing an empirical analysis on the relation between the structural information of the scene-captured via sparseness significance-and perceptual quality. Subsequently, we propose a new approach to integrate the significance of sparse coding features in the future imagequality measure (IQM) designs. We utilize the Fourier transform as a case study, which leads to a new IQM called sparseness significance ranking measure (SSRM). This measure essentially deploys a Fourier basis for sparse coding, a ranking mechanism based upon the amplitudes of the sparse coefficients and subsequently a complex correlation metric that assesses the correspondence between the ranked coefficient amplitude profiles of the reference and the distorted image. Moreover, we introduce a new methodology, namely separation ratio analysis, to assess the prediction quality of individual features or quality predictors given a target perceptual quality. The quality predictions by the proposed SSRM show excellent compatibility with perceptual quality scores. A set of routine benchmarking experiments utilizing the LIVE and CSIQ, IVC and TID2008 databases indicates a highly competitive performance with state of the art IQMs. Moreover, it delivers this performance at a low computational cost.

Objective Video Quality Assessment Based on Perceptually Weighted Mean Squared Error

Object quality assessment for compressed video is critical to various video compression systems that are essential in the video delivery and storage. Although mean squared error (MSE) is computationally simple, it may not be accurate to reflect the perceptual quality of compressed videos, which are also affected dramatically by the characteristics of the human visual system (HVS), such as contrast sensitivity, visual attention, and masking effect. In this paper, a video quality metric is proposed based on perceptually weighted MSE. A low-pass filter is designed to model the contrast sensitivity of the HVS with the consideration of visual attention. The imperceptible distortion is adaptively removed in the salient and nonsalient regions. To quantitatively measure the masking effect, the randomness of video content is proposed in both the spatial and temporal domains. Since the masking effect highly depends on the regularity of structure and motion in the spatial and temporal directions, the video signal is modeled as a linear dynamic system, and the prediction error of future frames from previous frames is used as randomness to measure the significance of masking. The relation is investigated between MSE and perceptual quality scores across various contents, and a masking modulation model is proposed to compensate the impact of the masking effect on the MSE. The performance of the proposed quality metric is validated on three video databases with various compression distortions. The experimental results demonstrate that the proposed algorithm outperforms other benchmark quality metrics.

Interactive Object Surface Retexturing using Perceptual Quality Indexes

This paper presents an intuitive retexturing system for editing object surfaces in images interactively. In the conventional retexturing methods, users were required to select texture images from a texture database. On the other hands, our system requires users to respond intuitive terms of material perception. Then, based on the user response, an optimal texture image is selected for the retexturing. For calculating an optimal texture image, we develop a material texture database with perceptual quality scores. The database is constructed on the basis of our subjective experiments with nine perceptual quality indexes which were suggested by Fleming et al. (2013). In the actual retexturing system, first, a user captures or selects an image including a retexturing target. Second, a target object region is extracted by Lazy Snapping which is an interactive image segmentation technique. Third, as to represent a target surface, the user sets the perceptual quality parameters and preferred object materials. Then, based on the user settings, an optimal texture is calculated from our material texture database. Finally the user achieves a retextured object image by adopting the optimal texture. For realizing interactive retexturing system, we have implemented our algorithm on tablet computers with Android and Windows OSs.

No-Reference Video Quality Assessment With 3D Shearlet Transform and Convolutional Neural Networks

In this paper, we propose an efficient general-purpose no-reference (NR) video quality assessment (VQA) framework that is based on 3D shearlet transform and convolutional neural network (CNN). Taking video blocks as input, simple and efficient primary spatiotemporal features are extracted by 3D shearlet transform, which are capable of capturing natural scene statistics properties. Then, CNN and logistic regression are concatenated to exaggerate the discriminative parts of the primary features and predict a perceptual quality score. The resulting algorithm, which we name shearlet- and CNN-based NR VQA (SACONVA), is tested on well-known VQA databases of Laboratory for Image & Video Engineering, Image & Video Processing Laboratory, and CSIQ. The testing results have demonstrated that SACONVA performs well in predicting video quality and is competitive with current state-of-the-art full-reference VQA methods and general-purpose NR-VQA algorithms. Besides, SACONVA is extended to classify different video distortion types in these three databases and achieves excellent classification accuracy. In addition, we also demonstrate that SACONVA can be directly applied in real applications such as blind video denoising.

Learning to rank for blind image quality assessment.

Blind image quality assessment (BIQA) aims to predict perceptual image quality scores without access to reference images. State-of-the-art BIQA methods typically require subjects to score a large number of images to train a robust model. However, subjective quality scores are imprecise, biased, and inconsistent, and it is challenging to obtain a large-scale database, or to extend existing databases, because of the inconvenience of collecting images, training the subjects, conducting subjective experiments, and realigning human quality evaluations. To combat these limitations, this paper explores and exploits preference image pairs (PIPs) such as the quality of image Ia is better than that of image Ib for training a robust BIQA model. The preference label, representing the relative quality of two images, is generally precise and consistent, and is not sensitive to image content, distortion type, or subject identity; such PIPs can be generated at a very low cost. The proposed BIQA method is one of learning to rank. We first formulate the problem of learning the mapping from the image features to the preference label as one of classification. In particular, we investigate the utilization of a multiple kernel learning algorithm based on group lasso to provide a solution. A simple but effective strategy to estimate perceptual image quality scores is then presented. Experiments show that the proposed BIQA method is highly effective and achieves a performance comparable with that of state-of-the-art BIQA algorithms. Moreover, the proposed method can be easily extended to new distortion categories.

Blind prediction of natural video quality.

We propose a blind (no reference or NR) video quality evaluation model that is nondistortion specific. The approach relies on a spatio-temporal model of video scenes in the discrete cosine transform domain, and on a model that characterizes the type of motion occurring in the scenes, to predict video quality. We use the models to define video statistics and perceptual features that are the basis of a video quality assessment (VQA) algorithm that does not require the presence of a pristine video to compare against in order to predict a perceptual quality score. The contributions of this paper are threefold. 1) We propose a spatio-temporal natural scene statistics (NSS) model for videos. 2) We propose a motion model that quantifies motion coherency in video scenes. 3) We show that the proposed NSS and motion coherency models are appropriate for quality assessment of videos, and we utilize them to design a blind VQA algorithm that correlates highly with human judgments of quality. The proposed algorithm, called video BLIINDS, is tested on the LIVE VQA database and on the EPFL-PoliMi video database and shown to perform close to the level of top performing reduced and full reference VQA algorithms.

Image quality assessment using a SVD-based structural projection

The development of objective image quality assessment (IQA) metrics aligned with human perception is of fundamental importance to numerous image-processing applications. Recently, human visual system (HVS)-based engineering algorithms have received widespread attention for their low computational complexity and good performance. In this paper, we propose a new IQA model by incorporating these available engineering principles. A local singular value decomposition (SVD) is first utilised as a structural projection tool to select local image distortion features, and then, both perceptual spatial pooling and neural networks (NN) are employed to combine feature vectors to predict a single perceptual quality score. Extensive experiments and cross-validations conducted with three publicly available IQA databases demonstrate the accuracy, consistency, robustness, and stability of the proposed approach compared to state-of-the-art IQA methods, such as Visual Information Fidelity (VIF), Visual Signal to Noise Ratio (VSNR), and Structural Similarity Index (SSIM).

Blind Image Quality Assessment Using Latent Dirichlet Allocation Model

In this paper, we propose a blind image quality assessment (IQA) method under latent Dirichlet allocation (LDA) model. To assess the image quality, firstly, we learn topic-specific word distribution by training a set of pristine and distorted images without human subjective scores. Secondly, LDA model is used to estimate probability distribution of topic for the regions in the test images. Finally, we calculate the perceptual quality score of the test image by comparing the estimated probabilities of topics of the test image with that for the pristine images. Note that the quality-aware visual words are used to represent the images, which generated with respect to the natural scene statistic features. Experimental evaluation on the publicly available subjective-rated database LIVE demonstrates that our proposed method correlates reasonably well with different mean opinion scores (DMOS).

3D 디스플레이 모니터의 주관적 화질 상관도 비교

최근 3DTV의 화질 평가와 관련된 국제 표준이 국제전기통신연합과 VQEG 등을 중심으로 진행되고 있다. 기존의 2D 디스플레이와 달리 3D 디스플레이는 영상의 재생 방식과 전달 방법에 따라 편광안경, 셔터안경, 무안경 방식 등 여러 종류가 있으며, 다양한 기술들이 개발되고 있다. 본 논문에서는 여러 방식의 3D 디스플레이 기기를 사용하여 일관된 3D 화질 평가가 가능한지 조사하기 위하여, 상이한 디스플레이를 대상으로 주관적 화질 평가 실험을 수행하였다. 실험 결과 각 디스플레이에서 측정된 화질점수는 영상의 재생 방식이나 디스플레이의 크기에 관계없이 매우 높은 상관관계를 보였으며, 이는 다양한 디스플레이를 사용하여 열화신호의 체감 화질을 측정하는 경우 유사한 결과를 얻을 수 있음을 의미한다. Recently, efforts have been made to develop international standards related to 3DTV quality assessment underway in International Telecommunication Union and Video Quality Experts Group. Unlike conventional 2D displays, there are several types of 3D display monitors: passive glasses, active glasses and auto-stereoscopic. In this paper, we performed subjective video quality tests using various 3D display monitors, in order to examine whether these display monitors can produce consistent perceptual video quality scores for processed video sequences. The experimental results show that the subjective scores of those 3D monitors are highly correlated and it appears that similar subjective scores will be obtained even when different types of 3D displays are used.