Stereoscopic Image Quality Assessment Research Articles

No-reference stereoscopic 3D image quality assessment via combined model

Currently, stereoscopic 3D image has been widely applied in many fields. However, it may suffer from various quality degradations during the acquisition and transmission. Therefore, an effective 3D image quality assessment (IQA) method has great significance for 3D multimedia applications. Since 3D image pair has two images, it is easily distorted asymmetrically. In this paper, we have designed a no-reference quality assessment algorithm for asymmetrically distorted 3D images by utilizing combined model. First, in order to extract the distorted information in different frequency, the Gabor filter bank is employed to decompose the 3D image pair. Second, the “Cyclopean” and difference maps, representing for binocular characteristic and asymmetric information, are generated from the Gabor filter results. Then, the statistical characteristics of “Cyclopean” and difference maps are estimated by utilizing the generalized Gaussian distribution (GGD) fitting. Finally, a SVR regression is learned to map the feature vector to the recorded subjective difference mean opinion scores (DMOS). Besides, we also make an attempt to utilize structural similarity index (SSIM) to measure the asymmetric information of 3D image pair. The performance of our algorithm is evaluated on the popular 3D IQA databases. Extensive results show that the proposed algorithm outperforms state-of-the-art no-reference 3D IQA algorithms and is comparable to some full-reference 3D IQA algorithms.

A ParaBoost stereoscopic image quality assessment (PBSIQA) system

The problem of stereoscopic image quality assessment, which finds applications in 3D visual content delivery such as 3DTV, is investigated in this work. Specifically, we propose a new ParaBoost (parallel-boosting) stereoscopic image quality assessment (PBSIQA) system. The system consists of two stages. In the first stage, various distortions are classified into a few types, and individual quality scorers targeting at a specific distortion type are developed. These scorers offer complementary performance in face of a database consisting of heterogeneous distortion types. In the second stage, scores from multiple quality scorers are fused to achieve the best overall performance, where the fuser is designed based on the parallel boosting idea borrowed from machine learning. Extensive experimental results are conducted to compare the performance of the proposed PBSIQA system with those of existing stereo image quality assessment (SIQA) metrics. The developed quality metric can serve as an objective function to optimize the performance of a 3D content delivery system.

Modeling the Perceptual Quality of Stereoscopic Images in the Primary Visual Cortex

Quality assessment of 3-D images encounters more challenges in better understanding of human visual system. In this paper, we propose a perceptual quality assessment approach for stereoscopic images by modeling visual properties of the primary visual cortex. For this purpose, we obtain a new feature encoding approach for the visual information, and define a new similarity measure approach to match the feature encoding to give more reliable and accurate quality assessment. Experimental results on three symmetrically, asymmetrically, and multiple distorted stereoscopic image quality assessment databases demonstrate that our method has high consistency with subjective assessment.

A stereoscopic image quality assessment model based on independent component analysis and binocular fusion property

Quality assessment for stereoscopic image is an important research issue in three-dimensional researches. In this paper, an independent component analysis(ICA) and binocular combination-based full reference image quality assessment (FR-IQA) method is proposed for color stereoscopic image. Specifically, image features that reflect the responds of simple cells in the cortex are first extracted by an ICA-based feature detector, which models the functions of the receptive fields of simple cells in the primary visual cortex. Both image feature similarity (IFS) and local luminance consistency (LLC) of the right and left images are calculated to measure the structure and brightness distortions respectively. Image patches that the mean pixel values have been removed are used for IFS calculation, while the computation of LLC is based on the removed mean pixel value. To simulate the binocular fusion properties of the complex cells in the cortex, feature energy of the extracted image features is utilized to calculate the weighting factors of binocular combination, following which a single feature similarity index is obtained by fusing the right and left images IFS. Furthermore, global relative luminance information of the selected image patches is used to integrate the right and left images LLC into a single luminance consistency index. Finally, image quality is obtained by combining above two indexes. Experimental results demonstrate that the proposed algorithm achieves high consistency with subjective assessment on 3D image quality assessment databases.

No-reference stereoscopic image-quality metric accounting for left and right similarity map and spatial structure degradation.

Blind quality assessment of 3D images is used to confront more real challenges than 2D images. In this Letter, we develop a no-reference stereoscopic image quality assessment (SIQA) model based on the proposed left and right (LR)-similarity map and structural degradation. In the proposed method, local binary pattern features are extracted from the cyclopean image that are effective for describing the distortion of 3D images. More importantly, we first propose the LR-similarity map that can indicate the stereopair quality and demonstrate that the use of LR-similarity information results in a consistent improvement in the performance. The massive experimental results on the LIVE 3D and IRCCyN IQA databases demonstrate that the designed model is strongly correlated to subjective quality evaluations and competitive to the state-of-the-art SIQA algorithms.

Parts-based stereoscopic image assessment by learning binocular manifold color visual properties

Existing stereoscopic image quality assessment (SIQA) methods are mostly based on the luminance information, in which color information is not sufficiently considered. Actually, color is part of the important factors that affect human visual perception, and nonnegative matrix factorization (NMF) and manifold learning are in line with human visual perception. We propose an SIQA method based on learning binocular manifold color visual properties. To be more specific, in the training phase, a feature detector is created based on NMF with manifold regularization by considering color information, which not only allows parts-based manifold representation of an image, but also manifests localized color visual properties. In the quality estimation phase, visually important regions are selected by considering different human visual attention, and feature vectors are extracted by using the feature detector. Then the feature similarity index is calculated and the parts-based manifold color feature energy (PMCFE) for each view is defined based on the color feature vectors. The final quality score is obtained by considering a binocular combination based on PMCFE. The experimental results on LIVE I and LIVE Π 3-D IQA databases demonstrate that the proposed method can achieve much higher consistency with subjective evaluations than the state-of-the-art SIQA methods.

Quality assessment metric of stereo images considering cyclopean integration and visual saliency

In recent years, there has been great progress in the wider use of three-dimensional (3D) technologies. With increasing sources of 3D content, a useful tool is needed to evaluate the perceived quality of the 3D videos/images. This paper puts forward a framework to evaluate the quality of stereoscopic images contaminated by possible symmetric or asymmetric distortions. Human visual system (HVS) studies reveal that binocular combination models and visual saliency are the two key factors for the stereoscopic image quality assessment (SIQA) metric. Therefore inspired by such findings in HVS, this paper proposes a novel saliency map in SIQA metric for the cyclopean image called “cyclopean saliency”, which avoids complex calculations and produces good results in detecting saliency regions. Moreover, experimental results show that our metric significantly outperforms conventional 2D quality metrics and yields higher correlations with human subjective judgment than the state-of-art SIQA metrics. 3D saliency performance is also compared with “cyclopean saliency” in SIQA. It is noticed that the proposed metric is applicable to both symmetric and asymmetric distortions. It can thus be concluded that the proposed SIQA metric can provide an effective evaluation tool to assess stereoscopic image quality.

No-reference Stereoscopic Image Quality Assessment Using Binocular Self-similarity and Deep Neural Network

Quality assessment of three-dimensional (3D) images is more challenging than that of 2D images. The quality of 3D visual experience is one of the most challenging areas of human binocular perception and is affected by multiple factors such as asymmetric stereo image/video compression, depth perception, visual discomfort, and single view quality. In this paper, we propose a new no-reference quality assessment method for stereoscopic images based on Binocular Self-similarity (BS) and Deep Neural Networks (DNN). To be more specific, a BS index is defined and computed according to binocular rivalry and suppression based on the depth image-based rendering technique. Then, a DNN is trained in an opinion unaware way to predict local quality. Binocular integration (BI) index is calculated by using the trained DNN, accounting for binocular integration behaviors. Finally, the final quality score of stereoscopic image is obtained by combining the BS and BI indexes together. Experimental results on four public 3D image quality assessment databases demonstrate that compared with existing methods, the proposed method can achieve high consistency with subjective perception on stereoscopic images with both symmetric and asymmetric distortions.

Reorganized DCT-based image representation for reduced reference stereoscopic image quality assessment

In this paper, a novel reduced reference (RR) stereoscopic image quality assessment (SIQA) is proposed by characterizing the statistical properties of the stereoscopic image in the reorganized discrete cosine transform (RDCT) domain. Firstly, the difference image between the left and right view images is computed. Afterwards, the left and right view images, as well as the difference image, are decomposed by block-based discrete cosine transform (DCT). The DCT coefficients are further reorganized into a three-level coefficient tree, resulting in ten RDCT subbands. For each RDCT subband, the statistical property of the coefficient distribution is modeled by the generalized Gaussian density (GGD) function. And the mutual information (MI) and energy distribution ratio (EDR) are employed to depict the statistical properties across different RDCT subbands. Moreover, EDR can further model the mutual masking property of the human visual system (HVS). By considering the GGD modeling behavior within each RDCT subband and MI together EDR characterizing behavior across RDCT subbands, the statistical properties of the stereoscopic image are fully exploited, including the left view, right view, and the difference image. Experimental results demonstrated that the statistical properties of the difference image can well represent the perceptual quality of the stereoscopic image, which outperforms the representative RR quality metrics for stereoscopic image and even some full reference (FR) quality metrics. By considering the left view, right view, and difference image together, the performances of the proposed RR SIQA can be further improved, which presenting a more closely relationship between the quality metric output and human visual perception.

Models of Monocular and Binocular Visual Perception in Quality Assessment of Stereoscopic Images

Assessing quality of experience (QoE) for three-dimensional (3-D) video is challenging. In this paper, we propose a new full-reference stereoscopic image quality metric, by simulating the behaviors of visual perception with simple and complex receptive field properties and constructing the models of monocular and binocular visual perception. To be more specific, the stereoscopic images are first classified into noncorresponding and corresponding regions. Then, monocular energy responses are generated for the noncorresponding region based on stimuli from different spatial frequencies and orientations, and binocular energy responses are generated for the corresponding region based on stimuli from different spatial frequencies, orientations and disparities, respectively. Finally, gradient similarities between the energy responses of the original and distorted stereoscopic images are measured for noncorresponding and corresponding regions, respectively, and all results are fused to get an overall score. Experiments on three 3-D image quality assessment (IQA) databases demonstrate that in comparison with the most related existing methods, the devised algorithm achieves higher agreement with subjective assessment, making it better suited for the evaluation and optimization of stereoscopic image processing algorithms.

Stereoscopic image quality assessment using disparity-compensated view filtering

Stereoscopic image quality assessment (IQA) plays a vital role in stereoscopic image/video processing systems. We propose a new quality assessment for stereoscopic image that uses disparity-compensated view filtering (DCVF). First, because a stereoscopic image is composed of different frequency components, DCVF is designed to decompose it into high-pass and low-pass components. Then, the qualities of different frequency components are acquired according to their phase congruency and coefficient distribution characteristics. Finally, support vector regression is utilized to establish a mapping model between the component qualities and subjective qualities, and stereoscopic image quality is calculated using this mapping model. Experiments on the LIVE 3-D IQA database and NBU 3-D IQA databases demonstrate that the proposed method can evaluate stereoscopic image quality accurately. Compared with several state-of-the-art quality assessment methods, the proposed method is more consistent with human perception.

Stereoscopic image quality assessment method based on binocular combination saliency model

The objective quality assessment of stereoscopic images plays an important role in three-dimensional (3D) technologies. In this paper, we propose an effective method to evaluate the quality of stereoscopic images that are afflicted by symmetric distortions. The major technical contribution of this paper is that the binocular combination behaviors and human 3D visual saliency characteristics are both considered. In particular, a new 3D saliency map is developed, which not only greatly reduces the computational complexity by avoiding calculation of the depth information, but also assigns appropriate weights to the image contents. Experimental results indicate that the proposed metric not only significantly outperforms conventional 2D quality metrics, but also achieves higher performance than the existing 3D quality assessment models.

Learning structure of stereoscopic image for no-reference quality assessment with convolutional neural network

In this paper, we propose to learn the structures of stereoscopic image based on convolutional neural network (CNN) for no-reference quality assessment. Taking image patches from the stereoscopic images as inputs, the proposed CNN can learn the local structures which are sensitive to human perception and representative for perceptual quality evaluation. By stacking multiple convolution and max-pooling layers together, the learned structures in lower convolution layers can be composed and convolved to higher levels to form a fixed-length representation. Multilayer perceptron (MLP) is further employed to summarize the learned representation to a final value to indicate the perceptual quality of the stereo image patch pair. With different inputs, two different CNNs are designed, namely one-column CNN with only the image patch from the difference image as input, and three-column CNN with the image patches from left-view image, right-view image, and difference image as the input. The CNN parameters for stereoscopic images are learned and transferred based on the large number of 2D natural images. With the evaluation on public LIVE phase-I, LIVE phase-II, and IVC stereoscopic image databases, the proposed no-reference metric achieves the state-of-the-art performance for quality assessment of stereoscopic images, and is even competitive to existing full-reference quality metrics.

Binocular visual characteristics-based stereoscopic image quality assessment metric for 3D video system

Binocular visual characteristics-based stereoscopic image quality assessment metric for 3D video system

Binocular visual characteristics-based stereoscopic image quality assessment metric for 3D video system

Compared with two-dimensional image quality assessment 2D-IQA, stereoscopic IQA SIQA has to take the binocular perception into consideration. In this paper, a new SIQA metric is proposed for three-dimensional video systems, which is based on binocular visual characteristics and contains two parts: binocular perceptual distortion evaluation BPDE and stereoscopic evaluation SE. To get the results of the BPDE and SE, a simple-complex cell along with different human visual system characteristics are used to compute the quality of binocular fusion maps and disparity maps between original and distorted stereoscopic image pairs. The final quality score is obtained by a nonlinear combination of BPDE value and SE value. Experimental results show that proposed SIQA metric in which the Pearson linear correlation coefficient PLCC and Spearman rank order correlation coefficient SROCC exceed 0.94 and 0.93 outperform current 2D-IQA and SIQA metrics.

Quality Index for Stereoscopic Images by Separately Evaluating Adding and Subtracting.

The human visual system (HVS) plays an important role in stereo image quality perception. Therefore, it has aroused many people’s interest in how to take advantage of the knowledge of the visual perception in image quality assessment models. This paper proposes a full-reference metric for quality assessment of stereoscopic images based on the binocular difference channel and binocular summation channel. For a stereo pair, the binocular summation map and binocular difference map are computed first by adding and subtracting the left image and right image. Then the binocular summation is decoupled into two parts, namely additive impairments and detail losses. The quality of binocular summation is obtained as the adaptive combination of the quality of detail losses and additive impairments. The quality of binocular summation is computed by using the Contrast Sensitivity Function (CSF) and weighted multi-scale (MS-SSIM). Finally, the quality of binocular summation and binocular difference is integrated into an overall quality index. The experimental results indicate that compared with existing metrics, the proposed metric is highly consistent with the subjective quality assessment and is a robust measure. The result have also indirectly proved hypothesis of the existence of binocular summation and binocular difference channels.

Reduced Reference Stereoscopic Image Quality Assessment Based on Binocular Perceptual Information

In this paper, we propose a novel reduced reference stereoscopic image quality assessment (RR-SIQA) metric by using binocular perceptual information (BPI). BPI is represented by the distribution statistics of visual primitives in left and right views’ images, which are extracted by sparse coding and representation . Specifically, entropy of the left view’s image and entropy of the right view’s image are used to represent monocular cue. Their mutual information is used to represent binocular cue. Constructively, we represent BPI as three numerical indicators . The difference of the original and distorted images’ BPIs is taken as perceptual loss vector. The perceptual loss vector is used to compute the quality score for a stereoscopic image by a prediction function which is trained using support vector regression (SVR). Experimental results show that the proposed metric achieves significantly higher prediction accuracy than the state-of-the-art reduced reference SIQA methods and better than several state-of-the-art full reference SIQA methods on the LIVE phase II asymmetric databases.

Monocular-binocular feature fidelity induced index for stereoscopic image quality assessment.

Perceptual three-dimensional image quality assessment (3D-IQA) aims to use computational models to measure image quality in a way that is consistent with human visual perception. Unlike previous studies that directly extended the two-dimensional metrics to the three-dimensional (3D) case, the major technical innovation of this paper is to simulate monocular and binocular visual perception and propose a monocular-binocular feature fidelity induced index for 3D-IQA. To be more specific, in the training stage, we train monocular and binocular dictionaries from the training database, so that the latent response properties can be represented as a set of basis vectors. In the quality estimation stage, we compute monocular feature fidelity and binocular feature fidelity indexes based on the estimated sparse coefficient vectors and compute a global energy response similarity index by considering global energy changes. The final quality score is obtained by incorporating them. Experimental results on four 3D-IQA databases demonstrate that in comparison with the most relevant existing methods, the devised algorithm achieves high consistency alignment with subjective assessment.

Using Binocular Feature Combination for Blind Quality Assessment of Stereoscopic Images

The quality assessment of 3D images is more challenging than its 2D counterparts, and little investigation has been dedicated to blind quality assessment of stereoscopic images. In this letter, we propose a novel blind quality assessment for stereoscopic images based on binocular feature combination. The prominent contribution of this work is that we simplify the process of binocular quality prediction as monocular feature encoding and binocular feature combination. Experimental results on two publicly available 3D image quality assessment databases demonstrate the promising performance of the proposed method.

MCL-3D: A Database for Stereoscopic Image Quality Assessment using 2D-Image-Plus-Depth Source

A new stereoscopic image quality assessment database rendered using the 2D-image- plus-depth source, called MCL-3D, is described and the performance benchmarking of several known 2D and 3D image quality metrics using the MCL-3D database is presented in this work. Nine image-plus-depth sources are first selected, and a depth image- based rendering (DIBR) technique is used to render stereoscopic image pairs. Distortions applied to either the texture image or the depth image before stereoscopic image rendering include: Gaussian blur, additive white noise, down-sampling blur, JPEG and JPEG-2000 (JP2K) compression and transmission error. Furthermore, the distortion caused by imperfect rendering is also examined. The MCL-3D database contains 693 stereoscopic image pairs, where one third of them are of resolution 1024*768 and two thirds are of resolution 1920*1080. The pair-wise comparison was adopted in the subjective test for user friendliness, and the Mean Opinion Score (MOS) were computed accordingly. Finally, we evaluate the performance of several 2D and 3D image quality metrics applied to MCL-3D. All texture images, depth images, rendered image pairs in MCL- 3D and their MOS values obtained in the subjective test are available to the public (http: //mcl.usc.edu/mcl-3d-database/) for future research and development.