Stereoscopic 3D Image Quality Assessment Research Articles

No-reference perceptual quality assessment of stereoscopic images based on binocular visual characteristics

Perceptual quality assessment of distorted stereoscopic images has attracted considerable attention and become an important yet challenging task in 3D multimedia applications. In this paper, we propose a blind/no-reference stereoscopic 3D image quality assessment scheme that utilizes binocular visual characteristics. The design of this scheme is motivated by studies on the perception of distorted stereoscopic images. The major technical contribution of this paper is the introduction of binocular features and binocular energy for stereoscopic image quality prediction. To be more specific, the proposed scheme extracts the local magnitude, local phase and visual saliency from a stereopair as binocular features. In addition, the binocular energy responses are also obtained as quality-predictive features. After feature extraction, a statistical regression model is used to map the features of distorted stereoscopic image to its subjective quality score. The effectiveness of the designed metric is verified on the publicly available 3D image quality assessment databases. Experimental results show that the proposed scheme achieves superiority over other related algorithms in terms of consistent alignment with subjective assessment of stereoscopic 3D images.

Siamese-Network-Based Learning to Rank for No-Reference 2D and 3D Image Quality Assessment

2D image quality assessment (IQA) and stereoscopic 3D IQA are considered as two different tasks in the literature. In this paper, we present an index for both no-reference 2D and 3D IQA. We propose to transform the IQA task into a task of quality comparison between images. By generating image pairs, the amount of training data reaches the square of the original amount of data, effectively solving the lacking of training samples. We also propose a learning to rank model using Siamese convolutional neural networks (LRSN) for quality comparison. The presented LRSN has two branches that have the same structure, share weights with each other, and take two image patches as inputs. The goal of LRSN is learning to rank the quality scores between the two input image patches. The relative quality score of a test image is obtained by first comparing its image patches with many image patches of other images and counts the number of times that its image patches are ranked superior to other patches. The experimental results on three 2D (LIVE, CSIQ, and TID2013) and three 3D (LIVE 3D Phase-I, LIVE 3D Phase-II, and NBU) IQA databases demonstrate that the proposed LRSN model works well for both 2D and 3D IQA and outperforms the state-of-the-art no-reference 2D and 3D IQA metrics.

Toward an unsupervised blind stereoscopic 3D image quality assessment using joint spatial and frequency representations

Existing blind stereoscopic 3D (S3D) image quality assessment (IQA) metrics usually require supervised learning methods to predict S3D image quality, which limits their applicability in practice. In this paper, we propose an unsupervised blind S3D IQA metric that utilizes the joint spatial and frequency representations of visual perception. The metric proposed in this work was inspired by the binocular visual mechanism; furthermore, it is unsupervised and does not require subject-rated samples for training. To be more specific, first, the various binocular quality-aware features in spatial and frequency domains are extracted from the monocular and cyclopean views of natural S3D image patches. Subsequently, these features are utilized to establish a pristine multivariate Gaussian (MVG) model to characterize natural S3D image regularities. Finally, with the learned MVG model, the final quality score for a distorted S3D image can be yielded using a Bhattacharyya-like distance. Our experimental results illustrate that, compared to related existing metrics, the devised metric achieves competitive prediction performance.

Blind stereoscopic 3D image quality assessment via analysis of naturalness, structure, and binocular asymmetry

Over recent years, stereoscopic three dimensional (S3D) images have grown explosively and received increasing attention. Quality assessment, as the fundamental problem, plays an important role in promoting the prevalence of S3D images as well as the associated products. In this paper, an effective blind quality assessment method of S3D images is proposed via analysis of naturalness, structure, and binocular asymmetry. To be specific, given that natural images obey certain regular statistical properties, natural scene statistic (NSS) features of left and right views are first extracted to quantify the naturalness. Second, by considering binocular visual characteristics, statistical features are extracted from a created cyclopean map. Moreover, gray level co-occurrence matrix (GLCM) is utilized to capture quality-sensitive features from the cyclopean phase map. Third, to quantify the asymmetric distortion, a simple but effective measurement is utilized, i.e., calculating the similarity between left and right views as well as statistical features of their difference map. Finally, all extracted quality-sensitive features are combined, and trained together with the subjective ratings to form a regression model using support vector regression (SVR). Experimental results on four publicly available databases (two symmetrically distorted databases and two asymmetrically distorted databases) demonstrate that the proposed method is superior to several mainstream image quality assessment (IQA) metrics.

Blind Deep S3D Image Quality Evaluation via Local to Global Feature Aggregation.

Previously, no-reference (NR) stereoscopic 3D (S3D) image quality assessment (IQA) algorithms have been limited to the extraction of reliable hand-crafted features based on an understanding of the insufficiently revealed human visual system or natural scene statistics. Furthermore, compared with full-reference (FR) S3D IQA metrics, it is difficult to achieve competitive quality score predictions using the extracted features, which are not optimized with respect to human opinion. To cope with this limitation of the conventional approach, we introduce a novel deep learning scheme for NR S3D IQA in terms of local to global feature aggregation. A deep convolutional neural network (CNN) model is trained in a supervised manner through two-step regression. First, to overcome the lack of training data, local patch-based CNNs are modeled, and the FR S3D IQA metric is used to approximate a reference ground-truth for training the CNNs. The automatically extracted local abstractions are aggregated into global features by inserting an aggregation layer in the deep structure. The locally trained model parameters are then updated iteratively using supervised global labeling, i.e., subjective mean opinion score (MOS). In particular, the proposed deep NR S3D image quality evaluator does not estimate the depth from a pair of S3D images. The S3D image quality scores predicted by the proposed method represent a significant improvement over those of previous NR S3D IQA algorithms. Indeed, the accuracy of the proposed method is competitive with FR S3D IQA metrics, having ~ 91% correlation in terms of MOS.

No-reference stereoscopic 3D image quality assessment via combined model

Currently, stereoscopic 3D image has been widely applied in many fields. However, it may suffer from various quality degradations during the acquisition and transmission. Therefore, an effective 3D image quality assessment (IQA) method has great significance for 3D multimedia applications. Since 3D image pair has two images, it is easily distorted asymmetrically. In this paper, we have designed a no-reference quality assessment algorithm for asymmetrically distorted 3D images by utilizing combined model. First, in order to extract the distorted information in different frequency, the Gabor filter bank is employed to decompose the 3D image pair. Second, the “Cyclopean” and difference maps, representing for binocular characteristic and asymmetric information, are generated from the Gabor filter results. Then, the statistical characteristics of “Cyclopean” and difference maps are estimated by utilizing the generalized Gaussian distribution (GGD) fitting. Finally, a SVR regression is learned to map the feature vector to the recorded subjective difference mean opinion scores (DMOS). Besides, we also make an attempt to utilize structural similarity index (SSIM) to measure the asymmetric information of 3D image pair. The performance of our algorithm is evaluated on the popular 3D IQA databases. Extensive results show that the proposed algorithm outperforms state-of-the-art no-reference 3D IQA algorithms and is comparable to some full-reference 3D IQA algorithms.

스테레오스코픽 3D영상 화질 평가 방법

For objective assessment of stereoscopic 3D image quality, we measure quality of left and right image with 2D image quality measurement method. However, this method is inconvenient because that we have to measure quality of left and right image individually. Therefore we propose a method of stereoscopic 3D image quality assessment using one overlaid image with left and right image. Using this method, One can measure quality of stereoscopic 3D image more easily and quickly. Keyword : stereoscopic, 3D, image quality assessment