Stereoscopic Image Quality Assessment Metrics Research Articles

Blind quality assessment for multiply distorted stereoscopic images towards IoT-based 3D capture systems

Empowered by 5G mobile communication networks, multimedia processing has been considered as a very promising application of Internet-of-Things (IoT). Stereoscopic image quality assessment (SIQA), as an important part of 3D capture system, can be embedded in the cloud or fog servers to automatically monitor the perceptual quality of the collected stereoscopic images. In this paper, a novel blind image quality assessment method towards IoT-based 3D capture systems is developed for multiply-distorted stereoscopic images (MDSIs), in which five complementary channels, including left view, right view, cyclopean map, summation map and difference map, are jointly considered in dictionary learning for characterizing the monocular receptive field (MRF) and binocular receptive field (BRF) properties of the visual cortex in response to MDSIs. Additionally, the high order statistics scheme is adopted by utilizing the statistical differences between the codebook and images to ensure the stable and robust quality prediction performance for MDSIs. The proposed method shows competitive prediction performances on four benchmark databases compared with the existing SIQA metrics.

AdaBoost neural network and cyclopean view for no-reference stereoscopic image quality assessment

Stereoscopic imaging is widely used in many fields. In many scenarios, stereo images quality could be affected by various degradations, such as asymmetric distortion. Accordingly, to guarantee the best quality of experience, robust and accurate reference-less metrics are required for quality assessment of stereoscopic content. Most existing stereo no-reference Image Quality Assessment (IQA) models are not consistent with asymmetrical distortions. This paper presents a new no-reference stereoscopic image quality assessment metric using a human visual system (HVS) modeling and an advanced machine-learning algorithm. The proposed approach consists of two stages. In the first stage, cyclopean image is constructed considering the presence of binocular rivalry in order to cover the asymmetrically distorted part. In the second stage, gradient magnitude, relative gradient magnitude, and gradient orientation are extracted. These are used as a predictive source of information for the quality. In order to obtain the best overall performance against different databases, Adaptive Boosting (AdaBoost) idea of machine learning combined with artificial neural network model has been adopted. The benchmark LIVE 3D phase-I, phase-II, and IRCCyN/IVC 3D databases have been used to evaluate the performance of the proposed approach. Experimental results have demonstrated that the proposed metric performance achieves high consistency with subjective assessment and outperforms the blind stereo IQA over various types of distortion.

Reduced Reference Stereoscopic Image Quality Assessment Using Sparse Representation and Natural Scene Statistics

An ideal quality assessment model should simulate the properties of the visual brain to be consistent with human evaluation. The visual brain appears to have both evolved to seek an efficient, decorrelated representation of image information and to “match” the statistics of the natural image. On one hand, the theoretical studies suggest that sparse representation resembles the strategy in the primary visual cortex of brain for representing natural images. On the other hand, the natural scene statistics have driven the evolution of human visual system and have also inspired the understanding and simulating of visual perception. Inspired by these observations, in this paper, we propose a novel reduced-reference stereoscopic image quality assessment metric using sparse representation and natural scene statistics to simulate the visual perception of the brain. Specifically, the distribution statistics of the classified visual primitives extracted by sparse representation are used to measure the visual information, which is closely related to the hierarchical progressive process of human visual perception. Particularly, the mutual information of classified primitives between two view images is derived as a binocular cue to simulate the binocular fusion process. The maximum mechanism that is applied to select the visual information is a pooling mechanism with which complex cells use the maximal stimuli from a group of simple cells during the transfer process in the primary visual cortex. The natural scene statistics of locally normalized luminance coefficients are used to evaluate the natural losses due to the presence of distortions. The differences of the visual information and the natural scene statistics between the original and distorted images are used to compute the quality score by a prediction function which is trained using support vector regression. Experimental results show that the proposed metric outperforms the state-of-the-art stereoscopic image quality assessment metrics on LIVE 3D IQA database and NBU-MDSID Phase-II database, and delivers competitive performance on Waterloo IVC 3D database.

A Blind Stereoscopic Image Quality Evaluator with Segmented Stacked Autoencoders Considering The Whole Visual Perception Route.

Most of the current blind stereoscopic image quality assessment (SIQA) algorithms cannot show reliable accuracy. One reason is that they do not have the deep architectures and the other reason is that they are designed on the relatively weak biological basis, compared with findings on human visual system (HVS). In this paper, we propose a Deep Edge and COlor Signal INtegrity Evaluator (DECOSINE) based on the whole visual perception route from eyes to the frontal lobe, and especially focus on edge and color signal processing in retinal ganglion cells (RGC) and lateral geniculate nucleus (LGN). Furthermore, to model the complex and deep structure of the visual cortex, Segmented Stacked Auto-encoder (S-SAE) is used, which has not utilized for SIQA before. The utilization of the S-SAE complements weakness of deep learning-based SIQA metrics that require a very long training time. Experiments are conducted on popular SIQA databases, and the superiority of DECOSINE in terms of prediction accuracy and monotonicity is proved. The experimental results show that our model about the whole visual perception route and utilization of S-SAE are effective for SIQA.

Perceptual stereoscopic image quality assessment method with tensor decomposition and manifold learning

Perceptual quality assessment of stereoscopic images is a challenge in three-dimensional video systems. Existing studies suggest that simply averaging the quality of left and right views can effectively predict the quality of symmetrically distorted stereoscopic images, but prediction deviation occurs in the case of asymmetrically distorted stereoscopic images. Most previous stereoscopic image quality assessment (SIQA) methods have been based only on the luminance component of the images; in addition, the basis of human visual perception is critical to image quality assessment and lies on the low-dimensional manifold. Inspired by this, a new perceptual SIQA method is proposed, which includes two stages: training stage and quality prediction stage. In the training stage, the authors apply Tucker decomposition to RGB images to reduce dimensions along colour channels to produce training sets, and the projection matrix is obtained through manifold learning. In the quality prediction stage, considering the binocular visual characteristics of visual perception, the overall stereoscopic estimate depends on the monocular image quality via a local energy ratio based pooling strategy and cyclopean based binocular quality. Extensive experiments on three available benchmark databases demonstrate that the proposed metric has better performance and achieves highly consistent alignment with subjective assessment compared with state-of-the-art SIQA metrics.

Joint binocular energy-contrast perception for quality assessment of stereoscopic images

. Binocular visual system (BVS) can perceive the difference between left and right retinal images to create a mental image with depth perception, which is in consequence of two binocular interactions, i.e., binocular fusion and rivalry. To study the effective method of accounting for binocular fusion and rivalry in stereoscopic image quality assessment (SIQA) design, in this paper, a novel full reference (FR) SIQA metric is proposed by jointly considering binocular energy-contrast perception (BECP). As a major technical contribution, we design a dual-channel model for SIQA that more effectively mimic binocular fusion and rivalry mechanisms of the BVS. Specifically, since the binocular visual sensitivity of stimulus at different spatial frequencies is different, each image of the reference and distorted stereopairs is first filtered independently by a contrast sensitivity function (CSF). Constructively, the weights of relative contribution of each view for binocular fusion are calculated based on a magnitude response of Log-Gabor filtering measure. Further, the weights of relative contribution of each view for dominant perception are calculated by utilizing a block-based contrast measure. Finally, the overall perceived quality of a stereoscopic image is obtained by the quality scores combining of the BECP. Experiments are performed on publicly available symmetric and asymmetric subjected stereoscopic image databases, which demonstrate that the proposed metric achieves high consistency with human opinions and significantly higher prediction accuracy than the state-of-the-art FR-SIQA methods.

Structured entropy of primitive: big data‐based stereoscopic image quality assessment

The ultimate receiver of image and video is human visual system (HVS). It is an important problem in the domain of image and video processing that how to establish visual information representation model meeting the HVS perception property. In this study, authors give theory analysis and experiment results to prove that l_1 norm-based entropy of primitive (EoP) is superior to the l_0 norm-based EoP for the monocular cue in image quality assessment. By developing the concept of mutual information of primitive (MIP) as the binocular cue, an l_1 EoP-based stereoscopic image quality assessment metric is proposed. With EoP as monocular cue and MIP as binocular cue, the relative entropy between the original stereoscopic image and the distorted one is explored to predict the quality score with support vector regression. To avoid destroying image's structured information, the structured EoP (SEoP) is further explored to measure the stereoscopic image information. Extensive experimental results demonstrate that the stereoscopic image quality assessment algorithm with SEoP as monocular cue and MIP as binocular cue outperforms many state-of-the-art ones.

Quality assessment metric of stereo images considering cyclopean integration and visual saliency

In recent years, there has been great progress in the wider use of three-dimensional (3D) technologies. With increasing sources of 3D content, a useful tool is needed to evaluate the perceived quality of the 3D videos/images. This paper puts forward a framework to evaluate the quality of stereoscopic images contaminated by possible symmetric or asymmetric distortions. Human visual system (HVS) studies reveal that binocular combination models and visual saliency are the two key factors for the stereoscopic image quality assessment (SIQA) metric. Therefore inspired by such findings in HVS, this paper proposes a novel saliency map in SIQA metric for the cyclopean image called “cyclopean saliency”, which avoids complex calculations and produces good results in detecting saliency regions. Moreover, experimental results show that our metric significantly outperforms conventional 2D quality metrics and yields higher correlations with human subjective judgment than the state-of-art SIQA metrics. 3D saliency performance is also compared with “cyclopean saliency” in SIQA. It is noticed that the proposed metric is applicable to both symmetric and asymmetric distortions. It can thus be concluded that the proposed SIQA metric can provide an effective evaluation tool to assess stereoscopic image quality.

PMFS: A Perceptual Modulated Feature Similarity Metric for Stereoscopic Image Quality Assessment

Stereoscopic image quality assessment (SIQA) is an important and challenging issue in three dimensional applications. In this letter, a perceptual modulated feature similarity (PMFS) metric for SIQA is proposed by considering the monocular and binocular perception properties. Specifically, stereoscopic image is first classified into monocular occlusion and binocular rivalry regions. Then, feature similarities between the original and distorted stereoscopic images are defined and measured for the monocular occlusion and binocular rivalry regions as the local monocular and binocular quality maps, respectively. Monocular and binocular just noticeable difference visual saliency models are presented to construct a modulation function to derive monocular and binocular quality scores. Finally, those scores are integrated into an overall quality score by support vector regression. Extensive experiments performed on LIVE phase II and MICT asymmetric databases demonstrate that the proposed PMFS metric can achieve much higher consistency with the subjective quality scores than some state-of-the-art SIQA metrics.

New Stereoscopic Image Quality Assessment Metric Based on Three Dimensional-discrete Cosine Transform for 3d Media

New Stereoscopic Image Quality Assessment Metric Based on Three Dimensional-discrete Cosine Transform for 3d Media

No‐Reference Stereoscopic IQA Approach: From Nonlinear Effect to Parallax Compensation

The last decade has seen a booming of the applications of stereoscopic images/videos and the corresponding technologies, such as 3D modeling, reconstruction, and disparity estimation. However, only a very limited number of stereoscopic image quality assessment metrics was proposed through the years. In this paper, we propose a new no‐reference stereoscopic image quality assessment algorithm based on the nonlinear additive model, ocular dominance model, and saliency based parallax compensation. Our studies using the Toyama database result in three valuable findings. First, quality of the stereoscopic image has a nonlinear relationship with a direct summation of two monoscopic image qualities. Second, it is a rational assumption that the right‐eye response has the higher impact on the stereoscopic image quality, which is based on a sampling survey in the ocular dominance research. Third, the saliency based parallax compensation, resulted from different stereoscopic image contents, is considerably valid to improve the prediction performance of image quality metrics. Experimental results confirm that our proposed stereoscopic image quality assessment paradigm has superior prediction accuracy as compared to state‐of‐the‐art competitors.