Binocular Energy Response Research Articles

No-reference perceptual quality assessment of stereoscopic images based on binocular visual characteristics

Perceptual quality assessment of distorted stereoscopic images has attracted considerable attention and become an important yet challenging task in 3D multimedia applications. In this paper, we propose a blind/no-reference stereoscopic 3D image quality assessment scheme that utilizes binocular visual characteristics. The design of this scheme is motivated by studies on the perception of distorted stereoscopic images. The major technical contribution of this paper is the introduction of binocular features and binocular energy for stereoscopic image quality prediction. To be more specific, the proposed scheme extracts the local magnitude, local phase and visual saliency from a stereopair as binocular features. In addition, the binocular energy responses are also obtained as quality-predictive features. After feature extraction, a statistical regression model is used to map the features of distorted stereoscopic image to its subjective quality score. The effectiveness of the designed metric is verified on the publicly available 3D image quality assessment databases. Experimental results show that the proposed scheme achieves superiority over other related algorithms in terms of consistent alignment with subjective assessment of stereoscopic 3D images.

Full-reference quality assessment of stereoscopic images by learning binocular visual properties.

Stereoscopic imaging technology has been growingly prevalent driven by both the entertainment industry and scientific applications in today's world. But objective quality assessment of stereoscopic images is a challenging task. In this paper, we propose a novel stereoscopic image quality assessment (SIQA) method by jointly considering monocular perception and binocular interaction. As the most significant contribution of this study, binocular perceptual properties of simple and complex cells are considered for full-reference (FR) SIQA. Specifically, the proposed scheme first simulates the receptive fields of simple cells (one class of V1 neurons) using a push-pull combination of receptive fields response, which is used to represent a monocular cue. Further, the receptive fields of complex cells (the other class of V1 neurons) are simulated by using binocular energy response and binocular rivalry response, which are used to represent a binocular cue. Subsequently, various quality-aware features are extracted from the response of area V1 by calculating the self-weighted histogram of the local binary pattern on four types of feature maps of similarity measurement that will change in the presence of distortions. Finally, kernel ridge regression is used to simulate a nonlinear relationship between the quality-aware features and objective quality scores. The performance of our method is evaluated over popular stereoscopic image databases and shown to be competitive with the state-of-the-art FR SIQA algorithms.

Binocular Responses for No-Reference 3D Image Quality Assessment

Perceptual quality assessment of distorted three-dimensional (3D) images has become a fundamental yet challenging issue in the field of 3D imaging. In this paper, we propose a general-purpose blind/no-reference (NR) 3D image quality assessment (IQA) metric that utilizes the complementary local patterns (the local magnitude pattern and the proposed generalized local directional pattern) of binocular energy response (BER) and binocular rivalry response (BRR). The main technical contribution of this research is that binocular visual perception and local structural distribution are considered for NR 3D-IQA. More specifically, the metric simulates the binocular visual perception using BER and BRR. Subsequently, the local patterns of the binocular responses’ encoding maps are used to form various binocular quality-predictive features, which will change in the presence of distortions. After feature extraction, we use k -nearest neighbors-based machine learning to drive the overall quality score. We tested our proposed metric against two publicly available 3D databases; these tests confirm that the proposed metric's results consistently align with human subjective judgments.

Models of Monocular and Binocular Visual Perception in Quality Assessment of Stereoscopic Images

Assessing quality of experience (QoE) for three-dimensional (3-D) video is challenging. In this paper, we propose a new full-reference stereoscopic image quality metric, by simulating the behaviors of visual perception with simple and complex receptive field properties and constructing the models of monocular and binocular visual perception. To be more specific, the stereoscopic images are first classified into noncorresponding and corresponding regions. Then, monocular energy responses are generated for the noncorresponding region based on stimuli from different spatial frequencies and orientations, and binocular energy responses are generated for the corresponding region based on stimuli from different spatial frequencies, orientations and disparities, respectively. Finally, gradient similarities between the energy responses of the original and distorted stereoscopic images are measured for noncorresponding and corresponding regions, respectively, and all results are fused to get an overall score. Experiments on three 3-D image quality assessment (IQA) databases demonstrate that in comparison with the most related existing methods, the devised algorithm achieves higher agreement with subjective assessment, making it better suited for the evaluation and optimization of stereoscopic image processing algorithms.

No-reference stereoscopic image quality measurement based on generalized local ternary patterns of binocular energy response

Perceptual no-reference (NR) quality measurement of stereoscopic images has become a challenging issue in three-dimensional (3D) imaging fields. In this article, we propose an efficient binocular quality-aware features extraction scheme, namely generalized local ternary patterns (GLTP) of binocular energy response, for general-purpose NR stereoscopic image quality measurement (SIQM). More specifically, we first construct the binocular energy response of a distorted stereoscopic image with different stimuli of amplitude and phase shifts. Then, the binocular quality-aware features are generated from the GLTP of the binocular energy response. Finally, these features are mapped to the subjective quality score of the distorted stereoscopic image by using support vector regression. Experiments on two publicly available 3D databases confirm the effectiveness of the proposed metric compared with the state-of-the-art full reference and NR metrics.

Perceptual quality measurement of 3D images based on binocular vision

Three-dimensional (3D) technology has become immensely popular in recent years and widely adopted in various applications. Hence, perceptual quality measurement of symmetrically and asymmetrically distorted 3D images has become an important, fundamental, and challenging issue in 3D imaging research. In this paper, we propose a binocular-vision-based 3D image-quality measurement (IQM) metric. Consideration of the 3D perceptual properties of the primary visual cortex (V1) and the higher visual areas (V2) for 3D-IQM is the major technical contribution to this research. To be more specific, first, the metric simulates the receptive fields of complex cells (V1) using binocular energy response and binocular rivalry response and the higher visual areas (V2) using local binary patterns features. Then, three similarity scores of 3D perceptual properties between the reference and distorted 3D images are measured. Finally, by using support vector regression, three similarity scores are integrated into an overall 3D quality score. Experimental results for two public benchmark databases demonstrate that, in comparison with most current 2D and 3D metrics, the proposed metric achieves significantly higher consistency in alignment with subjective fidelity ratings.

Blind Image Quality Assessment for Stereoscopic Images Using Binocular Guided Quality Lookup and Visual Codebook

The field of assessing three-dimensional (3-D) visual experience is challenging. In this paper, we propose a new blind image quality assessment for stereoscopic images by using binocular guided quality lookup and visual codebook. To be more specific, in the training stage, we construct phase-tuned quality lookup (PTQL) and phase-tuned visual codebook (PTVC) from the binocular energy responses based on stimuli from different spatial frequencies, orientations, and phase shifts. In the test stage, blind quality pooling can be easily achieved by searching the PTQL and PTVC, and the quality score is obtained by averaging the largest values of all patch’s quality. Experimental results on three 3-D image quality assessment databases demonstrate that in comparison with the most related existing methods, the devised algorithm achieves high consistency alignment with subjective assessment and low-complexity pooling.

Binocular energy response based quality assessment of stereoscopic images

Perceptual quality assessment in three-dimensional (3D) is challenging. In this research, we propose a binocular energy response based quality assessment metric of stereoscopic images. To be more specific, we first construct binocular energy responses of the original and distorted images, and measure the similarity between them as Image Quality Metric (IQM). Then, the binocular response and the binocular masking components are used to modulate the IQM, respectively. Finally, two evaluation results are nonlinearly integrated into an overall score by considering the importance of each component. Experimental results show that compared with the relevant existing metrics, the proposed metric can achieve higher consistency with the subjective assessment of stereoscopic image.

Binocular energy responses to natural images

Binocular energy responses to natural images

Binocular energy responses to natural images

The binocular energy model provides a good description of the first stages of cortical binocular processing. Three important determinants of the responses of neurons under this model are the disparity of a stimulus, its spatial variation in disparity and its second-order luminance statistics. The influence of the latter two factors on the disparity tuning of the energy model were investigated. While each can have a significant effect on the energy response, neither presents a significant challenge when one considers the range of variation expected in natural images. The response of the energy model to natural binocular images was also investigated. The strongest responses were found for model neurons tuned to small disparities. This trend was more evident for vertical than for horizontal disparity, and flattened rapidly as image eccentricity increased. These results are predicted on the basis of simple geometrical considerations, and are reflected in both physiological and psychophysical measures of the disparity tuning of the visual system.