Stereoscopic Video Research Articles

A Spatio-Temporal Perceptual Quality Index Measuring Compression Distortions of Three-Dimensional Video

Objective video quality assessment (VQA) of stereoscopic three-dimensional (3-D) video plays a vital role in the application of video compression and transmission. This letter presents an efficient full-reference metric, called 3-D perceptual quality index (3-D-PQI), to measure video compression distortion of stereoscopic videos. In the metric, local video compression distortions in left and right view are measured in both spatial and temporal channels by considering contrast and motion masking effect. Then, a stereo saliency based pooling strategy is used to accumulate local spatial and temporal distortions. Finally, 3-D-PQI is derived from texture energy based fusion of distortion measurement of left and right view. A verification test of the proposed metric is conducted on publicly available stereoscopic VQA databases, which shows that 3-D-PQI is superior to other VQA metrics with respect to correlation against subjective quality judgment.

The History of Stereoscopic Video Games for the Consumer Electronic Market

This paper focuses on at-home electronic gaming systems with true stereoscopic displays that were produced for the consumer electronic market and made available for retail purchase and aims to be the most accurate historical account to date. Systems which floor set in retailers and achieved some amount of consumer sell through will be organized chronologically by the date of their release, establishing an accurate timeline in which a historical account may be presented. The technical specifications, hardware designs, user interfaces and software releases will be summarized and determine which stereoscopic category each fall into based on the specific format the medium's content is presented in and device in which it is displayed.

Virtual simulation of a late antique shipwreck at Marzamemi, Sicily: Integrated processes for 3D documentation, analysis and representation of underwater archaeological data

AbstractThree‐dimensional (3D) models derived from digital survey techniques have increasingly become a mainstay of archaeological research and cultural heritage management. The high accuracy of such modelling makes it an attractive solution for a wide range of challenges from site recording and interpretation to object analysis and reconstruction. The present paper focuses on a new 3D digitization method using stereoscopic video for the documentation, analysis, and representation of archaeological contexts as part of shipwreck investigations off southeast Sicily at Marzamemi. This sixth‐century ce vessel sank in shallow water (7–8 m) while carrying a massive cargo of largely prefabricated architectural elements intended for the construction and decoration of a church somewhere in the late antique west. This dynamic site presents significant challenges to the interpretation of depositional and post‐depositional events as well the reconstruction of the original cargo and individual architectural elements. Therefore, 3D documentation underwater at Marzamemi has centred on new methods for stereoscopic scanning of the site, topography, and large architectural finds, allowing not only more rapid and precise mapping but insights into site formation processes and the organizational mechanisms behind the cargo assemblage. The adoption of this new system based on a precise workflow and simple, inexpensive hardware combined with readily available software allows more accurate and rapid digital recording underwater to a high standard across scales, from the site level down to the individual artefact. This process, named ISU3D (Integrated System for Underwater 3D Digitization), offers innovative solutions not only for archaeological field research but for heritage management and public outreach.

Special Input Devices Integration to LIRKIS CAVE

Abstract LIRKIS CAVE is a unique immersive virtual reality installation with a compact cylinder-based construction and a high-quality stereoscopic video output rendered on twenty 55” Full HD LCD panels. While the video output of the CAVE provides a high level of immersion to a virtual world, its original implementation of peripherals support had a negative impact because of a limited number of supported devices and certain performance issues. In this paper we describe a new, distributed, peripheral devices support implementation for the LIRKIS CAVE, which solves the performance issues and allows for ease of integration of new input devices into the CAVE. We also present a successful integration of a special input device, the Myo armband,which allows a natural and unobtrusive gesture-based control of virtual environments. The integration includes a newly developed control and monitoring application for the Myo, called MLCCS, which utilization is not limited to CAVE systems or virtual reality applications.

The Role of Camera Convergence in Stereoscopic Video See-through Augmented Reality Displays

In the realm of wearable augmented reality (AR) systems, stereoscopic video see-through displays raise issues related to the user’s perception of the three-dimensional space. This paper seeks to put forward few considerations regarding the perceptual artefacts common to standard stereoscopic video see-through displays with fixed camera convergence. Among the possible perceptual artefacts, the most significant one relates to diplopia arising from reduced stereo overlaps and too large screen disparities. Two state-of-the-art solutions are reviewed. The first one suggests a dynamic change, via software, of the virtual camera convergence, whereas the second one suggests a matched hardware/software solution based on a series of predefined focus/vergence configurations. Potentialities and limits of both the solutions are outlined so as to provide the AR community, a yardstick for developing new stereoscopic video see-through systems suitable for different working distances.

Technical note: using ISS videos in Earth observation – implementations for science and education

ABSTRACTA large variety of various passive and active satellite sensors producing panchromatic, multispectral or hyperspectral images of the Earth’s surface are currently in space and are used in different scientific fields such as earth science (e.g. geography, hydrology, geology, oceanography and glaciology), disaster management, as well as for military, commercial and economic purposes. In contrast, video files are a rather exotic data format in the field of Earth observation. Occasionally, video cameras are used in airborne remote sensing, but only recently video Earth observation from space has been established. This paper aims at the integration of video data in the scientific workflow, revealing advantages and disadvantages of moving images. Being the only available source for continuous video Earth observation data, the NASA experiment High Definition Earth Viewing on-board the International Space Station is the basis for our evaluation of the usability of video data. Following the rather coarse resolution of these data, we exemplify some potential fields of application in science and education. We show a basic workflow how to produce 3D models and stereoscopic videos of the Earth’s surface. As a pre-study for videos with better spatial as well as radiometric resolution, the delivered products serve not only scientific purposes, but are integrated in school lessons to evoke the pupils’ fascination for earth sciences and space.

The New Moneyball: How Ballpark Sensors Are Changing Baseball

Advancements in the capability of sensors, processors, and storage devices have led to an explosion in the amount of data that is captured during sporting events. The Statcast system, for example, uses Doppler radar and stereoscopic video from two arrays of high-resolution optical imagers to acquire seven terabytes of data during each Major League Baseball (MLB) game. One use of these data is to enhance the experience of sports fans. As a case in point, Statcast data are used to generate information and visualizations that are disseminated in real time through telecasts and other media such as an app which displays pitch parameters derived from sensor measurements.

Escaping blood-fed malaria mosquitoes minimize tactile detection without compromising on take-off speed.

To escape after taking a blood meal, a mosquito must exert forces sufficiently high to take off when carrying a load roughly equal to its body weight, while simultaneously avoiding detection by minimizing tactile signals exerted on the host's skin. We studied this trade-off between escape speed and stealth in the malaria mosquito Anopheles coluzzii using 3D motion analysis of high-speed stereoscopic videos of mosquito take-offs and aerodynamic modeling. We found that during the push-off phase, mosquitoes enhanced take-off speed using aerodynamic forces generated by the beating wings in addition to leg-based push-off forces, whereby wing forces contributed 61% of the total push-off force. Exchanging leg-derived push-off forces for wing-derived aerodynamic forces allows the animal to reduce peak force production on the host's skin. By slowly extending their long legs throughout the push-off, mosquitoes spread push-off forces over a longer time window than insects with short legs, thereby further reducing peak leg forces. Using this specialized take-off behavior, mosquitoes are capable of reaching take-off speeds comparable to those of similarly sized fruit flies, but with weight-normalized peak leg forces that were only 27% of those of the fruit flies. By limiting peak leg forces, mosquitoes possibly reduce the chance of being detected by the host. The resulting combination of high take-off speed and low tactile signals on the host might help increase the mosquito's success in escaping from blood-hosts, which consequently also increases the chance of transmitting vector-borne diseases, such as malaria, to future hosts.

Extraction technique of region of interest from stereoscopic video

A feature fusion approach is presented to extract the region of interest (ROI) from the stereoscopic video. [0]Based on human vision system (HVS), the depth feature, the color feature and the motion feature are chosen as vision features. [0]The algorithm is shown as follows. Firstly, color saliency is calculated on superpixel scale. Color space distribution of the superpixel and the color difference between the superpixel and background pixel are used to describe color saliency and color salient region is detected. Then, the classic visual background extractor (Vibe) algorithm is improved from the update interval and update region of background model. The update interval is adjusted according to the image content. The update region is determined through non-obvious movement region and background point detection. So the motion region of stereoscopic video is extracted using improved Vibe algorithm. The depth salient region is detected by selecting the region with the highest gray value. Finally, three regions are fused into final ROI. Experiment results show that the proposed method can extract ROI from stereoscopic video effectively. In order to further verify the proposed method, stereoscopic video coding application is also carried out on the joint model (JM) encoder with different bit allocation in ROI and the background region.

Perceptually Driven Nonuniform Asymmetric Coding of Stereoscopic 3D Video

Asymmetric stereoscopic video coding has already proven its effectiveness in reducing the bandwidth required for stereoscopic 3D delivery without degrading the visual quality. This approach, in which the left and right views are encoded with different levels of quality, relies on the perceptual theory of binocular suppression. However, to ensure comfortable 3D viewing, the just-noticeable level of asymmetry, i.e., the maximum quality gap between views, has to be carefully defined. Both subjectively and empirically fixed thresholds of asymmetry demonstrated either the maladjustment to content or dependency to the experimental design. This paper describes a new nonuniform asymmetric stereoscopic video coding method adaptively adjusting the level of asymmetry for each region of the image based on its perceptual significance. The proposed method uses a fully automated model that dynamically determines the best bounds of asymmetry for which the 3D viewing experience will not be altered. This is achieved by exploiting several human-visual-system-inspired models, namely, the binocular just-noticeable difference, and the visual saliency map and depth information. The simulation results show that the proposed method results in bit rate saving of up to 26% and provides better 3D visual quality compared with state-of-the-art asymmetric coding methods.

Design of an experimental four-camera setup for enhanced 3D surface reconstruction in microsurgery

Abstract Future fully digital surgical visualization systems enable a wide range of new options. Caused by optomechanical limitations a main disadvantage of today’s surgical microscopes is their incapability of providing arbitrary perspectives to more than two observers. In a fully digital microscopic system, multiple arbitrary views can be generated from a 3D reconstruction. Modern surgical microscopes allow replacing the eyepieces by cameras in order to record stereoscopic videos. A reconstruction from these videos can only contain the amount of detail the recording camera system gathers from the scene. Therefore, covered surfaces can result in a faulty reconstruction for deviating stereoscopic perspectives. By adding cameras recording the object from different angles, additional information of the scene is acquired, allowing to improve the reconstruction. Our approach is to use a fixed four-camera setup as a front-end system to capture enhanced 3D topography of a pseudo-surgical scene. This experimental setup would provide images for the reconstruction algorithms and generation of multiple observing stereo perspectives. The concept of the designed setup is based on the common main objective (CMO) principle of current surgical microscopes. These systems are well established and optically mature. Furthermore, the CMO principle allows a more compact design and a lowered effort in calibration than cameras with separate optics. Behind the CMO four pupils separate the four channels which are recorded by one camera each. The designed system captures an area of approximately 28mm × 28mm with four cameras. Thus, allowing to process images of 6 different stereo perspectives. In order to verify the setup, it is modelled in silico. It can be used in further studies to test algorithms for 3D reconstruction from up to four perspectives and provide information about the impact of additionally recorded perspectives on the enhancement of a reconstruction.

Using Bold-fMRI to detect cortical areas and visual fatigue related to stereoscopic vision

PurposeThe location of cortical areas and visual fatigue related to stereoscopic vision were explored by two types of fMRI designs, namely block stimulation and event-related stimulation. The stimulations consist of 2D/3D images and different depths of the stereoscopic 3D images. Method20 normal subjects were randomly divided into the block group and the event-related group. Blood oxygenation level-dependent functional magnetic resonance imaging (Bold-fMRI) was performed in two groups. Functional data was preprocessed and statistically analyzed by SPM8. The result was reported by REST. ResultIn the block stimulation group, compared to 2D image stimulation, 3D image stimulation results in more activated brain areas, including frontal lobes, occipital lobes and limbic lobes, especially in the frontal eye field (Brodmann Area 8, BA8) and middle occipital gyrus (BA18/19). In the event-related group, compared to 2D images, viewing 3D images causes significant activations in temporal lobe, mainly represented in BA19/13/31/37. Additionally, 3D image stimulation with the focus set at front depth can lead to the activation of more brain areas compared to the back depth, including inferior parietal lobule and posterior central gyrus. ConclusionThe formation of the stereoscopic vision requires the collaboration of more brain areas, so that viewing stereoscopic videos for a long period may result in visual fatigue; meanwhile, the front depth of field can contribute to more activated brain areas than the back depth of field. As a parameter of stereoscopic images, it is valid to state that the depth of field may affect visual fatigue.

3DTV at home

Stereoscopic 3D (S3D) movies have become widely popular in the movie theaters, but the adoption of S3D at home is low even though most TV sets support S3D. It is widely believed that S3D with glasses is not the right approach for the home. A much more appealing approach is to use automulti-scopic displays that provide a glasses-free 3D experience to multiple viewers. A technical challenge is the lack of native multiview content that is required to deliver a proper view of the scene for every viewpoint. Our approach takes advantage of the abundance of stereoscopic 3D movies. We propose a real-time system that can convert stereoscopic video to a high-quality multiview video that can be directly fed to automultiscopic displays. Our algorithm uses a wavelet-based decomposition of stereoscopic images with per-wavelet disparity estimation. A key to our solution lies in combining Lagrangian and Eulerian approaches for both the disparity estimation and novel view synthesis, which leverages the complementary advantages of both techniques. The solution preserves all the features of Eulerian methods, e.g., subpixel accuracy, high performance, robustness to ambiguous depth cases, and easy integration of inter-view aliasing while maintaining the advantages of Lagrangian approaches, e.g., robustness to large disparities and possibility of performing non-trivial disparity manipulations through both view extrapolation and interpolation. The method achieves real-time performance on current GPUs. Its design also enables an easy hardware implementation that is demonstrated using a field-programmable gate array. We analyze the visual quality and robustness of our technique on a number of synthetic and real-world examples. We also perform a user experiment which demonstrates benefits of the technique when compared to existing solutions.

Multiview Stereoscopic Video Hole Filling Considering Spatiotemporal Consistency and Binocular Symmetry for Synthesized 3D Video

This paper proposes a new hole-filling method with spatiotemporal consistency and binocular symmetry for synthesized 3D videos in view extrapolation. Disocclusion regions in the synthesized views at virtual viewpoints result in regions with missing content. These regions will be referred to as hole regions. To provide the high-quality synthesized 3D videos via 3D display, the hole regions need to be filled considering the characteristics of human visual perception. From the perceptual point of view, binocular asymmetry between synthesized left- and right-eye videos (i.e., stereo pair video) is one of the most important factors that induce visual discomfort in stereoscopic viewing. In addition, binocular symmetry without temporal consistency between temporally neighboring frames could cause visual discomfort by annoying flickering artifacts. In this paper, to maintain the spatiotemporal consistency and binocular symmetry in synthesized 3D videos at multiple virtual viewpoints, we propose a global optimization-based hole-filling method using the information from the already filled adjacent view and previous frame. Furthermore, to reduce the computational cost of the global optimization, we propose a label propagation method, which propagates reliable labels used in the adjacent view and previous frame to the target image to be filled. The performance of the proposed method has been evaluated by objective assessments of 3D image quality, temporal consistency, and computational efficiency. In addition, subjective assessment is conducted for measuring the visual comfort and overall quality. The experimental results proved that the proposed method provides hole filling results with spatiotemporal consistency and binocular symmetry.

Visual Attention Modeling for Stereoscopic Video: A Benchmark and Computational Model

In this paper, we investigate the visual attention modeling for stereoscopic video from the following two aspects. First, we build one large-scale eye tracking database as the benchmark of visual attention modeling for stereoscopic video. The database includes 47 video sequences and their corresponding eye fixation data. Second, we propose a novel computational model of visual attention for stereoscopic video based on Gestalt theory. In the proposed model, we extract the low-level features, including luminance, color, texture, and depth, from discrete cosine transform coefficients, which are used to calculate feature contrast for the spatial saliency computation. The temporal saliency is calculated by the motion contrast from the planar and depth motion features in the stereoscopic video sequences. The final saliency is estimated by fusing the spatial and temporal saliency with uncertainty weighting, which is estimated by the laws of proximity, continuity, and common fate in Gestalt theory. Experimental results show that the proposed method outperforms the state-of-the-art stereoscopic video saliency detection models on our built large-scale eye tracking database and one other database (DML-ITRACK-3D).

Perceptual stereoscopic video coding using disparity just-noticeable-distortion model

In this paper, we propose perceptual stereoscopic video coding using a disparity just-noticeable-distortion (JND) model. We obtain the disparity JND model in stereo videos by disparity masking effects of the human visual system (HVS). The disparity JND model represents the maximum distortion of stereo perception that HVS cannot perceive. Based on the disparity JND model, we adjust prediction residuals to remove the perceptual redundancy of stereo videos. Thus, we achieve significant bit-rate saving while maintaining visual quality. Experimental results demonstrate that the proposed method significantly improves coding efficiency without loss of stereoscopic perceptual quality.

PERANCANGAN VIDEO STEREOSCOPIC 3D DENGAN 2 KAMERA SMARTPHONE MENGGUNAKAN METODE COLOR ANAGLYPH

3D Stereoscopy also called stereoscopic or 3D imaging refers to a technique for creating or enhancing the illusion of depth to the picture by presenting two offset images separately to the left and right eye of the viewer. Two-dimensional images are then combined in the brain to give the perception of 3D depth. One method to represent the level of depth with stereoscopic. Two stereoscopic images are stacked, there will be the so-called image disparity (disparity image) that is separate duplicate images because of the different parallax. Disparity image is created because of stereobase, ie the distance between the camera lens axis. The distance between the two cameras, can be analogous to the distance of the two eyes are large 65 mm. Factors that affect the stereoscopic video is video processing performed by optimization of standard anaglyph, with color correction and gamma level of the video, shooting technique and the distance between the object lens with the camera, it is supported by the change in value significantly between techniques establishing shot (ES), tracking shot (TS) and a panning shot (PS). tracking shot (TS). distance between an object lens with the camera also has an effect, on the first visible value of 75 lux at a distance object with camera lenses over 20 meters were taken using a technique establishing shot (ES) has a value score an average of 4.6 to 6.8 while tracking shot (TS) have the highest scores, with an average score of 6.6 to 9, while for objects recorded average value is the highest PK (Pusat Keramaian) object with an average score of 2.6 object followed by a PA (Pemandangan Alam) object with an average score of 2.4. Keyword : stereoscopic, disparity, parallax, stereobase, gamma.

A no-reference optical flow-based quality evaluator for stereoscopic videos in curvelet domain

Most of the existing 3D video quality assessment (3D-VQA/SVQA) methods only consider spatial information by directly using an image quality evaluation method. In addition, a few take the motion information of adjacent frames into consideration. In practice, one may assume that a single data-view is unlikely to be sufficient for effectively learning the video quality. Therefore, integration of multi-view information is both valuable and necessary. In this paper, we propose an effective multi-view feature learning metric for blind stereoscopic video quality assessment (BSVQA), which jointly focuses on spatial information, temporal information and inter-frame spatio-temporal information. In our study, a set of local binary patterns (LBP) statistical features extracted from a computed frame curvelet representation are used as spatial and spatio-temporal description, and the local flow statistical features based on the estimation of optical flow are used to describe the temporal distortion. Subsequently, a support vector regression (SVR) is utilized to map the feature vectors of each single view to subjective quality scores. Finally, the scores of multiple views are pooled into the final score according to their contribution rate. Experimental results demonstrate that the proposed metric significantly outperforms the existing metrics and can achieve higher consistency with subjective quality assessment.

Using stereoscopic video cameras to evaluate seagrass meadows nursery function in the Mediterranean

Seagrasses provide an important ecological value as nursery habitats, hosting higher juvenile densities than their surrounding less-structured habitats by offering shelter and food to early stages of fish. However, the quantitative evaluations of this nursery function remain elusive. Surveys assessing abundances of juvenile fish over seagrass meadows have been largely conducted by diving visual censuses, where typically only a shallow depth range is adequately covered. Within this study, we developed a low-cost stereo-baited video camera (SBRUV) with high precision and accuracy able to deliver length frequency distributions, including the smallest juvenile fraction. The capacity of the SBRUV system increased the synopticity with respect to typical surveys. We tested the system over seagrass meadows of Posidonia oceanica in a Mediterranean bay, investigating the relationship of juvenile abundance and size of the sparid Diplodus annularis with depth, time of day and protection status (i.e. inside and outside of a Marine Protected Area, MPA). We found significant effects of the depth on the length of the older size classes fraction (larger sizes at deeper stations) and an effect of time of the day on the abundance (less abundant during the evenings), and an opposite pattern was observed for early juvenile’s abundances. The MPA protection had no effect in D. annularis population structure. Interestingly, the relative abundance of D. annularis early juveniles was comparable at all depths, from 2 to 20 m, which suggests a potentially higher nursery value of P. oceanica meadows than earlier thought.

Row-Interleaved Sampling for Depth-Enhanced 3D Video Coding for Polarized Displays

Passive stereoscopic displays create the illusion of three dimensions by employing orthogonal polarizing filters and projecting two images onto the same screen. In this article, a coding scheme targeting depth-enhanced stereoscopic video coding for polarized displays is introduced. We propose to use asymmetric row-interleaved sampling for texture and depth views prior to encoding. The performance of the proposed scheme is compared with several other schemes, and the objective results confirm the superior performance of the proposed method. Furthermore, subjective evaluation proves that no quality degradation is introduced by the proposed coding scheme compared to the reference method.