Stereoscopic Video Research Articles

Analysis of User Requirements in Interactive 3D Video Systems

The recent development of three dimensional (3D) display technologies has resulted in a proliferation of 3D video production and broadcasting, attracting a lot of research into capture, compression and delivery of stereoscopic content. However, the predominant design practice of interactions with 3D video content has failed to address its differences and possibilities in comparison to the existing 2D video interactions. This paper presents a study of user requirements related to interaction with the stereoscopic 3D video. The study suggests that the change of view, zoom in/out, dynamic video browsing, and textual information are the most relevant interactions with stereoscopic 3D video. In addition, we identified a strong demand for object selection that resulted in a follow-up study of user preferences in 3D selection using virtual-hand and ray-casting metaphors. These results indicate that interaction modality affects users’ decision of object selection in terms of chosen location in 3D, while user attitudes do not have significant impact. Furthermore, the ray-casting-based interaction modality using Wiimote can outperform the volume-based interaction modality using mouse and keyboard for object positioning accuracy.

Three-Dimensional Video Composition with Firewire Camera

This paper presents a system for the capture and the composition of three-dimensional video with firewire cameras. We have developed a software application for capturing synchronized video, so that a couple of videos corresponding to the right and left eyes can be recorded in the hard disk. The main features of this application are: the execution of camera calibration procedure, the correction of image distortion caused by the lens radial distortion, and the synchronization of cameras to capture video through the firewire bus. The final composition of the stereoscopic video is accomplished through appropriate reference software. An experimental stereoscopic video were generated through this system as a proof of concept. The resulting video file was satisfactory regarding the visualization of three-dimensional effect of the content of the captured video. According to our free subjective evaluation, there was no distortion in the images or unpleasant effects.

Role of a collision-sensitive neuron during collision avoidance in flight

Event Abstract Back to Event Role of a collision-sensitive neuron during collision avoidance in flight Raymond W. Chan1*, Fabrizio Gabbiani1, 2 and Reid Harrison3 1 Baylor College of Medicine, United States 2 Rice University, United States 3 Intan Technologies, LLC, United States The neural mechanisms underlying collision avoidance in flight are largely unknown territory. Using a custom designed telemetry chip we record from the DCMD (Descending Contralateral Movement Detector), a collision detecting neuron, and flight muscles of a locust (Schistocerca americana) flying in a wind tunnel. Simultaneously, two high-speed video cameras are used to record flight data of the locust while it performs collision avoidance manoeuvres in response to the simulated approach of an object on a collision course with the animal (i.e., a looming stimulus). Using custom built software, we reconstruct in 3D flight parameters like body orientation and wing height from the stereoscopic videos. Neural data is then correlated with behavioural data to shed light on the neural basis of this behaviour. In addition, we design a filter-based system using the muscle and neural signals to control a virtual drone. Figure 1 Keywords: collision avoidance, DCMD, flight, grasshopper, Locust, Robotics, Wireless Neural Recording Conference: Tenth International Congress of Neuroethology, College Park. Maryland USA, United States, 5 Aug - 10 Aug, 2012. Presentation Type: Poster (but consider for student poster award) Topic: Sensory: Vision Citation: Chan RW, Gabbiani F and Harrison R (2012). Role of a collision-sensitive neuron during collision avoidance in flight. Conference Abstract: Tenth International Congress of Neuroethology. doi: 10.3389/conf.fnbeh.2012.27.00119 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 25 Apr 2012; Published Online: 07 Jul 2012. * Correspondence: Dr. Raymond W Chan, Baylor College of Medicine, Houston, United States, rchan@cns.bcm.edu Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Raymond W Chan Fabrizio Gabbiani Reid Harrison Google Raymond W Chan Fabrizio Gabbiani Reid Harrison Google Scholar Raymond W Chan Fabrizio Gabbiani Reid Harrison PubMed Raymond W Chan Fabrizio Gabbiani Reid Harrison Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.

Content-based subjective quality prediction in stereoscopic videos with machine learning

A model exploiting machine learning and content analysis is proposed to predict the subjective quality of stereoscopic videos. This model offers an automated, accurate and consistent subjective quality prediction. The feasibility and accuracy of the proposed technique has been thoroughly analysed with extensive subjective experiments and simulations. Results illustrate that a performance measure of 0.954 in subjective quality prediction can be achieved with the proposed technique.

A six-band stereoscopic video camera system for accurate color reproduction

A six-band stereoscopic video camera system for accurate color reproduction

Effect of Accommodation Training on Visual Function of Visual Inspection Workers and Middle-Aged People

By relaxing the contracted focus-adjustment muscles around the eyeball, such as the ciliary and extraocular muscles, improvement of the pseudo-myopia is expected. This understanding has led to the accommodation training in which the visual target is given by stereoscopic video clips. In this study, we verify short-term effects of the apparatus on eyesight of visual inspection workers (22 females) suffering from eye fatigue and 12 middle-aged persons. In the Measurement 1, the workers were trained in 3 days. Moreover, the middle-aged were investigated on several trials of the eyesight recovering apparatus in the Measurement 2. As a result, the visual acuity was statistically improved by continuous accommodation training which will promote a ciliary muscle-stretching effect.

HEVC를 이용한 양안식 영상의 효율적인 부호화 방법

차세대 비디오 압축 표준으로 현재 개발 중인 HEVC(high efficiency video coding)는 기존 비디오 압축 표준인 H.264/AVC에 비해 40% 정도의 성능 향상을 보인다. 기존의 양안식 영상을 부호화하기 위한 코텍은 H.264/AVC 기반으로 제작되었기 때문에, HEVC를 이용하여 양안식 영상을 부호화 하면 성능 향상을 기대할 수 있다. 또한 양안식 영상은 기존의 각 화면의 시간 상관도{temporal correlation)에 더해서 화면간의 상관도(interview correlation)가 존재하기 때문에, 좌우 영상을 함께 부호화하는 것이 성능 개선에 유리하다. 본 논문에서는 HEVC의 참조 소프트웨어인 HM 3.4를 이용하여 화면간의 상관도를 고려한 양안식 영상 부호화 방법을 제안한다. 화면간의 예측을 이용한 제안한 코텍을 이용해 부호화한 오른쪽 영상과 기존 HEVC로의 오른쪽 화면을 부호화했을 때의 성능을 비교한 결과 BDBR이 약 36.24% 감소했으며 BDPSNR은 1.19 dB가 증가했다. The compression performance of HEVC (high efficiency video coding) is improved 40%, compared to H.264/AVC. Since the existing 3D video CODEC is based on H.264/AVC or MPEG-2, we can improve the compression performance when we use the proposed stereoscopic video coding method based on HEVC. Since the stereoscopic video has the temporal and inter-view correlations, the videos of the left and right cameras encode together to improve the performance. Especially, we implemented the proposed technique using HM(HEVC test model) 3.4. To compare the performance of the proposed method, we only compare the right view video which is coded using the inter-view prediction. The proposed method which is considered inter-view correlation is improved the performance which BDBR reduce about 36.24% and BDPSNR increase approximately 1.19 dB compared to the separated-coding method.

3DTV 방송을 위한 스테레오스코픽 비디오 영상 및 전송 포맷 기술규격의 검증

멀티미디어산업에 있어서 3DTV 서비스를 위한 스테레오스코픽 비디오는 Post-HD(post Hi호 Definition) 시대의 주요 콘텐츠로 부상하고 있으며, 콘텐츠 제작, 방송 및 가전 등 관련 산업 전반에 걸쳐 파급을 일으키며 3DTV 서비스 인프라를 구축시키고 있다. 이에 국내에서는 3DTV 방송 송수신 정합규격 표준화를 목표로 TTA에서 3DTV 송수신 규격 실무반(WG8061)을 구성하여, "3DTV 방송을 위한 스테레오스코픽 비디오 영상 및 전송 포맷 기술규격"의 표준안을 작성하고 있다. 본 논문에서는 현재 표준화 작업이 진행 중인 상기 표준규격을 적용한 test-bed를 설계하여, 규격의 기능을 검증하고 기존의 기술과의 역호환성에 대한 검증을 하며 향후 보완 및 검토되어야 하는 기술적 문제들을 도출하고자 한다. Stereoscopic video for 3DTV service is emerging as a main content of the Post-HD age in the multimedia industry, and also it starts a ripple effect to build infrastructure for 3DTV service throughout the related industries such as creating contents, consumer electronics and the broadcasting. Thus, "Technical Specification of Stereoscopic Video over MPEG-2 TS for 3DTV Service" for providing a standard for 3DTV broadcasting transmitter and receiver interface is under development by the 3DTV WG8061 in ITA, Korea. This paper provides verification of the functionalities, problems and future work of the above specification with designed test-bed. Through making a test bed, this paper try to verify not only the functionality of standard but also backward compatible for the present technology and abstract problems to would be reviewed and supplemented.

Network convergence and QoS for future multimedia services in the VISION project

The emerging use of real-time 3D-based multimedia applications imposes strict quality of service (QoS) requirements on both access and core networks. These requirements and their impact to provide end-to-end 3D videoconferencing services have been studied within the Spanish-funded VISION project, where different scenarios were implemented showing an agile stereoscopic video call that might be offered to the general public in the near future. In view of the requirements, we designed an integrated access and core converged network architecture which provides the requested QoS to end-to-end IP sessions. Novel functional blocks are proposed to control core optical networks, the functionality of the standard ones is redefined, and the signaling improved to better meet the requirements of future multimedia services. An experimental test-bed to assess the feasibility of the solution was also deployed. In such test-bed, set-up and release of end-to-end sessions meeting specific QoS requirements are shown and the impact of QoS degradation in terms of the user perceived quality degradation is quantified. In addition, scalability results show that the proposed signaling architecture is able to cope with large number of requests introducing almost negligible delay.

서비스호환 3DTV 방송서비스를 위한 시그널링 및 다중화기법

This paper proposes a signaling and multiplexing mechanism in service-compatible 3DTV broadcasting for guaranteeing backward-compatibility with legacy DTV. Recently, 3DTV experimental broadcasting services are being developed around advanced countries and the standardization of service-compatible 3DTV broadcasting which is able to provide high-quality stereoscopic videos in progress. Based on experimental results, we confirm that proposed method can be used as a common technology and a reference model for stereoscopic video transmission over terrestial, satellite and cable DTV platforms in 3DTV broadcasting service and system development.

Human brain functional MRI and DTI visualization with virtual reality.

Magnetic resonance diffusion tensor imaging (DTI) and functional MRI (fMRI) are two active research areas in neuroimaging. DTI is sensitive to the anisotropic diffusion of water exerted by its macromolecular environment and has been shown useful in characterizing structures of ordered tissues such as the brain white matter, myocardium, and cartilage. The diffusion tensor provides two new types of information of water diffusion: the magnitude and the spatial orientation of water diffusivity inside the tissue. This information has been used for white matter fiber tracking to review physical neuronal pathways inside the brain. Functional MRI measures brain activations using the hemodynamic response. The statistically derived activation map corresponds to human brain functional activities caused by neuronal activities. The combination of these two methods provides a new way to understand human brain from the anatomical neuronal fiber connectivity to functional activities between different brain regions. In this study, virtual reality (VR) based MR DTI and fMRI visualization with high resolution anatomical image segmentation and registration, ROI definition and neuronal white matter fiber tractography visualization and fMRI activation map integration is proposed. Rationale and methods for producing and distributing stereoscopic videos are also discussed.

A lightweight augmented virtuality system for providing a faithful and spatially manipulable visual hand representation.

This paper introduces the technical foundations of a system designed to embed a lightweight, faithful and spatially manipulable representation of the user's hand into an otherwise virtual world - Augmented Virtuality (AV). A highly intuitive control during pointing-like near space interaction can be provided to the user, as well as a very flexible means to experimenters, in a variety of non-medical and medical contexts. Our approach essentially relies on stereoscopic video see-through Augmented Reality (AR) technology and a generic, extendible framework for managing 3-D visual hand displacements. Research from human-computer interaction, perception and motor control has contributed to the elaboration of our proposal which combines a) acting in co-location, b) avoiding occlusion violations by assuring a correct scene depth ordering and c) providing a convincing visual feedback of the user's hand. We further present two cases in which this system has already successfully been used and then outline some other applications that we think are promising, for instance, in the fields of neuromotor rehabilitation and experimental neuroscience.

An Objective Video Quality Metric for Compressed Stereoscopic Video

Stereoscopic video create the impression of depth by using two slightly different viewpoints. Conventional video quality assessment methods for 2D video are not suitable for stereoscopic video, so a new video quality assessment model for stereoscopic video is needed. In this paper, we propose a new objective video quality metric for compressed stereoscopic video. The proposed algorithm uses blocking artifacts, blurring in edge regions and video quality difference between two views. Blocking artifacts and blurring in edge regions are distortion appeared in the compressed video, and they are widely used in conventional video quality model. Difference in video quality between two views considers 3D effect of stereoscopic video. To verify the performance of the proposed algorithm, we performed subjective evaluation of stereoscopic video, and we confirmed that the proposed algorithm is superior to the conventional algorithms in respect to correlation with the subjective evaluation.

A novel rate control technique for asymmetric-quality stereoscopic video

Three-dimensional (3D) video technology is becoming increasingly popular, as it can provide stereoscopic perception and immersive experience to end users. Some asymmetric stereoscopic video coding methods had been developed by utilizing binocular psycho-visual redundancy. However, rate control problem had not been well considered in these methods to further control the quality of stereoscopic video. In this paper, a novel rate control method is proposed for asymmetric-quality stereoscopic video. In order to model the asymmetric-quality stereoscopic video, we use a fixed threshold to quantize the binocular psycho-visual redundancy and establish the relationship between the distortion and quantization for the asymmetric-quality stereoscopic video. Then, a rate control method is engineered in stereoscopic video coding to control the rate and quality of left and right views. Experimental results show that the proposed method can accurately control the rate and quality of stereoscopic video, while having a lower computational complexity1.

Error concealment for stereoscopic video using illumination compensation

Illumination changes of different views of the 3D video can cause performance degradation in error concealment. In this paper we propose a new method of the error concealment for stereoscopic video to overcome this problem. We explore four implementations of an inter-view macro block (MB) matching algorithm which successfully compensates for illumination changes between stereo pairs. Experimental results show that our proposed method can obtain average gain of approximately 1.9dB compared to the Joint Model of H.264/AVC (JM) method for various test sequences and different MB loss ratio1.

Hybrid Codec-Based Intra-Frame Joint Rate Control for Stereoscopic Video

An intra-frame joint rate control scheme is first proposed for a hybrid coder to encode stereoscopic video, which is based on an optimization framework with a gradient-based quadratic rate-quantization model and a gradient-based linear distortion-quantization model. The proposed rate control scheme jointly works on the left and right encoders for stereoscopic video input by controlling the output bit rates of both encoders in the sense that the sum of the two decoded video qualities is maximized and the quality difference is maintained around a desired level for a given target bit budget at the same time. In experiments, the proposed intra-frame joint rate control scheme for the hybrid coder produces the average 0.62 dB gain in PSNR, 64.93% reduction in the mean PSNR differences and 72.04% reduction in the MSE of PSNR difference, compared with the independent rate control schemes of the MPEG-2 TM5 and H.264 JM 16.

Content generation for 3D video/TV

A lack of suitable 3D content currently constitutes a major bottleneck for transferring the recent success of 3D cinema to our home TV. In this paper we describe a comprehensive 3D content generation system which enables to generate 3D content from either stereoscopic or monoscopic input videos. In particular, we present a method to convert original 2D image sequences to 3D content by adding depth information with only little user support. Furthermore, we show results of a stereo algorithm which provides the basis for automatic conversion of stereoscopic film material for viewing on different types of displays. In this context, we discuss the potential of inpainting techniques for filling in image regions that were originally occluded.

고화질 스테레오스코픽 비디오 방송서비스를 위한 MPEG-2 전송스트림과 MPEG-C part 3의 확장 방안

현재 대중화 되고 있는 디지털 방송 이후의 발전방향인 Post_HD(High-Definition)서비스로 3DTV(Three Dimensional Television)가 대두되고 있다. 이와 같은 3DTV 서비스는 스테레오스코픽 영상을 구성하는 방법에서 이를 기존의 방송망에서 서비스하기 위한 방법까지 다양한 방법들이 연구되어지고 있다. 본 논문에서는 3DTV 서비스를 제공하기 위한 다양한 비디오 영상의 구성 방법에 모두 적용 가능한 전송방법으로 MPEG-2 전송스트림의 PMT에 스테레오스코픽 기술자를 추가한 방안과 MPEG-C part 3의 확장을 통해 스테레오스코픽 비디오를 전송하는 방안의 분석하여 고화질 스테레오스코픽 비디오 방송서비스 방안을 제안한다. 제안된 기술은 기존의 3D 방송을 지원하지 않는 단말에서도 기준영상만을 출력하여 기존의 MPEG-2 전송스트림과 호환성을 유지하고 있다. 따라서 기존의 디지털 방송과 스테레오 스코픽 영상 서비스 지원이 가능한 규격으로써 향후 통신과 방송 영역에서 스테레오스코픽 기반 3D 서비스 활성화에 기여할 것이다. Currently, 3DTV technologies are being developed as the future services of the HD digital broadcast environment. As one of the various research topics to apply 3DTV technologies to the conventional broadcasting network, methods to configure stereoscopic videos are being studied. In this paper, we proposed a method to broadcast high quality stereoscopic videos based on analysis of a method to add a stereoscopic descriptor to the PMT of MPEG-2 transport streams and a method to transmit stereoscopic videos by the expansion of MPEG-C part 3 which are from precedent studies. The proposed technique maintains compatibility with conventional MPEG-2 transport streams by showing only reference video for models that do not support 3D broadcasting. Therefore, the compatibility between conventional broadcasting and stereoscopic videos should make this method useful when activating 3D services in the communications and broadcasting area

Experimental Investigations of Pupil Accommodation Factors

PURPOSE. The contraction and dilation of the iris muscle that controls the amount of light entering the retina causes pupil accommodation. In this study, experiments were performed and two of the three factors that influence pupil accommodation were analyzed: lighting conditions and depth fixations. The psychological benefits were not examined, because they could not be quantified. METHODS. A head-wearable eyeglasses-based, eye-capturing device was designed to measure pupil size. It included a near-infrared (NIR) camera and an NIR light-emitting diode. Twenty-four subjects watched two-dimensional (2D) and three-dimensional (3D) stereoscopic videos of the same content, and the changes in pupil size were measured by using the eye-capturing device and image-processing methods: RESULTS. The pupil size changed with the intensity of the videos and the disparities between the left and right images of a 3D stereoscopic video. There was correlation between the pupil size and average intensity. The pupil diameter could be estimated as being contracted from approximately 5.96 to 4.25 mm as the intensity varied from 0 to 255. Further, from the changes in the depth fixation for the pupil accommodation, it was confirmed that the depth fixation also affected accommodation of pupil size. CONCLUSIONS. It was confirmed that the lighting condition was an even more significant factor in pupil accommodation than was depth fixation (significance ratio: approximately 3.2:1) when watching 3D stereoscopic video. Pupil accommodation was more affected by depth fixation in the real world than was the binocular convergence in the 3D stereoscopic display.

Frame loss concealment for stereoscopic video plus depth sequences

In this work, an efficient frame loss concealment algorithm for stereoscopic video plus depth sequences is proposed. To conceal an erroneous left color frame, we extrapolate motion vectors for the temporal error concealment. To conceal a left depth frame, we exploit the motion vectors of the corresponding left color frame. To conceal a right color frame, we employ a 3D image warping technique to determine matching pixels between inter-views, and perform the reconstruction based on the similarities of the motion vectors and the intensity differences of matching pixels. Finally, to conceal a right depth frame, we use the motion vectors of the right color frame and the left color frame. Simulation results show that the proposed algorithm is more effective than conventional algorithms in recovering erroneous color and depth frames. Moreover, it is also demonstrated that the proposed algorithm can be used for efficient virtual view synthesis in error prone environments.