Stereoscopic Content Research Articles

SMPTE Activities in Stereoscopic Content [Standards Report from the Engineering Vice President

As stereoscopic content production, distribution, and presentation infrastructure is rolled out in Digital Cinema theaters and broadcasting organizations around the world, market acceptance of 3D content is forcing a need for new technology standards for stereoscopic content.

Guidelines for an improved quality of experience in 3‐D TV and 3‐D mobile displays

AbstractStereoscopic (3‐D) movies have become widely popular all over the world. In addition, 3‐D TVs and mobile devices have already been introduced to the consumer market. However, while some manufacturers are introducing 3‐D cameras and movie studios are using proprietary solutions, there are no guidelines for consistently capturing high‐quality stereoscopic content. In this paper, a comprehensive stereoscopic image and video database with content captured at various distances from the camera lenses and under different lighting conditions will be presented. Subjective tests to assess the perceived 3‐D quality of these videos and images, which were shown on displays of different sizes, have been conducted. In addition, the horizontal parallax of the content was adjusted to verify via subjective tests whether this change could increase the viewer's quality of experience. Finally, guidelines of acquisition distances between the cameras and the real scene will be published.

깊이정보를 활용한 입체 편집 프로세스 연구

The 3D stereoscopic image contents have been emerging as the blue chip of the contents market of the next generation since the . However, all the 3D contents created commercially in the country have failed to enter box office. It is because the quality of Korean 3D contents is much lower than that of overseas contents and also current 3D post production process is based on 2D. Considering all these facts, the 3D editing process has connection with the quality of contents. The current 3D editing processes of the production case of are using the way that edits with the system on basis of 2D, followed by checking with 3D display system and modifying, if there are any problems. In order to improve those conditions, I suggest that the 3D editing process contain more objectivity by visualizing the depth data applied in some composition work such as Disparity map, Depth map, and the current 3D editing process. The proposed process has been used in the music drama , comparing with those of the film . The 3D values could be checked among cuts which have been changed a lot since those of , while the 3D value of drew an equal result in general. Since the current process is based on an artist`s subjective sense of 3D, it could be changed according to the condition and state of the artist. Furthermore, it is impossible for us to predict the positive range, so it is apprehended that the cubic effect of space might be perverted by showing each different 3D value according to cuts in the same space or a limited space. On the other hand, the objective 3D editing by applying the visualization of depth data can adjust itself to the cubic effect of the same space and the whole content equally, which will enrich the 3D contents. It will even be able to solve some problems such as distortion of cubic effect and visual fatigue, etc.

Stereoscopic Volume Perception

In the past five years, technological improvements have made it possible to capture, modify, and present technically good stereoscopic images. With devices able to capture high-resolution digital pictures and robust computer vision algorithms in post-production, today we are able to produce high-quality live-action stereoscopic content—from a technical point of view. This paper tackles an analytical question living in the art domain of stereoscopic cinema: When does a ball look flat like a frisbee disk, and when does it appear elongated like an egg? To analyze volume perception, we introduce a simple but powerful basic concept: A 1×1×1 m cube is used to virtually sweep through space away from the observer's eyes to infinity. For each distance, we measure the angles of width and depth on the observer's retina, which cause the retinal disparity that elicits stereopsis. This ratio leads to a width-to-depth sweep through space (width-to-depth ratio versus distance). The same test setup is then duplicated. Instead of the eyes two cameras are connected to a virtual cinema where an observer sits. Again a cube is swept through space and the width and depth angles are measured. Having established this basic concept, we then modify the parameters of focal length and interaxial distance and see how they affect the width-to-depth ratio. During evaluation, new dependencies are found between all parameters. Moreover, new measurement parameters are presented, which simplify the usage of depth volume in stereoscopic photography.

MPEG DASH 기반 service-compatible 3D 콘텐츠 대상 HTTP adaptive streaming 적용방안

최근 3D 입체영상에 대한 소비자들의 관심 증대로 가전 업체들은 3D TV, 3D Monitor등의 제품들을 출시하고 있고, 이에 따라 3D 콘텐츠 시장도 급격히 성장할 것으로 예상된다. 그러나 이러한 3D 콘텐츠를 서비스함에 있어서 보다 높은 대역폭이 요구되어지므로 QoS(Quality of Service)를 보장하지 않는 IP 기반 전송환경에서의 고화질 3D 스트리밍 서비스는 소비자에게 끊김 없는 영상서비스를 제공하는데 제약이 있다. 이러한 문제를 해결하기 위해서, IP 망 네트워크 환경을 고려하여 사용자들이 원하는 콘텐츠를 네트워크 환경에 따라 적절한 품질로 제공 할 수 있는 적응적 스트리밍 서비스 기술이 현재 국제 표준화 기구인 MPEG(Moving Picture experts Group)에서 DASH(Dynamic Adaptive Streaming over HTTP)라는 이름으로 표준화가 진행 중에 있다. 그러나 DASH에서는 독립된 좌, 우 영상이 단일 segment안에 2개의 Elementary Stream(ES)으로 구성되어 있는 service- compatible 기반의 스테레오스코픽 콘텐츠에 대한 명확한 scheme이 나타나지 않는다. 따라서 본 논문에서는 DASH를 이용한 service-compatible 기반의 효율적인 적응적 3D 스트리밍 서비스 제공 방안을 제안한다. DASH를 이용한 3D 서비스는 사용자들에게 IP 망을 통한 다양한 품질의 3D 콘텐츠를 제공함으로써 끊김 없는 3D 스트리밍 서비스가 가능하고, 또한 하나의 3D 콘텐츠로 다양한 디바이스에 적용 가능하다는 이점이 있다. Recently, many consumer electronics manufacturers have produced 3D devices such as 3DTVs and 3D monitors as interests in a stereoscopic video service are getting increased. However, most of 3D services are focused on local storage or bandwidth guaranteed service since 3D stereoscopic video service require bandwidth more stable and larger. This property causes difficulties in seamless stereoscopic video streaming services under IP based open network environment that cannot guarantee quality of services. In order to achieve a seamless video streaming service the international standard organization MPEG (Moving Pictures Experts Group) has developed the adaptive HTTP streaming technology called as DASH (Dynamic Adaptive Streaming over HTTP). However, the DASH doesn't have obvious scheme which can express the two elementary video streams based service-compatible stereoscopic contents in one segment. Therefore, this paper proposes a scheme of efficient 3D adaptive streaming service based on the DASH, which covers not only frame-packing stereoscopic contents but also service-compatible ones. The 3D adaptive HTTP streaming scheme introduced in this paper is able to provide 3D contents with various qualities to user and also has benefit that single 3D content can be applied to a variety of devices.

Reading performance as screening tool for visual complaints from stereoscopic content

Some people experience visual discomfort when watching stereoscopic content, which to a large extent can be predicted by their binocular visual status. Earlier research has proven that the binocular status can be assessed by the difference in reading speed between 2-D and 3-D. In this paper, we further substantiate the evidence for a tool to screen people on their susceptibility to such visual complaints based on this reading speed difference.In an experiment participants were asked to (1) perform the Wilkins Rate of Reading Test (WRRT) at three disparities (−1.5°, 0° and 1.5°) preceded and followed by fusion measurements and self-reports, and (2) scale the screen disparity of stimuli to a threshold of visual discomfort. Participants were categorized based on the WRRT-ratio, being the ratio of the number of words read at 0° disparity over the number of words read at −1.5° disparity.Our results showed that only participants with a high WRRT-ratio revealed a tendency in changed fusion range indicating visual fatigue. They also reported significantly more visual discomfort in stereoscopic conditions and had lower thresholds in screen disparity for visual discomfort than participants with a normal WRRT-ratio. Hence, the WRRT-ratio has potential as a visual screening tool in 3-D consumer applications to warn viewers that are susceptible to visual discomfort when watching stereoscopic content.

Stereo-video inpainting

As the production of stereoscopic content increases, so does the need for post-production tools for that content. Video inpainting has become an important tool for rig removal, but there has been little consideration of the problem in stereo. We present an algorithm for stereo-video inpainting that builds on existing exemplar-based video completion and also considers the issues of view consistency. Given user-selected regions in the sequence, which may be in the same location in several frames and in both views, the objective is to fill in this area using all the available picture information. Existing algorithms lack temporal consistency, causing flickering and other artifacts. We explore the use of long-term picture information across many frames in order to achieve temporal consistency at the same time as exploiting inter-view dependencies within the same framework.

A perceptual metric for stereoscopic image quality assessment based on the binocular energy

Stereoscopic imaging is becoming very popular and its deployment by means of photography, television, cinema. . .is rapidly increasing. Obviously, the access to this type of images imposes the use of compression and transmission that may generate artifacts of different natures. Consequently, it is important to have appropriate tools to measure the quality of stereoscopic content. Several studies tried to extend well-known metrics, such as the PSNR or SSIM, to 3D. However, the results are not as good as for 2D images and it becomes important to have metrics dealing with 3D perception. In this work, we propose a full reference metric for quality assessment of stereoscopic images based on the binocular fusion process characterizing the 3D human perception. The main idea consists of the development of a model allowing to reproduce the binocular signal generated by simple and complex cells, and to estimate the associated binocular energy. The difference of binocular energy has shown a high correlation with the human judgement for different impairments and is used to build the Binocular Energy Quality Metric (BEQM). Extensive experiments demonstrated the performance of the BEQM with regards to literature.

3D 컴퓨터 그래픽을 이용한 입체 애니메이션 콘텐츠 제작기법 연구

The remarkable development of digital technologies and the success of 3D stereoscopic films have led to started drawing attention to stereoscopic images. The stereoscopic images have not become a passing fad in the present time unlike in the past, but have constantly showed their potentials. Unfortunately, the domestic stereoscopic images market is faced with difficulties, such as lack of the capital strength and of insufficient production infrastructure, and the production of 3D from 2D. As a result, it is not easy to produce creative stereoscopic animation contents. Therefore, in Korea, more efforts should be made in accumulating a plenty of data on production techniques. In order to make film production reflecting stereoscopy and effective stereoscopic implementation, which are necessary to produce high-quality stereoscopic contents, this thesis gave an explanation about a production technique studied on the basis of 3D graphic. Stereoscopic images are definitely one of the most promising image contents in the 21st digital contents industry. For the reason, in order to produce stereoscopic animation contents with high quality, fundamental studies on the stereoscopic images should be performed in a constant and cautions way.

Analysis of User Requirements in Interactive 3D Video Systems

The recent development of three dimensional (3D) display technologies has resulted in a proliferation of 3D video production and broadcasting, attracting a lot of research into capture, compression and delivery of stereoscopic content. However, the predominant design practice of interactions with 3D video content has failed to address its differences and possibilities in comparison to the existing 2D video interactions. This paper presents a study of user requirements related to interaction with the stereoscopic 3D video. The study suggests that the change of view, zoom in/out, dynamic video browsing, and textual information are the most relevant interactions with stereoscopic 3D video. In addition, we identified a strong demand for object selection that resulted in a follow-up study of user preferences in 3D selection using virtual-hand and ray-casting metaphors. These results indicate that interaction modality affects users’ decision of object selection in terms of chosen location in 3D, while user attitudes do not have significant impact. Furthermore, the ray-casting-based interaction modality using Wiimote can outperform the volume-based interaction modality using mouse and keyboard for object positioning accuracy.

OSCAM - optimized stereoscopic camera control for interactive 3D

This paper presents a controller for camera convergence and interaxial separation that specifically addresses challenges in interactive stereoscopic applications like games. In such applications, unpredictable viewer- or object-motion often compromises stereopsis due to excessive binocular disparities. We derive constraints on the camera separation and convergence that enable our controller to automatically adapt to any given viewing situation and 3D scene, providing an exact mapping of the virtual content into a comfortable depth range around the display. Moreover, we introduce an interpolation function that linearizes the transformation of stereoscopic depth over time, minimizing nonlinear visual distortions. We describe how to implement the complete control mechanism on the GPU to achieve running times below 0.2ms for full HD. This provides a practical solution even for demanding real-time applications. Results of a user study show a significant increase of stereoscopic comfort, without compromising perceived realism. Our controller enables 'fail-safe' stereopsis, provides intuitive control to accommodate to personal preferences, and allows to properly display stereoscopic content on differently sized output devices.

Susceptibility to Visual Discomfort of 3-D Displays by Visual Performance Measures

People with some signs of binocular dysfunctioning can be susceptible to visual complaints associated with viewing stereoscopic content at large viewing distances. Two performance measurements enabled to distinguish people by their binocular status (BS) in previous research: the ratio of performance of the Wilkins rate of reading test (WRRT) between 2-D and 3-D, and the vergence facility. In an experiment, first, an extensive optometric screening was carried out to differentiate visually asymptomatic young adults with good BS (GBS) (N = 27) from those with a moderate BS (MBS) (N = 6). Second, participants had to perform the WRRT at short viewing distance followed by a questionnaire under different screen disparity settings. The results reveal that the ratio of the WRRT between 0 and -1.5 screen disparity is an appropriate indicator of participants with MBS in comparison with participants with GBS. In addition, the results show that 0.75° of screen disparity is already problematic for people with MBS. We conclude that the WRRT-ratio has potential as a BS test in consumer applications to provide individual settings for comfortable screen disparities based on viewers' BS.

A Study on space perception and contentment change of 3D stereoscopic content by scene directing -Around the camera angle and object distance-

A Study on space perception and contentment change of 3D stereoscopic content by scene directing -Around the camera angle and object distance-

3D입체영화관객의 만족에 영향을 미치는 영화적 요인

본 연구는 3D입체영화 관객의 만족에 영향을 미치는 요인들을 도출하고, 이를 바탕으로 3D입체영화 관객의 관람만족도를 증가시키는 방안을 모색하여 3D입체영화 산업의 발전전략을 제시함을 목적으로 한다. 이를 위해 선행연구를 분석한 결과, 영화 관객의 만족에 영향을 주는 요인들은 크게 영화 내부적 요소, 영화 환경적 요소, 영화 관객요소로 나뉘며, 그 중 영화 내부적 요소의 영향력이 나머지 둘에 비해 높은 것으로 나타났다. 하지만 3D입체영화는 기존의 영화와 촬영, 상영, 관람방식, 인지과정 등 영화 환경적 요소와 영화 관객요소에 해당하는 부분에서 큰 차이점을 보이므로 3D입체영화에서는 영화 환경적 요소와 영화 관객요소의 영향력이 높을 것으로 예상되었다. 이에 본 연구는 3D입체영화 관객을 대상으로 관람만족에 대한 설문을 실시하였으며 주요 분석결과는 다음과 같다. 첫째, 시각적 피로감 개선 방안의 필요성이 대두된다. 둘째, 영화관 구조와 좌석배치에 대한 개선이 요구된다. 셋째, 관객의 입장에서 납득할 수 있는 3D입체영화 가격수준을 책정해야한다. The purpose of this study is to examine factors that affect 3D stereoscopic movie audiences' satisfaction in Korea, and to provide strategic solutions to improve audiences' satisfaction. According to study about movie audiences' satisfaction, factors that affect movie audiences' satisfaction are separated to movie internal factors, movie environmental factors and movie audience factors. And leverage of movie internal factors is bigger than either. But 3D stereoscopic movie is different from ordinary movie in shooting, screening, seeing and recognizing method. So it is expected that movie environmental factors and audience factors' influence is bigger than internal factors. So this study carried out a survey targeting 3D stereoscopic audiences. Main result of this study are shown as follows. First, the method to reduce visual fatigue is necessary for satisfaction. Second, it is required that improvement method of seat arrangement to provide high quality 3D stereoscopic contents. Third, level of 3D stereoscopic movies' price need to adjust to valid level.

Effects of stereoscopic 3D contents on the process of learning to build a handmade PC

We previously developed Web3D contents that simulate the procedure for assembling a handmade PC. The original motivation for this project was to enable media-informatics students to deepen their understanding of PC hardware. However, it was found that due to their lack of perspective, conveying information that relates to such virtual manufacturing was quite difficult using conventional 2D displays. In this study, we optimized the graphical contents for a stereoscopic environment that would allow proper perspective to be realized. No significant differences were identified between stereoscopic 3D cases and 2D cases. This result is considered to be due to learner fatigue specific to stereoscopic 3D cases.

Subjective evaluation of wide-viewing-angle stereoscopic contents in a dome theater

This study was conducted for the purpose of evaluating the impressions gained by the members of the audience who have seen contents in a dome theater, based on their seating positions. Dome Theater Gaia provided the environment where the contents for evaluation were to be presented, and enquête (survey) was used as the investigative method. The survey results showed that the presentation of wide-viewing-angle three-dimensional (3D) contents proved effective in enhancing the 3D effect and the presence in a dome theater. Moreover, the study results confirmed that in relation to the impression evaluation of the contents for different seating positions, the optimal seating position varied according to the presentation method of the 3D and 2D contents.

Ergonomic evaluation of stereoscopic contents for a museum exhibition

This research entailed the production of stereoscopic 3D (S3D) contents using 2D-to-S3D conversion for exhibition at a museum and subjective evaluation. Hybrid production combining S3D images of existing live-action videos using the 2D-to-S3D conversion technology and computer graphic ones created via stereo rendering was conducted. Design and control of the chronological analysis of the parallactic angle was conducted on the produced contents, using binocular information as well as subjective evaluations, with the intent of conducting an investigation on the characteristics of such contents from the perspectives of the producers and viewers. An investigation was also conducted on the effects of the viewing position on the impressions of the S3D images.

Computational stereo camera system with programmable control loop

Stereoscopic 3D has gained significant importance in the entertainment industry. However, production of high quality stereoscopic content is still a challenging art that requires mastering the complex interplay of human perception, 3D display properties, and artistic intent. In this paper, we present a computational stereo camera system that closes the control loop from capture and analysis to automatic adjustment of physical parameters. Intuitive interaction metaphors are developed that replace cumbersome handling of rig parameters using a touch screen interface with 3D visualization. Our system is designed to make stereoscopic 3D production as easy, intuitive, flexible, and reliable as possible. Captured signals are processed and analyzed in real-time on a stream processor. Stereoscopy and user settings define programmable control functionalities, which are executed in real-time on a control processor. Computational power and flexibility is enabled by a dedicated software and hardware architecture. We show that even traditionally difficult shots can be easily captured using our system.

Visual discomfort of 3D TV: Assessment methods and modeling

To gain knowledge on how visual discomfort is built up while watching stereoscopic content an experiment was designed with two objectives: (1) to compare the continuous evaluation method with other assessment methods that potentially can substitute the continuous evaluation for the assessment of visual discomfort of e.g., feature-length movies, and (2) to relate the impact of time-variant content characteristics, such as motion and disparity, to the assessment of visual discomfort. In an experiment a 24 min 3D movie ‘Spy Kids 3-D: game over’ converted from 2D to 3D was displayed on a 9-view autostereoscopic lenticular LCD, and continuously assessed in terms of visual comfort by 24 participants. Additional assessment methods included the assessment of six 10 s sequences captured from the 3D movie and a single retrospective assessment of the entire 3D movie. Time-variant content characteristics, such as derivatives of motion and screen disparity values, were extracted from the 3D movie with motion and depth estimation algorithms. The moment-to-moment values of these characteristics were correlated to the continuous assessment scores of visual discomfort. With respect to the first objective, results reveal that the correlation between the assessment of the 10 s sequences captured from the 3D movie and their corresponding part within the continuous assessment is low, whereas the correlation between the retrospective assessment and the mean of the continuous assessment score over scene parts with a high screen disparity is higher. With respect to the second objective, for static scenes the visual comfort can be largely described by the screen disparity offset and range. For dynamic scenes the visual comfort is largely related to the screen disparity range, lateral motion and to the change in screen disparity.

Novel Stereoscopic Content Production Tools

Stereoscopic 3D is on the cusp of becoming a mass consumer product. But despite the recent rise of stereoscopic movies and technological advances in the industry, many challenges regarding 3D content creation and display remain unsolved. A fundamental issue is control and modification of the stereoscopic depth of live-action content after it has been recorded, since changing depth effectively amounts to changing the baseline and vergence of the capture rig. With existing post-production tools, the modification of these parameters involves cumbersome and expensive segmentation, dense depth reconstruction, and inpainting. This paper provides an overview of existing technologies for stereoscopic content editing, identify current challenges, and presents a number of recent research results that provide novel solutions to problems such as disparity correction and depth authoring, display adaptation, and 2D- to 3D conversion.