360-degree Video Research Articles

Applying Cinematic Virtual Reality with Adaptability to Indigenous Storytelling

Cinematic Virtual Reality (CVR) is a style of narrative-based Virtual Reality (VR) experience built on filmed or computer-generated 360-degree videos. Since CVR is becoming more popular and widely accessible, researchers and practitioners have been trying to address challenges such as the conflict between the viewer’s freedom of choice and the creator’s control over where to look, or the risk of missing key story elements due to such freedom. As part of the solution, CVR creators employ attention-guiding cues, introduce viewer interaction, and combine these two techniques into all-encompassing CVR production frameworks. However, there are very few CVR projects that embrace the various differences in the backgrounds, preferences, and expectations of each individual viewer. Further to this, they do not consider the content creator/owners’ perspective when presenting and digitizing stories from the real world, especially when considering viewer’s connection to the cultural significance contained. In this article, a case study is presented to explore the use of adaptability to viewer situations and the coherence to Māori (the indigenous people of New Zealand) storytelling contexts in CVR experiences. In the case study, we began with co-design sessions with storytellers from Te Rau Aroha Marae (an active Māori cultural heritage site in the deep south of New Zealand), about appropriate features to collect from visitors to a virtual storytelling event , then co-built personas as representative tools. 360-degree videos of pūrākau (stories) were then captured and presented via an adaptable VR system. Evaluations were conducted with the storytellers to validate the system and to collect reflections and opinions on both the use of CVR in Māori storytelling and the cultural appropriateness of CVR with adaptability. We conclude this article with a discussion of possible improvements for future CVR frameworks.

Assessing alexithymia: the proposal of a psychometric tool based on spheric videos.

The present perspective introduces a novel psychometric tool designed to enhance the evaluation of alexithymia. Alexithymia, a condition marked by difficulties in recognizing and expressing emotions, along with a propensity to direct attention outside rather than toward one's own interior experiences, is commonly investigated through self-report questionnaires. These instruments assume that individuals have sufficient self-awareness and abstraction capabilities, which restricts the understanding of the underlying mechanisms of emotional recognition in individuals who do not possess these capacities. To address this lack, emerging technologies like virtual reality (VR) and 360° videos facilitate the recreation of immersive contexts, enabling subjects to engage with scenarios even remotely. Our innovative tool employs spherical video technology to recreate social and non-social scenarios that elicit emotions. Psychophysiological measures are collected during video observation; then, questions are asked to investigate how the subject consciously processes the emotions they experienced. This multimodal approach aims to capture both implicit and explicit emotion processing, providing a comprehensive assessment. Overall, the proposed psychometric tool offers the potential for a more nuanced understanding of alexithymic traits and their real-life impact, empowering clinicians to tailor treatment processes to individual needs based on a richer set of information.

Impact of virtual reality on promoting understanding of research in rural Africa: The KEMRI-Wellcome Trust virtual tour of research laboratories

Background Virtual Reality (VR) is poised to transform how science is communicated to the public. There is limited evidence of the use of VR for science education or engagement in sub-Saharan Africa. We produced a virtual reality video tour of the KEMRI-Wellcome Trust Research Programme (KWTRP) laboratories and tested its suitability as an engagement tool for secondary school students. Methods A group of 14 students from various secondary schools, but residents of Kilifi County, situated on the coast of Kenya, were invited for a guided tour of KWTRP laboratories. After that, they were split into three groups to discuss their understanding of the tour and what was relatable to what they were learning in school. The discussion informed the script of a 360 degree video of the KWTRP laboratories. The video was uploaded to the VR headsets and shown to select students (n=59) from three secondary schools in Kilifi County. We administered pre and post survey questionnaires to assess the impact of the virtual tour of the research laboratories. Results Most students reported having never watched videos on laboratory or research before, with few having ever used a virtual reality (VR) headset. After watching the virtual reality video of the laboratory, most students indicated that they found the VR experience good and agreed that it made them feel like they were touring the laboratory in person. They also indicated that they had an increased interest in working in a research laboratory. We observed significant gains in knowledge related to KWTRP among the students after watching the VR video. Conclusions Enabling the public to understand how research is conducted is important for garnering community confidence and scientific support. The virtual reality experience of research laboratories offers a promising tool for science engagement in young adults.

Omnidirectional Video Quality Assessment With Causal Intervention

Omnidirectional Video Quality Assessment With Causal Intervention

Virtual reality and tourism destinations marketing: can it transform travel? evaluating the impact of immersive experiences on travel intentions

ABSTRACT This study investigates the relationship between the immersive VR experience, destination cognitive image, destination affective image, and tourists’ travel intention. The sample of this research is 167 visitors of three tourist attractions of Isfahan Province in Iran, through 360-degree virtual reality videos. In order to collect research data, an online survey and convenience sampling methods were used. PLS-SEM was utilised to assess both the measurement and structural models. The findings indicated that among factors of external technical environment stimuli, media richness has a stronger effect on the formation of the destination's cognitive image than the affective image. Also, perceived interactivity compared to media richness has a significant impact on the formation of the destination's affective image. In addition, vision consistency as an internal technical environment stimulus has a positive effect on the formation of the cognitive and affective image of the destination. Finally, the affective image of the destination resulting from the immersive VR experience has a stronger effect on travel intention than the cognitive image.

Design and Development of e-MARZ: A Practical Guide to Implementing Cost-effective VR Experiences for Music Education

The study introduces e-MARZ, a Virtual Reality (VR) software designed to tackle challenges faced by music educators in enhancing music appreciation lessons within the Malaysian lower secondary school music curriculum. The research aims to improve student engagement, provide immersive and interactive learning experiences, and develop a user-friendly VR application. A video production process was employed to create 360-degree music performance videos, which were integrated into the e-MARZ software via interactive modules. This article mainly discusses the development and evaluation aspects, utilising the modified ADDIE Instructional model, m-ADDIE, during the development phase. The e-MARZ prototype underwent evaluation using the System Usability Scale (SUS) and the Post-Study System Usability Questionnaire (PSSUQ) for quantitative data collection and analysis to assess usability, with a sample of five music education experts and 82 secondary school music students. Limitations of the study include the use of 360 media of live performances tailored for secondary school students' music appreciation and employing 3DOF (degrees of freedom) instead of the more advanced 6DOF, potentially impacting the generalisability of findings and the scope of achievable VR experiences. Despite these limitations, the high usability scores obtained during the evaluation, as measured by SUS and PSSUQ, indicate the software's effectiveness in addressing its intended challenges. The e-MARZ software offers an immersive, user-friendly VR experience that is cost-effective and compatible with various devices. This research provides insights into producing cost-effective, user-friendly VR applications using 360 media, promoting accessible VR application development in education. As a result, the study transforms students' engagement with music appreciation, fostering a more immersive, interactive, and enjoyable learning experience.

Does increasing biodiversity in an urban woodland setting promote positive emotional responses in humans? A stress recovery experiment using 360-degree videos of an urban woodland.

Green spaces can support human stress reduction and foster positive emotional well-being. Previous research has suggested that biodiversity (i.e. the variety of species of plants and animals in a given location) can enhance recovery from stress even further. However, there is limited experimental evidence testing this hypothesis and results, to date, have been mixed. This study aimed to provide further understanding of the role of biodiversity (actual or perceived) on human well-being by experimentally manipulating species richness and stress. Participants (372 in total) took part in an online experiment, where they received an episode of mild stress before watching a 360-degree video to recover. The video showed the same location, an urban woodland, but at one of four artificially manipulated levels of biodiversity. The participants reported their Positive and Negative Affect before and after the stress induction and after watching the video, providing a measure of their stress and well-being throughout the experiment. Participants also reported their perceptions of biodiversity (i.e. how diverse they thought the location was) and elaborated on their responses with brief comments. Repeated Measure Analysis of Variance revealed that exposure to all levels of biodiversity reduced the participants' Negative Affect, but with no significant difference between the conditions. However, the analysis showed higher Positive Affect in those participants who perceived the environment as more biodiverse. Comments from participants indicated that those who reported noticing flowers and trees in the environment also showed higher Positive Affect. This suggests that perceiving biodiversity promotes more positive emotions, but critically one needs to actually notice (engage with) the components of biodiversity to elicit these extra benefits.

A Survey on Video Streaming for Next-Generation Vehicular Networks

As assisted driving technology advances and vehicle entertainment systems rapidly develop, future vehicles will become mobile cinemas, where passengers can use various multimedia applications in the car. In recent years, the progress in multimedia technology has given rise to immersive video experiences. In addition to conventional 2D videos, 360° videos are gaining popularity, and volumetric videos, which can offer users a better immersive experience, have been discussed. However, these applications place high demands on network capabilities, leading to a dependence on next-generation wireless communication technology to address network bottlenecks. Therefore, this study provides an exhaustive overview of the latest advancements in video streaming over vehicular networks. First, we introduce related work and background knowledge, and provide an overview of recent developments in vehicular networking and video types. Next, we detail various video processing technologies, including the latest released standards. Detailed explanations are provided for network strategies and wireless communication technologies that can optimize video transmission in vehicular networks, paying special attention to the relevant literature regarding the current development of 6G technology that is applied to vehicle communication. Finally, we proposed future research directions and challenges. Building upon the technologies introduced in this paper and considering diverse applications, we suggest a suitable vehicular network architecture for next-generation video transmission.

Performance analysis of multiview video compression based on MIV and VVC multilayer

AbstractTo represent immersive media providing six degree‐of‐freedom experience, moving picture experts group (MPEG) immersive video (MIV) was developed to compress multiview videos. Meanwhile, the state‐of‐the‐art versatile video coding (VVC) also supports multilayer (ML) functionality, enabling the coding of multiview videos. In this study, we designed experimental conditions to assess the performance of these two state‐of‐the‐art standards in terms of objective and subjective quality. We observe that their performances are highly dependent on the conditions of the input source, such as the camera arrangement and the ratio of input views to all views. VVC‐ML is efficient when the input source is captured by a planar camera arrangement and many input views are used. Conversely, MIV outperforms VVC‐ML when the camera arrangement is non‐planar and the ratio of input views to all views is low. In terms of the subjective quality of the synthesized view, VVC‐ML causes severe rendering artifacts such as holes when occluded regions exist among the input views, whereas MIV reconstructs the occluded regions correctly but induces rendering artifacts with rectangular shapes at low bitrates.

Comparing the Effectiveness of 360-Degree Live-Action VR and 2D Video for Initial Medical Examinations by Medical Students: A Randomized Controlled Trial

Comparing the Effectiveness of 360-Degree Live-Action VR and 2D Video for Initial Medical Examinations by Medical Students: A Randomized Controlled Trial

An attention-based approach for assessing the effectiveness of emotion-evoking in immersive environment

Visual stimuli within an immersive virtual environment impact human perception and behavior in notably different ways compared to the real world. Previous studies have presented evidence indicating that individuals in various emotional states exhibit an unconscious attentional bias toward either positive or negative stimuli. However, whether these findings can be replicated within an immersive virtual environment remains uncertain. In this study, we devised an attention-based experiment to explore whether the correlation between participants' emotional states and the valence of visual stimuli influences their attentional bias. Participants (n=28) viewed 360-degree videos with varying valence levels (positive and negative) to evoke emotions. Subsequently, we utilized standard emotional human faces as stimuli to assess how the consistency in video valence and emotional faces affects reaction time (RT) in Go tasks and error rates in No-go tasks. We employed the Ex-Gaussian approach to analyze the RT data. The parameters—mu (μ), sigma (σ), and tau (τ)—were computed to denote response speed and attentional lapses, respectively. Our findings revealed a significant increase in tau (τ) when the valence of the video and emotional faces aligned. This suggests that the Go/No-go paradigm is effective in evaluating the impact of emotion-evoking stimuli within an immersive environment.

Towards a Co-Creative Immersive Digital Storytelling Methodology to Explore Experiences of Homelessness in Loughborough

Despite the potential use of digital storytelling with marginalised groups, there are few examples of its application in homelessness or examinations of co-creative relationships in this context. Along with digital storytelling, this research used immersive media (virtual reality and 360 degree video) to explore place-based social exclusion. In the feasibility study, with four doctoral researchers at Loughborough University as participants, immersive digital stories were co-created. The aim of this study was to understand how to create place-based immersive digital stories, through adapting existing digital storytelling methods and the co-creation of virtual reality, to inform best practices for future studies involving participants who have experienced homelessness. Participants created maps and empathy timelines, shared stories, recorded voiceovers and edited footage. The researcher facilitated this and recorded the 360-degree filmed footage. The final stories proved to explore place-based social exclusion. Co-creative relationships were found to be more significant between the researcher and individual participant than amongst the participants as a group. With immersive media, the researcher’s experience formed an active part of the finished pieces. Despite this, participants described their role as director, being ultimately in control. These findings will influence the methods that will be used in the future with those who have experienced homelessness in Loughborough. They also show how immersive media in digital storytelling can strengthen co-creation and acknowledge the researcher in the story.

Ray tracing-based construction of 3D background model for real-time stereoscopic rendering of live immersive video

Immersive video stored in multiview video-plus-depth format can provide viewers with vivid immersive experiences. However, rendering such video in real time in immersive environments remains a challenging task due to the high resolution and refresh rate demanded by recent extended reality displays. An essential issue in this immersive rendering is the disocclusion problem that inevitably occurs when virtual views are synthesized via the de facto standard 3D warping technique. In this paper, we present a novel virtual view synthesis framework that, from a live immersive video stream, renders stereoscopic images in real time for a freely moving virtual viewer. The main difference from previous approaches is that the surrounding background environment of the immersive video’s virtual scene is progressively reproduced on the fly directly in the 3D space while the input stream is being rendered. To allow this, we propose a new 3D background modeling scheme that, based on GPU-accelerated real-time ray tracing, efficiently and incrementally builds the background model in compact 3D triangular mesh. Then, we demonstrate that the 3D background environment can effectively alleviate the critical disocclusion problem in the immersive rendering, eventually reducing spatial and temporal aliasing artifacts. It is also suggested that the 3D representation of background environment enables extension of the virtual environment of immersive video by interactively adding 3D visual effects during rendering.

Comparison of Two Simulation Tools to Develop Empathic Communication Skills in Nursing Students Breaking Bad News: A Randomized Controlled Study

BackgroundThe announcement of a pathology that could have a negative impact on a patient's future (i.e., breaking bad news, BBN) is a common and difficult task that nurses feel unprepared for. This study compared the effectiveness of two online simulation-based training tools: a standardized patient, currently the most commonly used method, and 360° immersive videos, which could represent an alternative that demands fewer resources. MethodSixty-nine nursing students were involved in a randomized controlled noninferiority trial and allocated to one of three conditions: standardized patient (SP); 360° immersive videos (360IV); and control group (CG). In all three conditions, pre, post, and one-month follow-up measurements were based on recorded role-plays with patient-actors and questionnaires. Using recordings, external raters assessed students’ communication skills along three dimensions related to BBN: empathic communication, building shared understanding, and shared decision-making. Right after the prepost role-plays, patient-actors assessed perceived empathy and confidence. Participants completed questionnaires assessing empathy, stress, and self-efficacy. ResultsEmpathy perceived by the patient-actors increased in all conditions, but this increase was only maintained in the 360IV condition. In contrast, the SP condition produced the best communication results, as assessed by external raters. ConclusionsThe two training tools induced complementary changes in BBN skills among undergraduate nurses, opening up new possibilities for BBN training.

Scalable and resilient 360-degree-video adaptive streaming over HTTP/2 against sudden network drops

Realistic Virtual Reality is supported by 360° video, which provides viewers with an immersive watching experience. However, 360°video is bulky in size, while the transmission system has limited ability to provide bandwidth. As such, intelligent adaptive delivery solutions play a crucial role in enabling users to stream high-quality 360°video. In this research, we propose a novel approach for 360-degree video streaming over HTTP/2 that can provide consumers with a good watching experience (QoE) even in varying network circumstances and head-eye movements over time. The proposed method deploys the so-called BBAG algorithm (Buffer and Bandwidth Allocation Algorithm) using Scalable Video Coding to choose appropriate tile layers to resolve the trade-off between network and user adaptivity. With the support of HTTP/2’s stream termination capability, the delivery of late tile layers is terminated to handle abrupt interruptions. By employing multiple buffer thresholds, BBAG is able to adapt bitrates to changes in users’ perspective while watching a 360-degree video. BBAG is proven to improve QoE up to about 90% by maintaining high and stable buffer level, while enhancing average viewport bitrates by about 80% compared to state-of-the-art methods in different scenarios of network bandwidths.

Using Call Annie as a Generative Artificial Intelligence Speaking Partner for Language Learners

Developing English speaking skills can be challenging for many English language learners. The advent of generative artificial intelligence (GAI) has prompted the emergence of a growing number of artificial intelligence (AI)-powered chatbots designed to tackle these challenges. One popular tool is ‘Call Annie,’ a GAI video chatbot that can act as a virtual assistant, enabling users to engage in immersive video calls with AI avatars. This technology review discusses its functionality, how it can be used in supporting learners’ language development, how teachers can collaborate with it in class and its potential limitations.

MADRL-Based Joint Edge Caching and Bitrate Selection for Multicategory 360° Video Streaming

360-degree video streaming has gained increasing attraction in the current popular VR, AR, and MR applications, which can provide users with an immersive experience. In tile-based 360-degree video streaming, edge caching and bitrate selection strategies are jointly designed to improve users’ Quality of Experience (QoE), which incorporates video quality and rebuffer. However, the existing QoE-driven approaches use a unified QoE function to guide the decisions of edge caching and bitrate selection, which neglect the impact of video quality and rebuffer on different categories of 360-degree videos, thus failing to provide high average QoE for users. In this paper, we propose a MADRL-based joint edge caching and bitrate selection strategy for multicategory 360-degree video streaming to improve users’ average QoE. The key idea is to employ different edge caching and bitrate selection strategies for different video categories to enable fine-grained performance optimization. Based on multicategory 360-degree video streaming, we first model a joint edge caching and bitrate selection problem as a multi-agent cooperative Markov decision process with the goal of maximizing users’ average QoE. Next, an FoV-aware multi-agent soft actor-critic (FA-MASAC) algorithm is designed to help agents collaboratively learn optimal edge caching and bitrate selection decisions in a distributed way, in which each video category is treated as an agent. Finally, experimental results on real-world datasets show that our proposed strategy can greatly benefit users’ average QoE compared to existing strategies.

Latitude-Redundancy-Aware All-Zero Block Detection for Fast 360-Degree Video Coding.

The sphere-to-plane projection of 360-degree video introduces substantial stretched redundant data, which is discarded when reprojected to the 3D sphere for display. Consequently, encoding and transmitting such redundant data is unnecessary. Highly redundant blocks can be referred to as all-zero blocks (AZBs). Detecting these AZBs in advance can reduce computational and transmission resource consumption. However, this cannot be achieved by existing AZB detection techniques due to the unawareness of the stretching redundancy. In this paper, we first derive a latitude-adaptive redundancy detection (LARD) approach to adaptively detect coefficients carrying redundancy in transformed blocks by modeling the dependency between valid frequency range and the stretching degree based on spectrum analysis. Utilizing LARD, a latitude-redundancy-aware AZB detection scheme tailored for fast 360-degree video coding (LRAS) is proposed to accelerate the encoding process. LRAS consists of three sequential stages: latitude-adaptive AZB (L-AZB) detection, latitude-adaptive genuine-AZB (LG-AZB) detection and latitude-adaptive pseudo-AZB (LP-AZB) detection. Specifically, L-AZB refers to the AZB introduced by projection. LARD is used to detect L-AZB directly. LG-AZB refers to the AZB after hard-decision quantization and zeroing redundant coefficients. A novel latitude-adaptive sum of absolute difference estimation model is built to derive the threshold for LG-AZB detection. LP-AZB refers to the AZB in terms of rate-distortion optimization considering redundancy. A latitude-adaptive rate-distortion model is established for LP-AZB detection. Experimental results show that LRAS can achieve an average total encoding time reduction of 25.85% and 20.38% under low-delay and random access configurations compared to the original HEVC encoder, with only 0.16% and 0.13% BDBR increases and 0.01dB BDPSNR loss, respectively. The transform and quantization time savings are 60.13% and 59.94% on average.

Omnidirectional Video Super-Resolution Using Deep Learning

Omnidirectional Videos (or 360° videos) are widely used in Virtual Reality (VR) to facilitate immersive and interactive viewing experiences. However, the limited spatial resolution in 360° videos does not allow for each degree of view to be represented with adequate pixels, limiting the visual quality offered in the immersive experience. Deep learning Video Super-Resolution (VSR) techniques used for conventional videos could provide a promising software-based solution; however, these techniques do not tackle the distortion present in equirectangular projections of 360° video signals. An additional obstacle is the limited 360° video datasets to study. To address these issues, this paper creates a novel 360° Video Dataset (360VDS) with a study of the extensibility of conventional VSR models to 360° videos. This paper further proposes a novel deep learning model for 360° Video Super-Resolution (360° VSR), called Spherical Signal Super-resolution with a Proportioned Optimisation (S3PO). S3PO adopts recurrent modelling with an attention mechanism, unbound from conventional VSR techniques like alignment. With a purpose-built feature extractor and a novel loss-function addressing spherical distortion, S3PO outperforms most state-of-the-art conventional VSR models and 360° specific super-resolution models on 360° video datasets. A step-wise ablation study is presented to understand and demonstrate the impact of the chosen architectural sub-components, targeted training and optimisation.

Design of a High Speed 360-degree Panoramic Video Acquisition System Based on FPGA and USB 3.0

This paper presents a FPGA-based hardware architecture for capturing real-time panoramic images and the parallel implementation of an image stitching algorithm based on the scale-invariant feature transform (SIFT) features. First, the system transfers the images captured by five CMOS sensors to the designed FPGA board through the FMC adapter plate. Then, the system employs the FPGA-based parallel computing to extract the SIFT features and detect the best matching region and points for image registration. Finally, in order to obtain the panoramic image with smooth changes at the junctions, images of the adjacent images are fused. Using the designed FPGA platform makes it possible to extract the SIFT and stitch the images in real-time, which is not realized in most panoramic imaging systems. The panoramic system, developed using the Xilinx Zynq®-7000 all programmable SOC, can capture video of 5*640*480 images at the speed of 60 fps. To enable high-speed image transfer from the imaging system to the host computer, the USB3.0 interface with the transmission speed of 359MB/s has been developed. The real-time efficiency of the parallel system will be demonstrated by the simulations and experiments.