360-degree Video Research Articles

SVGC-AVA: 360-Degree Video Saliency Prediction With Spherical Vector-Based Graph Convolution and Audio-Visual Attention

SVGC-AVA: 360-Degree Video Saliency Prediction With Spherical Vector-Based Graph Convolution and Audio-Visual Attention

Early Explorations using KNN to Classify Emotions in Virtual Reality based on Heart Rate (HR) and Electrodermography (EDG)

To detect multimodal emotions using Virtual Reality (VR), this research demonstrates the findings and results of using a KNN Classifier by merging Heart Rate and Electrodermography signals. The participants in the study were shown 360-degree videos using a VR headset to elicit their emotional reactions. A wearable that measures skin activity and pulse rate in real time was used to record their emotional response. The experiment had a total of 30 participants, and the KNN classifier was used to classify intra-subject data. With the HR combined with EDG signals paired with KNN as the classifier, the study’s 30 participants’ data went through intra-subject classification where 11 out of 30 participants achieved a peak accuracy of 100%. These findings show that by combining HR and EDG signals, KNN may be used as the classifier to produce highly accurate results. This study’s possible applications include VR rehabilitation, gaming, and entertainment.

Assessing 360-Degree Guided Virtual Tour Video as a Viable Tourism Destination Marketing Tool: An Affective Appraisal Theory Approach

Due to the intense competition, tourism destinations confront greater challenges with marketing, branding, and positioning. Although virtual tours are increasingly viewed as a potential innovative marketing tool to attract tourists worldwide, empirical research in this area is rather limited. The study examines consumers’ affective appraisals of 360-degree video virtual tours of tourism destinations. A qualitative content analysis was employed in which YouTube’s 360-degree guided virtual tour video with 90 comments of 1,087 total words was analyzed. Using the Voyant Tool software, the study found that viewers generally have a positive sentiment toward the 360-degree guided virtual video tourism destination. This positive sentiment indicates the potential success of 360-degree video-guided virtual tourism destinations in providing a satisfying and immersive experience for the audience. Practically, the study offers valuable insights for destination marketing organizations to leverage on a 360-degree guided virtual tour video. Theoretically, the study contributes to the virtual tourism literature by extending the application of affective appraisal theory to the destination marketing domain. Further empirical work is suggested to validate and support the current study’s findings.

Local and Global Viewport History Sampling for Improved User Quality of Experience in Viewport-Aware Tile-Based 360-Degree Video Streaming

Local and Global Viewport History Sampling for Improved User Quality of Experience in Viewport-Aware Tile-Based 360-Degree Video Streaming

Optimization for 6G Wireless Communications with Heterogeneous VR and Non-VR 360-Degree Videos: A Differentiated Reinforcement Learning Approach

Optimization for 6G Wireless Communications with Heterogeneous VR and Non-VR 360-Degree Videos: A Differentiated Reinforcement Learning Approach

Non-Orthogonal Multiple Access Enhanced Scalable 360-Degree Video Multicast

Non-Orthogonal Multiple Access Enhanced Scalable 360-Degree Video Multicast

Effects of physical functional characteristics of older drivers on spatial cognition at unsignalized intersections

This study analyzed the relationship between gaze behavior and physical functions of elderly drivers at unsignalized intersections, which have spatial characteristics that make elderly drivers more likely to be on the at-fault side of a collision. Twenty-nine elderly and 14 non-elderly drivers were asked to view 360-degree videos from five different routes of two unsignalized intersections under conditions similar to actual driving. The results were as follows. (1) The eye movement distance (EMD) was shorter for elderly drivers in most of the driving sections, with the section "At slowdown" on non-priority roads being the section where a significant difference was likely to occur. (2) For both elderly and non-elderly drivers, the amount of EMD was significantly related to visual and general physical functions such as flexibility, equilibrium, and agility.

Unified Audio-Visual Saliency Model for Omnidirectional Videos With Spatial Audio

Unified Audio-Visual Saliency Model for Omnidirectional Videos With Spatial Audio

Collecting Data in an Immersive Video Environment to Set up an Agent-Based Model of Pedestrians’ Compliance with COVID-Related Interventions

Collecting Data in an Immersive Video Environment to Set up an Agent-Based Model of Pedestrians’ Compliance with COVID-Related Interventions

How Cognitive Absorption Influences Responses to Immersive Narratives of Environmental Threats.

Nongovernment organizations are increasingly leveraging the metaverse and its suite of extended reality technologies, such as 360° video and virtual reality, to immerse audiences in situations depicting environmental threats. The promise of immersive storytelling as a conservation tool is predicated on the verisimilitude of the mediated experience, with exposure to environmental threats in immersive video akin to in vivo exposure. However, the psychological mechanisms explaining users' environmental responses to immersive stories remain ambiguous. In three controlled laboratory experiments, we examined unique properties (e.g., interactivity and modality) of immersive technologies vis-à-vis environmental stories and their influence on proenvironmental outcomes. Study 1 (N = 48) implemented a two-condition (interactivity: high vs. low) between-subjects experiment, showing that interactive 360° video significantly influenced attitudes through the mechanism of cognitive absorption. A 2 (interactivity: high/low) × 2 (distance: proximal/distant) between-subjects experiment (study 2; N = 76) further supported this mediation model by showing the mediating effects of cognitive absorption on attitudes and threat perceptions toward both distant and proximal threats. Lastly, study 3 (N = 68) replicated the mediating effects of cognitive absorption across 360° video modalities, though head-mounted display-based, rather than screen-based, 360° videos only significantly increased attitudes and absorption when threats were proximal. We discuss the results' practical and theoretical implications and propose avenues for future research. Moreover, we outline important considerations for environmental organizations seeking to leverage metaverse platforms for communicating environmental threats.

MANSY: Generalizing Neural Adaptive Immersive Video Streaming With Ensemble and Representation Learning

MANSY: Generalizing Neural Adaptive Immersive Video Streaming With Ensemble and Representation Learning

Unravelling Student Learning: Exploring Nonlinear Dynamics in Science Education

Abstract Background: Traditional education research often relies on static linear approaches to measure dynamic systems involved in student information processing, overlooking the complexity of learning. Emerging research in related fields acknowledges the highly dynamic and nonlinear nature of cognitive states and information processing. Current educational research methods, predominantly based on quantitative and qualitative "snapshot" examinations, inadequately capture the dynamic and nonlinear aspects of cognitive processing during learning. Objective: This study aims to explore nonlinear dynamics as a means to describe and understand student learning processes. Methods: This study analyzed actions of 158,000 high school students in science-based immersive video games, specifically focusing on task completion within a virtual setting. Students aged 14-18, enrolled in Earth Science, Biology, Chemistry, and Physics programs, participated. Tasks, resembling Piagetian tasks, centered on volume conservation within a chemistry classroom context, employing the Student Task and Cognition Model (STAC-M) to emphasize computational cognition modeling. Results: The study tracked alterations in cognitive activations during information processing using derivatives, modeled through parameters from the authors' cognitive dataset. Employing the STAC-M model, achaotic attractors depicted convergence and sensitivity to initial conditions, reflecting cognitive associations and stability. Random data lacked the observed dynamic properties found in cognitive data, while bifurcation plots illustrated transitions from stability to chaos in cognitive processing pathways, highlighting the system's intricate nature. Conclusion: Modern science education explores beyond conventional assessments, acknowledging teaching methods' impact on students' cognitive processing. Achaotic attractors depict shifts from stable to unstable mental activities, highlighting the potential for diverse teaching approaches to minimize misconceptions and enable quicker transitions to responsive, stable learning states, aligning with educational objectives. Keyword: Cognition, Nonlinear Dynamics, Student Learning, Science Education.

The effect of evaluation and feedback in 360-degree video-based VR environments on language learning outcomes

Recently, 360-degree video technology (360VT) received considerable attention in the field of computer-assisted language learning as it creates authentic and immersive virtual reality environments to assist the language learning process. However, not many studies focused on applications of 360VT to assist the evaluation of learner’ performance and providing them with relevant feedback. Therefore, the present study was set to shed light on this aspect. We carried out an experiment to explore the effects of evaluation and feedback in two different video learning environments, i.e. a traditional video technology-based (TVT) and 360VT-based, on learning outcomes. In the context of the present study, evaluation referred to evaluating the content of videos created by students, and feedback referred to providing students with relevant feedback for further improvement of their videos. English as a foreign language (EFL) course was administered for 52 university students. The participants were equally divided into TVT and 360VT groups. They attended EFL classes, and after classes, they created videos using either a traditional video camera (TVT group) or a 360-degree video camera (360VT group) on four learning topics. The evaluators watched videos, evaluated them, and then provided feedback to students. Students then revised their works based on received feedback. The results showed an insignificant difference between the two groups in their pre-test scores when measuring prior knowledge. However, when measuring knowledge after the study, 360VT group students outperformed their counterparts in the post-test. According to the results, students in the 360VT group received significantly more feedback, and it was more detailed compared to that of their counterparts. In addition, the evaluators scored affordances of 360VT higher than those of TVT. The results showed many advantages of 360VT such as it was useful and easy to use during the evaluation process, 360VT created immersive environments so that the evaluators had a realistic experience and a high level of vividness when watching and evaluating videos. The results suggest that evaluation and feedback in 360VT environments had better effects on learning outcomes, and the evaluators positively perceived their experiences in 360VT settings. Therefore, it is suggested that educators and researchers consider implementing 360VT in situations when language learners are tasked with creating their own content, which can subsequently be evaluated with relevant feedback provided.

Using VR for Science Communication: Presence, Message Perception, and Pro-Environmental Effects

This research investigates the effectiveness of virtual reality (VR) as a science communication tool to promote pro-environmental change. A laboratory experiment ( N = 70) was conducted to examine the short-term effects of VR immersive video, in comparison with flat screen video, on level of presence, message perception, and cognition. Viewing the VR video produced greater levels of presence, which was associated with increased positive perceptions of the video, perceived importance of water issues, and pro-environmental attitudes, and behavioral intentions. Implications are discussed in terms of how immersive VR can be used as science communication tool.

Comparing technologies in teaching training: A study on the impact of cognitive load and 360-degree videos on pre-service teachers’ observations

Comparing technologies in teaching training: A study on the impact of cognitive load and 360-degree videos on pre-service teachers’ observations

A Sense of Ambient Entrapment in Hito Steyerl’s Factory of the Sun

This article proposes the notion of ambient entrapment to conceptualize the affective experience of surveillance in the current age of ubiquitous computing and smart technologies. A sense of ambient entrapment is identified as a vague, yet pervasive feeling of a controlled environment saturated by surveillance and exploitation, where machine perception and algorithmic processes are hard at work. Arguing that artworks are particularly adept at expressing affective experiences and emerging cultural feelings of surveillance, the article offers a reading of German artist Hito Steyerl’s immersive video installation environment Factory of the Sun (2015) to further explore the theoretical argument. Inside the dark installation space, visitors are immersed in a blue LED grid environment and encouraged to recline in beach chairs facing a large screen. What is perceptible as time passes inside Factory of the Sun, the paper argues, is an indefinable yet all-encompassing sense of something working and conditioning in the background, of technologies extracting and exploiting personal data while we, at the same time, desire and feel the lure of said technologies and devices. The article concludes that artworks can make us aware of the often invisible and barely perceptible forces at work in our environments and everyday life and suggests we should turn to contemporary art as sites of knowledge of the affective experience of ambient surveillance.

Immersive insights: Unveiling the impact of 360-degree videos on preservice teachers’ classroom observation experiences and teaching-quality ratings

Classroom videos are commonly used to observe and assess teaching quality in both teacher education and research on teaching and learning. In recent years, traditional video environments have increasingly been complemented by 360-degree videos, which promise a more immersive and realistic classroom experience and may affect the teaching-quality ratings that result. The aim of the present study was to explore differences between immersive 360-degree and traditional video environments in preservice teachers' (PSTs') classroom observation experiences and teaching-quality ratings. Seventy-five PSTs observed two classroom videos: one using an immersive 360-degree video environment and one using a traditional video environment. For 360-degree videos, observers reported higher cognitive, affective, and physiological involvement in the classroom situation, higher motivation, and higher mental effort when making classroom observations. For one aspect of teaching quality (focus on key concepts), the observation-based ratings resulting from 360-degree videos were better aligned with experts’ assessments of the videos. Furthermore, the results showed that the differences between video environments remained significant after the novelty of VR technology was controlled for.

Influence of visual information on the perception of environmental sounds

The purpose of this study is to clarify the influence of visual information on perception of environmental sounds. Audiovisual stimuli (360-degree video and ambisonic recordings) sampled at 6 locations in a region were presented to subjects through VR system with headphone and were rated by subjects in various evaluation items on a 7-step categorical scale. Stimuli presentation has three modes: only auditory stimulation (auditory single-mode), only visual stimulation (visual single-mode), and simultaneous presentation of audiovisual stimulation (multi-mode). Evaluation results were subjected to analysis of variance, significance tests and factor analysis to clarify differences in evaluation tendencies between modes. The dispersion of the evaluation values for auditory aspect under multi-mode tends to be smaller than that under auditory single-mode. It was found that visual information generally has the influence of relieving the impression of prominent environmental sounds. In addition, multi-mode evaluation tends to be related with evaluation obtained in visual single-mode, especially in evaluation items common to audio and visual. And it will be tried to explain the evaluation results in relation to the expected sound state for visual stimuli.

Effect of masker selection schemes on the perceived affective quality of soundscapes: A pilot study

The perceived quality of a given soundscape may be improved by adding "pleasant" sounds via a speaker or electroacoustic system to the existing soundscape as maskers. Ideally, the choice of sounds should be automated to minimize the time and labor required to maintain such a system. Hence, we investigate the effect of several automated masker selection schemes on improving the ISO Pleasantness of urban soundscapes in a controlled laboratory environment. An ambisonic recording and 360-degree video of a location exposed to traffic noise was made and reproduced via a stacked planar speaker array and virtual reality headset. Simultaneously, maskers from a fixed candidate pool were overlaid on the reproduced soundscape in one of the following methods: random selection across the entire pool, fixed selection based on results in the literature, random selection across a restricted pool based on model-agnostic analysis of a pre-existing dataset, and random selection across a restricted pool based on a deep neural network trained on a pre-existing dataset. We also investigated a control condition with no added maskers, and evaluated the results based on changes in ISO Pleasantness ratings given by participants to the soundscape under each masker selection scheme, relative to the control condition.

Research on Quantization Parameter Decision Scheme for High Efficiency Video Coding

High-Efficiency Video Coding (HEVC) is one of the most widely studied coding standards. It still uses the block-based hybrid coding framework of Advanced Video Coding (AVC), and compared to AVC, it can double the compression ratio while maintaining the same quality of reconstructed video. The quantization module is an important module in video coding. In the process of quantization, quantization parameter is an important factor in determining the bitrate in video coding, especially in the case of limited channel bandwidth. It is particularly important to select a reasonable quantization parameter to make the bitrate as close as possible to the target bitrate. Aiming at the problem of unreasonable selection of quantization parameters in codecs, this paper proposes using a differential evolution algorithm to assign quantization parameter values to the coding tree unit (CTU) in each frame of 360-degree panoramic video based on HEVC so as to strike a balance between bitrate and distortion. Firstly, the number of CTU rows in a 360-degree panoramic video frame is considered as the dimension of the optimization problem. Then, a trial vector is obtained by randomly selecting the vectors in the population for mutation and crossover. In the mutation step, the algorithm generates a new parameter vector by adding the weighted difference between two population vectors to a third vector. And the elements in the new parameter vector are selected according to the crossover rate. Finally, the trial vector is regarded as the quantization parameter of each CTU in the CTU row to encode, and the vector with the least rate distortion is selected. The algorithm will produce the optimal quantization parameter combination for the current video. The experimental results show that compared to the benchmark algorithm of HEVC reference software HM-16.20, the proposed algorithm can provide a bit saving of 1.86%, while the peak signal-to-noise ratio (PSNR) can be improved by 0.07 dB.