Panoramic Video Research Articles

The Interactive Effects of Video Type and Viewing Device on Pre-service Teachers’ Learning Performance

Classroom video use in teacher education is rising, but the interplay of video type (2D, panoramic) and viewing device (PC, HMD) on learning remains unexplored. This study examined the interactive effects of video type (2D, panoramic) and viewing device (PC, Head-Mounted Display [HMD]) on pre-service teachers’ learning performance. 143 pre-service teachers from a China university participated in a 2 × 2 randomized controlled experiment, viewing a 10-minute instructional video. Results indicate that panoramic video with HMD yielded the best learning outcomes, enhancing academic achievement, attention, presence, and immersion. Conversely, 2D video on HMD increased cognitive load. Interestingly, while PC-viewed panoramic video maximized attention, it also generated the highest cognitive load. Focus group interviews revealed the potential of immersive learning alongside challenges like motion sickness and navigation difficulties. This research provides empirical evidence for the interaction of video type and viewing devices in teacher education, informing the design of effective video-based learning.

Hub Traveler Guidance Signage Evaluation via Panoramic Visualization Using Entropy Weight Method and TOPSIS

Signage functions as guidance and distribution assistance, directly affecting the operational efficiency of traffic in and around the comprehensive transportation hubs. Among the elements of signage, the visual guidance effect is the key factor affecting the information conveyance, which should be evaluated during the design and optimization process. This study conducted field investigations and developed panoramic videos for multiple transportation hubs in China and designed a survey accordingly. Human subjects were recruited to watch panoramic videos via virtual reality (VR) and respond to the surveys. The results show that the degree of visual attention of travelers significantly affects the evaluation results of guidance signage, with the influence being inversely proportional. Key factors affecting visual attention include accurate legibility, obstruction and defacement rates, informativeness, and whether signage is set up in a hierarchical manner. In unfamiliar environments, travelers focus on the overall context and closely observe the interaction between directional signs and their surroundings. The prominence and visibility of signage are influenced by interactions within the spatial environment. Notably, simple and clear signs are more likely to attract travelers’ attention, and their directional information is more easily comprehended. Moreover, when the destination is clearly defined, visual attention significantly directs pedestrians’ wayfinding behavior.

Learning-based CU partition prediction for fast panoramic video intra coding

In recent years, Virtual Reality (VR) technology has been applied in various aspects of life. As an indispensable component of VR, panoramic video offers viewers immersive experiences, thereby attracting widespread attention. The high resolution of panoramic video poses significant challenges for its storage and transmission, requiring efficient video compression. The Versatile Video Coding (VVC) exhibits superior coding efficiency, albeit at the expense of a several dozen-fold increase in encoding complexity. In this paper, we propose a fast Coding Unit (CU) partition algorithm for intra-coding based on Support Vector Machine (SVM), which selects appropriate features for online training of multiple classifiers based on frame texture characteristics. Additionally, in response to distortion issues in panoramic video coding, we divide video frames into different regions and adjust the Quantization Parameter (QP) values of CUs within each region to improve the encoding efficiency. Experimental results show that the proposed method achieves an average encoding time reduction of 46.21%, with an average BDBR increase of only 1.01%, achieving a good trade-off between time-saving and encoding efficiency.

A study on the impact of open source metaverse immersive teaching method on emergency skills training for medical undergraduate students

BackgroundIn recent years, the traditional simulation-based medical teaching approach has faced challenges in meeting the requirements of practical emergency medicine education. This study utilized open-source tools and software to develop immersive panoramic videos using virtual reality technology for emergency medical teaching. It aims to investigate the efficacy of this novel teaching methodology. This transformation shifted the focus from physical simulation to virtual simulation in medical education, establishing a metaverse for emergency medical teaching.MethodsIn accordance with the curriculum guidelines, the instructors produced panoramic videos demonstrating procedures such as spinal injury management, humeral fracture with abdominal wall intestinal tube prolapse, head and chest composite injuries, cardiopulmonary resuscitation, and tracheal intubation. Using Unity software, a virtual training application for bronchoscopy was developed and integrated into the PICO4 VR all-in-one device to create a metaverse teaching environment. Fourth-year medical undergraduate students were allocated into either an experimental group (n = 26) or a control group (n = 30) based on student IDs. The experimental group received instruction through the metaverse immersive teaching method, while the control group followed the traditional simulation-based medical teaching approach. Both groups participated in theoretical and practical lessons as usual. Subsequently, all students underwent a four-station Objective Structured Clinical Examination (OSCE) to assess the effectiveness of the teaching methods based on their performance. Additionally, students in the experimental group provided subjective evaluations to assess their acceptance of the new teaching approach.ResultsBefore the training commenced, there were no significant statistical differences in the first aid test scores between the experimental and control groups. Following the training, the experimental group outperformed the control group in the four-station OSCE examination, with all P-values being less than 0.05. The satisfaction rate among the experimental group regarding the new teaching method reached 88.46%, reflecting levels of satisfaction and extreme satisfaction.ConclusionThe open-source metaverse immersive teaching method has demonstrated a positive impact on enhancing the emergency skills of medical undergraduate students, with a high level of acceptance among students. In comparison to traditional simulated medical teaching methods, this approach requires less time and space, incurring lower costs, and is deemed worthy of wider adoption.

Exploring geospatial digital twins: a novel panorama-based method with enhanced representation of virtual geographic scenes in Virtual Reality (VR)

An important step in implementing geospatial digital twins is to enhance the expressiveness of virtual geographical scenes for the physical world. However, the existing virtual geographical scenes cannot quickly express the dynamically changing geographic environment for remote users due to the inefficient handling of modeling processes, user perception, and remote sharing. The research analysed the concept and characteristics of geospatial digital twins, and constructed the virtual geographical scene ontology, based on which we developed geographical spatiotemporal semantic rules and designed a dynamic annotation algorithm to enhance the representation of virtual geographical scenes. Finally, we investigated a real-time transmission method of panoramic video based on 5 G and used immersive virtual reality (IVR) to realize the user experience of remote immersion in geographical scenes. We selected a specific geographic environment containing multiple typical geographic entities to develop three prototype systems for experimental analyses. The results showed that the proposed method enabled users to view the virtual geographical scene on a VR device. The average latency for this process was 14.72 seconds. Compared with the virtual geographical scenes constructed by traditional methods, the experiments showed the proposed method advantageous in comprehensiveness, timeliness, and photorealism and abilities to enhance the user’s geographical scene perception.

Employing 360° Video Panorama Technology to Determine the Impact of Details on the Collective Memory of the Urban Scene

Employing 360° Video Panorama Technology to Determine the Impact of Details on the Collective Memory of the Urban Scene

Animated VR and 360-degree VR to assess and train team sports decision-making: a scoping review.

In team sports, athletes' ability to make quick decisions plays a crucial role. Decision-making proficiency relies on the intricate balance of athletes' perceptual and cognitive abilities, enabling them to assess the competitive environment swiftly and select the most appropriate actions from various options. Virtual reality (VR) technology is emerging as a valuable tool for evaluating and refining athletes' decision-making skills. This study systematically examined the integration of VR technology into decision-making processes in team sports, aiming to identify more effective methods for presenting and interacting with virtual decision-making systems, thus enhancing the evaluation and refinement of athletes' decision making abilities. Following the preferred reporting items for systematic reviews and meta-analyses (PRISMA) guidelines, a thorough search of respected research databases, including Web of Science, PubMed, SPORTDiscus, ScienceDirect, PsycINFO, and IEEE, was conducted using carefully selected keywords. Twenty research papers meeting predefined inclusion criteria were included after careful evaluation. These papers were systematically analyzed to delineate the attributes of virtual decision-making task environments, the interactive dynamics inherent in motor decision-making tasks, and the significant findings. This review indicate that (1) the effectiveness of VR technology in assessing and improving athletes' decision-making skills in team sports; (2) the construction of virtual environments using the Head-Mounted Display (HMD) system, characterized by enhanced ease and efficiency; (3) the potential for future investigations to explore computer simulations to create more expansive virtual motion scenarios, thus efficiently generating substantial task scenario material, diverging from the constraints posed by 360-degree panoramic videos; and (4) the integration of motion capture technology for identifying and monitoring athletes' decision-making behaviors, which not only enhances ecological validity but also augments the transfer validity of virtual sports decision-making systems. Future research endeavors could explore integrating eye-tracking technology with virtual reality to gain insights into the intrinsic cognitive-action associations exhibited by athletes.

Spherical Distortion Temporal Propagation and Spatial Mapping Model for Efficient Panoramic Video Coding

Spherical Distortion Temporal Propagation and Spatial Mapping Model for Efficient Panoramic Video Coding

Three-dimensional motion vector translation for geometric partitioning mode in panoramic video coding using H.266/VVC

Three-dimensional motion vector translation for geometric partitioning mode in panoramic video coding using H.266/VVC

Predicting Team Well-Being through Face Video Analysis with AI

Well-being is one of the pillars of positive psychology, which is known to have positive effects not only on the personal and professional lives of individuals but also on teams and organizations. Understanding and promoting individual well-being is essential for staff health and long-term success, but current tools for assessing subjective well-being rely on time-consuming surveys and questionnaires, which limit the possibility of providing the real-time feedback needed to raise awareness and change individual behavior. This paper proposes a framework for understanding the process of non-verbal communication in teamwork, using video data to identify significant predictors of individual well-being in teamwork. It relies on video acquisition technologies and state-of-the-art artificial intelligence tools to extract individual, relative, and environmental characteristics from panoramic video. Statistical analysis is applied to each time series, leading to the generation of a dataset of 125 features, which are then linked to PERMA (Positive Emotion, Engagement, Relationships, Meaning, and Accomplishments) surveys developed in the context of positive psychology. Each pillar of the PERMA model is evaluated as a regression or classification problem using machine learning algorithms. Our approach was applied to a case study, where 80 students collaborated in 20 teams for a week on a team task in a face-to-face setting. This enabled us to formulate several hypotheses identifying factors influencing individual well-being in teamwork. These promising results point to interesting avenues for research, for instance fusing different media for the analysis of individual well-being in teamwork.

Research on the design of panoramic virtual learning environment screen elements.

Panoramic video and virtual reality technologies create learning environments that provide learners with an "immersive" experience. In recent years, panoramic video design to create immersive learning environments, in particular, has become an increasingly popular topic in teacher education and educational research. However, few studies have explored the elements of panoramic virtual learning environment screens regarding the design of learning environments. Therefore, this experimental study uses eye-tracking technology to investigate how learners are guided by panoramic video elements in a panoramic virtual learning environment. Participants (n = 90) were randomly assigned to one of six conditions: (1) no caption + live interpretation, (2) no caption + AI interpretation, (3) 120-degree caption + live interpretation, (4) 120-degree caption + AI interpretation, (5) static follow caption + live interpretation, and (6) static follow caption + AI interpretation. The results of the study show that when learners experience a panoramic virtual learning environment with different narration methods, the live interpretation method is more likely to attract learners' attention and bring better emotion and experience than the AI interpretation method. When experiencing a panoramic virtual learning environment with different caption presentation methods, the caption presentation methods induced learners' attention, learning emotions, and experiences in the order of no caption >120-degree caption > static following caption. Finally, the rules for optimizing the design of panoramic virtual learning environment screens are given based on the findings of the study, which provide new ideas for designing and developing panoramic video teaching resources.

Real‐time generation of spherical panoramic video using an omnidirectional multi‐camera system

AbstractThis paper presents a novel method for real‐time generation of seamless spherical panoramic videos from an omnidirectional multi‐camera system (OMS). Firstly, a multi‐view video alignment model called spherical projection constrained thin‐plate spline (SP‐TPS) was established and estimated using an approximately symmetrical seam‐line, maintaining the structure inconsistency around the seam‐line. Then, a look‐up table was designed to support real‐time processing of video re‐projection, video dodging and seam‐line updates. In the table, the overlapping areas in OMS multi‐view videos, the seam‐lines between spherical panoramas and OMS multi‐view videos and the pixel coordinate mapping relationship between spherical panoramas and OMS multi‐view videos were pre‐stored as a whole. Finally, a spherical panoramic video was outputted in real‐time through look‐up table computation under an ordinary GPU processor. The experiments were conducted on multi‐view video taken by “1 + 4” and “1 + 7” OMS, respectively. Experimental results demonstrate that compared with four state‐of‐the‐art methods reported in the literature and two bits of commercial software for video stitching, the proposed method excels in eliminating visual artefacts and demonstrates superior adaptability to scenes with varying depths of field. Assuming that OMS is not movable in the scene, this method can generate seamless spherical panoramic videos with a resolution of 8 K in real time, which is of great value to the surveillance field.

A 3D urban scene reconstruction enhancement approach based on adaptive viewpoint selection of panoramic videos

AbstractThe widely used unmanned aerial vehicle oblique photogrammetry often suffers from information loss in complex urban environments, leading to geometric and textural defects in the resulting models. In this study, a close‐range panoramic optimal viewpoint selection assisted 3D urban scene reconstruction enhancement method is proposed for areas prone to defects. We first introduce the ground panoramic data acquisition equipment and strategy, which are different from those of the single‐lens supplementary photography method. Data acquisition is accomplished through a single and continuous surround‐style collection approach. The full coverage of the panoramic video in the space–time dimension enables the acquisition of texture details without considering camera station planning. Then, a panoramic multiview image generation approach is proposed. Adaptive viewpoint selection is achieved using unbiased sampling points from the rough scene model, and viewpoint optimisation is adopted to ensure sufficient image overlap and intersection effects, thus improving the scene reconstructability. Finally, the 3D model is generated by photogrammetric processing of the panoramic multiview images, resulting in an enhanced modelling effect. To validate the proposed method, we conducted experiments using real data from Qingdao, China. Both the qualitative and quantitative results demonstrate a significant improvement in the quality of geometric and textural reconstruction. The tie‐point reprojection errors are less than 1 pixel, and the registration accuracy with the model from oblique photogrammetry is comparable to that of optimised‐view photography. By eliminating the need for on‐site camera station planning or manual flight operations and effectively minimising the redundancy of panoramic videos, our approach significantly reduces the photography and computation costs associated with reconstruction enhancement. Thus, it presents a feasible technical solution for the generation of urban 3D fine models.

Design of a High Speed 360-degree Panoramic Video Acquisition System Based on FPGA and USB 3.0

This paper presents a FPGA-based hardware architecture for capturing real-time panoramic images and the parallel implementation of an image stitching algorithm based on the scale-invariant feature transform (SIFT) features. First, the system transfers the images captured by five CMOS sensors to the designed FPGA board through the FMC adapter plate. Then, the system employs the FPGA-based parallel computing to extract the SIFT features and detect the best matching region and points for image registration. Finally, in order to obtain the panoramic image with smooth changes at the junctions, images of the adjacent images are fused. Using the designed FPGA platform makes it possible to extract the SIFT and stitch the images in real-time, which is not realized in most panoramic imaging systems. The panoramic system, developed using the Xilinx Zynq®-7000 all programmable SOC, can capture video of 5*640*480 images at the speed of 60 fps. To enable high-speed image transfer from the imaging system to the host computer, the USB3.0 interface with the transmission speed of 359MB/s has been developed. The real-time efficiency of the parallel system will be demonstrated by the simulations and experiments.

OJUMP: Optimization for joint unicast‐multicast panoramic VR live streaming system

AbstractVirtual reality (VR) videos have garnered increasing attention due to their ability to deliver immersive multimedia experiences to users. Nonetheless, the demanding requirements of high bandwidth and low latency for VR services present significant challenges, especially in the context of network infrastructure. In this article, we introduce a novel system called OJUMP, aimed at Optimizing the transmission and distribution of Joint Unicast‐Multicast Panoramic video field of view live streaming using Bit Index Explicit Replication (BIER). The proposed system encompasses a BIER‐based network transmission approach that enhances video transmission efficiency and facilitates large‐scale service deployment. Furthermore, OJUMP includes a dynamic disk‐state‐based tile slice distribution storage algorithm and a prefetch method designed for edge nodes to enhance the distribution efficiency of live streaming services. Experimental results illustrate that the proposed system offers distinct advantages over the baseline system in terms of bandwidth, latency, disk utilization, and central processing unit utilization.

Joint optimization of task scheduling and computing resource allocation for VR video services in 5G‐advanced networks

AbstractWith the development of 5G networks and related hardware equipment, the demand for various VR video services including panoramic video live broadcast and immersive game is increasing day by day. Due to the explosive growth of data volume and the trend toward lightweight VR equipment, traditional local computing architecture faces problems of insufficient computing power and high cost. As an emerging computing paradigm, the remote computing architecture of cloud‐edge collaboration frees users from cumbersome local restrictions. However, edge computing resources are often limited, and the remote cloud has inconsistent network latency. Therefore, how to adaptively schedule and allocate computing resources is the key to determining VR service quality. Based on an in‐depth analysis of the characteristics of VR video services and network environments, this article first builds a cloud‐edge‐network coorperation computing and transmission architecture for VR video services. This architecture can adaptively offload concurrent computing tasks and allocate computing power to them in VR scenes based on network conditions provided by the 5G‐Advanced networks via network information exposure interface. This paper proposes a general optimization problem to maximize user experience. And a single‐slot delay optimization model was constructed. Combining convex optimization theory and deep learning algorithms, this paper designs a task scheduling and computing power allocation strategy based on deep reinforcement learning. The policy gradually learns optimal task scheduling and resource allocation decisions to maximize delay‐related rewards in the long run. This article further introduces the graph neural network under the attention mechanism to solve the problem of changes in the number of concurrent tasks. Finally, through Cloudsim simulation, this article verifies the superiority of task scheduling and computing power allocation strategies based on deep reinforcement learning compared to other common strategies in terms of resource utilization, client latency, scalability and decision time.

Digital Twin and Artificial Intelligence-Empowered Panoramic Video Streaming: Reducing Transmission Latency in the Extended Reality-Assisted Vehicular Metaverse

Digital Twin and Artificial Intelligence-Empowered Panoramic Video Streaming: Reducing Transmission Latency in the Extended Reality-Assisted Vehicular Metaverse

Research on Quantization Parameter Decision Scheme for High Efficiency Video Coding

High-Efficiency Video Coding (HEVC) is one of the most widely studied coding standards. It still uses the block-based hybrid coding framework of Advanced Video Coding (AVC), and compared to AVC, it can double the compression ratio while maintaining the same quality of reconstructed video. The quantization module is an important module in video coding. In the process of quantization, quantization parameter is an important factor in determining the bitrate in video coding, especially in the case of limited channel bandwidth. It is particularly important to select a reasonable quantization parameter to make the bitrate as close as possible to the target bitrate. Aiming at the problem of unreasonable selection of quantization parameters in codecs, this paper proposes using a differential evolution algorithm to assign quantization parameter values to the coding tree unit (CTU) in each frame of 360-degree panoramic video based on HEVC so as to strike a balance between bitrate and distortion. Firstly, the number of CTU rows in a 360-degree panoramic video frame is considered as the dimension of the optimization problem. Then, a trial vector is obtained by randomly selecting the vectors in the population for mutation and crossover. In the mutation step, the algorithm generates a new parameter vector by adding the weighted difference between two population vectors to a third vector. And the elements in the new parameter vector are selected according to the crossover rate. Finally, the trial vector is regarded as the quantization parameter of each CTU in the CTU row to encode, and the vector with the least rate distortion is selected. The algorithm will produce the optimal quantization parameter combination for the current video. The experimental results show that compared to the benchmark algorithm of HEVC reference software HM-16.20, the proposed algorithm can provide a bit saving of 1.86%, while the peak signal-to-noise ratio (PSNR) can be improved by 0.07 dB.

Panoramic Video Surveillance: An Analysis of Burglary Detection Based on YOLO Framework in Residential Areas

Panoramic Video Surveillance: An Analysis of Burglary Detection Based on YOLO Framework in Residential Areas

Using the Perseverance MEDA-RDS to identify and track dust devils and dust-lifting gust fronts

In the framework of the Europlanet 2024 Research Infrastructure Transnational Access programme, a terrestrial field campaign was conducted from 29 September to 6 October 2021 in Makgadikgadi Salt Pans (Botswana). The main goal of the campaign was to study in situ the impact of the dust devils (DDs) on the observations made by the radiometer Radiation and Dust Sensor (RDS), which is part of the Mars Environmental Dynamics Analyzer instrument, on board NASA’s Mars 2020 Perseverance rover. Several DDs and dust lifting events caused by non-vortex wind gusts were detected using the RDS, and the different impacts of these events were analyzed in the observations. DD diameter, advection velocity, and trajectory were derived from the RDS observations, and then, panoramic videos of such events were used to validate these results. The instrument signal variations produced by dust lifting (by vortices or wind gusts) in Makgadikgadi Pans are similar to those observed on Mars with the RDS, showing the potential of this location as a Martian DD analog.