Three-dimensional Video Research Articles

Computationally Efficient Light Field Video Compression Using 5-D Approximate DCT

Five-dimensional (5-D) light field videos (LFVs) capture spatial, angular, and temporal variations in light rays emanating from scenes. This leads to a significantly large amount of data compared to conventional three-dimensional videos, which capture only spatial and temporal variations in light rays. In this paper, we propose an LFV compression technique using low-complexity 5-D approximate discrete cosine transform (ADCT). To further reduce the computational complexity, our algorithm exploits the partial separability of LFV representations. It applies two-dimensional (2-D) ADCT for sub-aperture images of LFV frames with intra-view and inter-view configurations. Furthermore, we apply one-dimensional ADCT to the temporal dimension. We evaluate the performance of the proposed LFV compression technique using several 5-D ADCT algorithms, and the exact 5-D discrete cosine transform (DCT). The experimental results obtained with LFVs confirm that the proposed LFV compression technique provides a more than 250 times reduction in the data size with near-lossless fidelity with a peak-signal-to-noise ratio greater than 40 dB and structural similarity index greater than 0.9. Furthermore, compared to the exact DCT, our algorithms requires approximately 10 times less computational complexity.

Blind 3D Video Stabilization with Spatio-Temporally Varying Motion Blur

Video stabilization is a challenging task that attempts to compensate for the overall frame shake during video acquisition. Existing three-dimensional video stabilization methods aim at modeling camera perspective projection through either data-driven training or explicit motion estimation. However, the above methods are difficult to effectively solve the issue of shaky videos with abrupt object movements, resulting in local motion blur in the direction of the movement. This phenomenon is prevalent in real-world scenarios featuring foreground blind motion scenes. Unfortunately, directly combining stabilization and deblurring methods poses challenges when dealing with this situation. In the video, the intensity of motion blur undergoes continuous changes, and the direct combination method inadequately utilizes spatiotemporal information, providing insufficient clues for cross-frame compensation. To alleviate this problem, the Cross-frame-temporal Module framework is proposed to address blind motion blur induced by various conditions, which utilizes cross-frame temporal features to estimate depth maps and camera motion. In this framework, a Blur Transform Network (BTNet) is designed to adapt to spatially varying motion blur, which transforms local regions according to the impact of blur intensities to adapt to the effects of non-uniform motion blur; furthermore, our Temporal-Aware Network (TANet) further suppresses motion blur by leveraging cross-frame temporal features. In addition, the limited availability of pair-training video data containing motion blur limits the application of this approach in practice. The Cross-frame-temporal Module framework adopts an un-pretrained in-test training strategy. Extensive experimental results have demonstrated that our method outperforms state-of-the-art methods.

Methodologies and Procedures for the Audiovisual Creation and Direction of Virtual Reality Short Films with Volumetric Capture. An Innovation Project for Higher Education

Virtual reality (VR) has become a cutting-edge medium for interactive experiences and immersive storytelling. In this setting, virtual reality short films have become more popular. They provide audiences with the chance to fully submerge themselves in three-dimensional environments and experience the staging of fictional events firsthand. This article presents a method for producing virtual reality short films using volumetric capture, which enables the implementation of real-time acting captured in three-dimensional video in virtual space-time.The production process is explained, starting with the idea of creating hybrid virtual reality cinematic experiences that blend various methods and assets, such as volumetric capture, computer graphics, and 360-degree film. For this experience, the production of the second episode of the virtual reality series The Stigma Machine (Martínez-Cano, 2022-currently) was carried out within the framework of the educational innovation project PIEU_B/2023_69: Production of prosocial-immersive audiovisual narratives in virtual reality, which involves 20 students from Miguel Hernández University’s Degree in Audiovisual Communication and the Double Degree in Audiovisual Communication and Journalism.In order to experiment with the students on the various processes and techniques for the audiovisual creation of VR, a review of exceptional examples of VR short films with volumetric capture was conducted during the development of this project. Case studies were analyzed to illustrate the various solutions implemented to create immersive experiences from an emotional approach.The findings highlight future directions for the creation of VR short films with volumetric capture and stress the value of multidisciplinary cooperation between filmmakers, visual artists, and VR specialists in advancing the development of this type of immersive storytelling.

Action observation intervention using three-dimensional movies improves the usability of hands with distal radius fractures in daily life-A nonrandomized controlled trial in women.

Prolonged immobilization of joints after distal radius fracture (DRF) causes cerebral disuse-dependent plasticity (DDP) and deterioration of upper extremity function. Action observation therapy (AOT) can improve DDP. This nonrandomized controlled trial (UMIN 000039973) tested the hypothesis that AOT improves hand-use difficulties during activities of daily living in patients with DRF. Right-handed women with volar locking plate fixation for DRF were divided into AOT and Non-AOT groups for a 12-week intervention. The primary outcome was difficulty in using the fractured hand, assessed with the Japanese version of the Patient-related Wrist Evaluation (PRWE). The secondary outcomes were range of motion (ROM) of the injured side and gap between measured ROM and patient-estimated ROM. The survey was administered immediately post operation and at postoperative weeks 4, 8, and 12. The AOT group used a head-mounted display and three-dimensional video during ROM exercises. The Non-AOT group used active ROM exercises alone. A generalized linear model (GLM) was used to confirm interactions and main effects by group and time period, and multiple comparisons were performed. Thirty-five patients were assigned to the AOT group (n = 18, median age, 74 years) or the Non-AOT group (n = 17, median age, 70 years). In the GLM, PRWE Total, PRWE Specific, and PRWE Usual scores revealed interactions between groups and periods. The post-hoc test revealed that the PRWE Specific scores (z = 3.43, p = 0.02) and PRWE Usual scores (z = 7.53, p<0.01) were significantly lower in the AOT group than in the Non-AOT group at 4 weeks postoperatively, whereas PRWE Total scores (z = 3.29, p = 0.04) were lower at 8 weeks postoperatively. These results suggested that AOT can improve hand-use difficulties in right-handed women after DRF surgery. AOT positively affects the motor imagery of patients with DRF and can reverse the patient's perceived difficulty in using the fractured hand during rehabilitation.

High Agreement Between 3D Motion Capture and 2D Video Analysis Systems and Raters for Knee Flexion Range of Motion During a Single-Leg Drop Landing Task

OBJECTIVE: To determine the agreement between a two-dimensional (2D) and three-dimensional (3D) video capture system and between multiple raters during a single-leg drop landing task. DESIGN: Cross-sectional observational study. METHODS: Single-leg drop landing trials of 23 males (n = 12) and females (n = 11) were recorded using the 2D and 3D systems. Reliability and agreement were quantified through intraclass correlation coefficients (ICC2, k), standard error of the measurement (SEM), and Bland-Altman plots. Analysis of variance assessed systematic error between the 2D raters and 3D system. RESULTS: A 1-way analysis of variance on 35 drop-landing trials across raters detected a difference across the 3 estimates of knee flexion range of motion ( P&lt;.001) where rater 1 did not differ (mean difference ± standard error: 1.15° ± 0.48°, P = .54) from the 3D system, rater 2 &gt; 3D system (3.30° ± 0.76°, P&lt;.001), and raters 1 and 2 differed (2.14° ± 0.82°, P = .04). There was high reliability between the two 2D raters (ICC2, k = .89, SEM = 3.44°). Rater 1 completed analyses on 87 additional trials (total n = 117) and was comparable to the 3D system (0.87° ± 0.47°, P = .06; ICC2, k = .90; SEM = 3.52°). CONCLUSION: Two-dimensional video analysis can be used in a clinical setting where 3D capture of movement mechanics is not feasible. JOSPT Open 2025;3(1):1-8. Epub 10 September 2024. doi:10.2519/josptopen.2024.0051

Characterizing the structure of mouse behavior using Motion Sequencing.

Spontaneous mouse behavior is composed from repeatedly used modules of movement (e.g., rearing, running or grooming) that are flexibly placed into sequences whose content evolves over time. By identifying behavioral modules and the order in which they are expressed, researchers can gain insight into the effect of drugs, genes, context, sensory stimuli and neural activity on natural behavior. Here we present a protocol for performing Motion Sequencing (MoSeq), an ethologically inspired method that uses three-dimensional machine vision and unsupervised machine learning to decompose spontaneous mouse behavior into a series of elemental modules called 'syllables'. This protocol is based upon a MoSeq pipeline that includes modules for depth video acquisition, data preprocessing and modeling, as well as a standardized set of visualization tools. Users are provided with instructions and code for building a MoSeq imaging rig and acquiring three-dimensional video of spontaneous mouse behavior for submission to the modeling framework; the outputs of this protocol include syllable labels for each frame of the video data as well as summary plots describing how often each syllable was used and how syllables transitioned from one to the other. In addition, we provide instructions for analyzing and visualizing the outputs of keypoint-MoSeq, a recently developed variant of MoSeq that can identify behavioral motifs from keypoints identified from standard (rather than depth) video. This protocol and the accompanying pipeline significantly lower the bar for users without extensive computational ethology experience to adopt this unsupervised, data-driven approach to characterize mouse behavior.

Imagining flying increases jumping performance in volleyball players: A pilot study

ObjectiveThe study aimed to examine the effect of mental imagery on sports performance. Specifically, it was tested whether imagining flying (i.e., air travel) increases jumping performance in a group of female volleyball players. MethodsThe study included 46 female young athletes (mean age = 15.23 years; standard deviation = 2.4) divided into two groups: the experimental group viewed a three-dimensional video that simulated a flying experience, while the control group watched neutral footage. The jump performance of both groups was measured before and after viewing the videos, using the Vertec Like® instrument to assess jump height. Participants were assessed for their flow disposition, mental imagery skills, and image vividness. ResultsComparing pre-post jump performance scores, the experimental group showed a significant improvement over the control group, with a medium-large effect size (d = 0.634). There was no significant association between flow disposition, mental imagery skills, image vividness, and pre-post jumping performance differences (respectively: β = −0.107, p = .484; β = −0.008, p = .957; β = 0.024, p = .913). ConclusionsThese findings suggest that the experience of imagining flying, enhanced with an immersive video, has a positive effect on jumping performance in a one-session study with young female volleyball players. This effect does not appear to be associated with pre-existing characteristics or the vividness of the image itself.

Playing three-dimensional video games boosts stereo vision

Playing two-dimensional video games has been shown to result in improvements in a range of visual and cognitive tasks, and these improvements appear to generalize widely1,2,3,4,5,6. Here we report that young adults with healthy vision, surprisingly, showed a dramatic improvement in stereo vision after playing three-dimensional, but not two-dimensional, video games for a relatively short period of time. Intriguingly, neither group showed any significant improvement in binocular contrast sensitivity. This dissociation suggests that the visual enhancement was specific to genuine stereoscopic processing, not indirectly resulting from enhanced contrast processing, and required engaging in a disparity cue-rich three-dimensional environment.

USING THE AI SERVICE RUNWAYML FOR DATA VISUALIZATION

The article provides a comprehensive analysis of domestic and international developments regarding the utilization of artificial intelligence (AI) tools for generating high-quality images and realistic three-dimensional video materials in the preparation process of advertisers. It is no secret that the main condition for successful company development is the effective operation of advertising agencies and qualified PR managers who skillfully influence public opinion in favor of the firm. To achieve this goal, they must possess modern information technologies and AI tools, understand company development strategies, and successfully conduct presentations, press conferences, and other events. The article material is dedicated to the professional training of advertisers and explores the possibilities of using artificial intelligence (AI) in their work with advertising companies. It is important to note that the subject matter we are investigating is not related to news journalism. There are other resources and opportunities for training professionals in this segment, distinct from technologies in the advertising field. The features of using the RunwayML service, which allows expanding the possibilities of visualizing complex data, are analyzed. Its potential in creating animations, 3D videos, and three-dimensional models and characters opens up new perspectives for presenting visual information by advertising and PR professionals. Particularly impressive is the use of RunwayML in creating 3D videos to tell stories, which provides the opportunity to create captivating and innovative ways of presenting content. These tools open up new horizons in data visualization making them more accessible, engaging, and effective for audience perception. This indicates that AI and neural networks are not just trendy key technologies but also essential tools in the modern process of creating and presenting visual information. Their powerful potential enables process optimization, increased efficiency, and improved product quality. However, the adaptation of AI to the needs of teaching disciplines in higher education institutions still requires further study.

Three-dimensional video recordings: Accuracy, reliability, clinical and research guidelines - Reliability assessment of a 4D camera.

In addition to studying facial anatomy, stereophotogrammetry is an efficient diagnostic tool for assessing facial expressions through 3D video recordings. Current technology produces high-quality recordings but also generates extremely excessive data. Here, we compare various recording speeds for three standardized movements using the 3dMDface camera system, to assess its accuracy and reliability. A linear and two circular movements were performed using a 3D-printed cube mounted on a robotic arm. All movements were recorded initially at 60 fps (frames/second) and then at 30 and 15 fps. Recording accuracy was tested with best-fit superimpositions of consecutive frames of the 3D cube and calculation of the Mean Absolute Distance (MAD). The reliability of the recordings were tested with evaluation of the inter- and intra-examiner error. The accuracy of movement recordings was excellent at all speeds (60, 30 and 15 fps), with variability in MAD values consistently being less than 1 mm. The reliability of the camera recordings was excellent at all recording speeds. This study demonstrated that 3D recordings of facial expressions can be performed at 30 or even at 15 fps without significant loss of information. This considerably reduces the amount of produced data facilitating further processing and analyses.

Influence of collision conditions between aerosol flows of liquid droplets and solid particles typical for wet vortex dust collectors

Various industrial devices and systems make use of collisions of droplets with solid particles in aerosol flows. The characteristics of these collisions are usually predicted adopting the same approaches as for binary collisions of a liquid droplet with a solid particle. However, the conditions of such binary collisions are different from those in the spray composition. There are very few experimental findings for sprays. The purpose of this research was to experimentally determine the characteristics of interaction of droplets with particles of slurry fuel components in sprays. High-speed three-dimensional video recording made it possible to identify conditions when liquid droplets break up, and droplets and particles agglomerate to form slurry droplets. Droplet-particle interaction regimes were identified. Collective effects in a cascade of droplet-particle interactions were recognized. The frequency of collision regime occurrence was determined, considering the recorded collective effects in sprays. The characteristics of droplet-particle collisions in sprays were compared with those of binary interactions. Conditions were determined when the findings on binary collisions can be adapted to different systems with multi-phase sprays. The findings obtained promote research and development in the field of secondary atomization of composite fuel droplets.

Usefulness of three-dimensional imaging in a flexible endoscopic surgery platform with multi-degrees-of-freedom articulating devices.

Recent studies on endoscopic submucosal dissection have aimed to reduce the difficulty of the procedure by using multi-degrees-of-freedom articulating devices. In this study, we evaluated the usefulness of adding three-dimensional (3D) video imaging into simulated endoscopic submucosal dissection tasks using multi-degrees-of-freedom devices. We designed an endoscopic platform with a 3D camera and two multi-degrees-of-freedom devices. Four ex vivo bench tasks were created, and a crossover study comparing 2D and 3D conditions was conducted on 15 volunteers. In each task, performance such as procedure time and accuracy were objectively evaluated. Additionally, a comprehensive visual analogue scale questionnaire was conducted. In the tasks simulating submucosal flap grasping, marking, and full-area incision, the use of 3D imaging significantly improved the speed and accuracy of the multi-degrees-of-freedom device manipulation (p < .01). No significant differences were observed in the task that simulated the dissection procedure. Furthermore, it appears that the accuracy of recognizing curved surfaces may be reduced in the 3D environment. Operators reported subjective increases in recognizability and operability with the 3D camera, along with an increase in asthenopia (p < .01). 3D vision improves the technical accuracy of certain simulated multi-degrees-of-freedom endoscopic submucosal dissection tasks and subjectively improved operating conditions, at the cost of increased eye strain.

Measuring 3D Video Quality of Experience (QoE) Using A Hybrid Metric Based on Spatial Resolution and Depth Cues.

A three-dimensional (3D) video is a special video representation with an artificial stereoscopic vision effect that increases the depth perception of the viewers. The quality of a 3D video is generally measured based on the similarity to stereoscopic vision obtained with the human vision system (HVS). The reason for the usage of these high-cost and time-consuming subjective tests is due to the lack of an objective video Quality of Experience (QoE) evaluation method that models the HVS. In this paper, we propose a hybrid 3D-video QoE evaluation method based on spatial resolution associated with depth cues (i.e., motion information, blurriness, retinal-image size, and convergence). The proposed method successfully models the HVS by considering the 3D video parameters that directly affect depth perception, which is the most important element of stereoscopic vision. Experimental results show that the measurement of the 3D-video QoE by the proposed hybrid method outperforms the widely used existing methods. It is also found that the proposed method has a high correlation with the HVS. Consequently, the results suggest that the proposed hybrid method can be conveniently utilized for the 3D-video QoE evaluation, especially in real-time applications.

Incoherent Digital Holography Aiming to Capture Three-Dimensional Video

Incoherent Digital Holography Aiming to Capture Three-Dimensional Video

Three-dimensional video imaging of dynamic temperature field of transparent objects recorded by a single-view parallel phase-shifting digital holography

Three-dimensional video imaging of dynamic temperature field of transparent objects recorded by a single-view parallel phase-shifting digital holography

Discrepancy between subjective and objective health indicators when performing monotonous operator activities against the background of chronic sleep deprivation

The aim of the study was to investigate various psychophysiological predictors of the quality of monotonous activity performance against the background of sleep deprivation. Fourteen subjects aged 18–22 years took part in the experiments. Each subject participated in four experimental sessions with partially automated control scenarios in a computerized driving simulator: the first experiment (15 minutes) – training session in the simulator without sleep deprivation; the second experiment – sleep deprivation (90 minutes); the third experiment – sleep deprivation and periodic exposure to a massage seat cover (90 minutes); the fourth experiment – sleep deprivation and periodic spraying of peppermint oil from an aroma diffuser (90 minutes). Every 3–7 minutes, subjects were asked to respond to two types of stimuli: 1) gradually changing; 2) appearing unexpectedly and requiring maneuvering. Stimulus intervals and stimulus order were varied randomly. Car driving simulator data, electroencephalogram using the standard 10–20 system with a sampling rate of 1000 Hz, electrocardiogram and skin-galvanic response were recorded. A three-dimensional video camera was mounted on a monitor in front of the subject to track head position and changes of the subject's facial expressions throughout the experiment. Subjects completed questionnaires on sleepiness and general well-being, and reaction time was also tested before the experiment. According to the results of the questionnaires and test performance, the subjects were in a state of chronic sleep deprivation and chronic stress. No statistically significant differences between the experiments after partial sleep deprivation without additional stimulation, with the use of a massage seat cover and periodic spraying of peppermint oil every 10 minutes were seen. The eye closure rate (PERCLOS) and spectral index of heart rate variability showed a significant statistically reliable increase before errors. The values of physiological indices before errors indicate different causes of errors in experiments without and with exposure to peppermint oil or massage seat cover.

Depth Perception Assessment of 3D Videos Based on Stereoscopic and Spatial Orientation Structural Features

Depth quality of stereoscopic three-dimensional (S3D) videos is a significant factor which directly affects the quality of experience (QoE) associated with 3D video applications and services. Nevertheless, there remain limited reports on the investigation of depth perception and depth quality evaluation of S3D videos, which impedes further advancement and deployment of 3D video technology. This paper reports a series of subjective experiments which have been conducted to investigate the depth perception and its related properties of the human visual system (HVS) using S3D video compressed by the H.264/AVC standard. The experimental results reveal that the HVS response in depth perception varies at different frequencies and in varying orientations, and the distortions introduced by video coding can cause the loss of and/or variation in depth perception. By integration of binocular and monocular features (BM) extracted from left and right views of S3D video with respect to depth perception, a depth quality assessment model, herein referred to as BM-DQAM, is devised by training these stereoscopic and spatial orientation structural features with a support vector regression model. It is shown that the BM-DQAM provides a novel no-reference metric for evaluation of the depth quality of S3D videos. Based on two publicly available 3D video databases and the proposed depth perception assessment database, the experimental results show that the BM-DQAM has demonstrated better performance in assessing the depth quality in S3D video viewing than that of other metrics reported in the published literatures, correlating well with the HVS response in the depth perception assessment experiment.

Submerged single-photon LiDAR imaging sensor used for real-time 3D scene reconstruction in scattering underwater environments.

We demonstrate a fully submerged underwater LiDAR transceiver system based on single-photon detection technologies. The LiDAR imaging system used a silicon single-photon avalanche diode (SPAD) detector array fabricated in complementary metal-oxide semiconductor (CMOS) technology to measure photon time-of-flight using picosecond resolution time-correlated single-photon counting. The SPAD detector array was directly interfaced to a Graphics Processing Unit (GPU) for real-time image reconstruction capability. Experiments were performed with the transceiver system and target objects immersed in a water tank at a depth of 1.8 meters, with the targets placed at a stand-off distance of approximately 3 meters. The transceiver used a picosecond pulsed laser source with a central wavelength of 532 nm, operating at a repetition rate of 20 MHz and average optical power of up to 52 mW, dependent on scattering conditions. Three-dimensional imaging was demonstrated by implementing a joint surface detection and distance estimation algorithm for real-time processing and visualization, which achieved images of stationary targets with up to 7.5 attenuation lengths between the transceiver and the target. The average processing time per frame was approximately 33 ms, allowing real-time three-dimensional video demonstrations of moving targets at ten frames per second at up to 5.5 attenuation lengths between transceiver and target.

Continual creation: Idealism, metaphor and the representation of spatio-temporality

In this examination of spatio-temporality’s representation in two- and three-dimensional video games, diegetic time and how a game’s overriding metaphors may be reflected in both gameplay and a storyworld’s virtual environment are considered at length through the idealist philosophy of George Berkeley, with an important adaptation of what Ludwig Wittgenstein referred to as ‘information-time’ experienced by conscious players operating in ‘memory-time’. After considering a number of examples from Outer Wilds and the Legend of Zelda franchise, the question as to whether video games are ontically presentist is resolved by way of a synthesis between the expression of time in spatial terms and every video game’s minimum of interactivity, a distinction I adapt from Henri Bergson and Maurice Merleau-Ponty. The continual creation of a digital world, while materially a fact of hardware limitations, is shown to lend itself metaphorically to an eidetic reading, bound to our intuitive association of time and space with inhabitation.

Binocular Rivalry Impact on Macroblock-Loss Error Concealment for Stereoscopic 3D Video Transmission

Three-dimensional video services delivered through wireless communication channels have to deal with numerous challenges due to the limitations of both the transmission channel's bandwidth and receiving devices. Adverse channel conditions, delays, or jitters can result in bit errors and packet losses, which can alter the appearance of stereoscopic 3D (S3D) video. Due to the perception of dissimilar patterns by the two human eyes, they can not be fused into a stable composite pattern in the brain and hence try to dominate by suppressing each other. Thus, a psychovisual sensation that is called binocular rivalry occurs. As a result, undetectable changes causing irritating flickering effects are seen, leading to visual discomforts such as eye strain, headache, nausea, and weariness. This study addresses the observer's quality of experience (QoE) by analyzing the binocular rivalry impact on the macroblock (MB) losses in a frame and its error propagation due to predictive frame encoding in stereoscopic video transmission systems. To simulate the processing of experimental videos, the Joint Test Model (JM) reference software has been used as it is recommended by the International Telecommunication Union (ITU). Existing error concealing techniques were then applied to the contiguous lost MBs for a variety of transmission impairments. In order to validate the authenticity of the simulated packet loss environment, several objective evaluations were carried out. Standard numbers of subjects were then engaged in the subjective testing of common 3D video sequences. The results were then statistically examined using a standard Student's t-test, allowing the impact of binocular rivalry to be compared to that of a non-rivalry error condition. The major goal is to assure error-free video communication by minimizing the negative impacts of binocular rivalry and boosting the ability to efficiently integrate 3D video material to improve viewers' overall QoE.