Virtual Display Research Articles

Experimental Analysis of Freehand Multi-object Selection Techniques in Virtual Reality Head-Mounted Displays

Object selection is essential in virtual reality (VR) head-mounted displays (HMDs). Prior work mainly focuses on enhancing and evaluating techniques for selecting a single object in VR, leaving a gap in the techniques for multi-object selection, a more complex but common selection scenario. To enable multi-object selection, the interaction technique should support group selection in addition to the default pointing selection mode for acquiring a single target. This composite interaction could be particularly challenging when using freehand gestural input. In this work, we present an empirical comparison of six freehand techniques, which are comprised of three mode-switching gestures (Finger Segment, Multi-Finger, and Wrist Orientation) and two group selection techniques (Cone-casting Selection and Crossing Selection) derived from prior work. Our results demonstrate the performance, user experience, and preference of each technique. The findings derive three design implications that can guide the design of freehand techniques for multi-object selection in VR HMDs.

Ultra-high brightness Micro-LEDs with wafer-scale uniform GaN-on-silicon epilayers

Owing to high pixel density and brightness, gallium nitride (GaN) based micro-light-emitting diodes (Micro-LEDs) are considered revolutionary display technology and have important application prospects in the fields of micro-display and virtual display. However, Micro-LEDs with pixel sizes smaller than 10 μm still encounter technical challenges such as sidewall damage and limited light extraction efficiency, resulting in reduced luminous efficiency and severe brightness non-uniformity. Here, we reported high-brightness green Micro-displays with a 5 μm pixel utilizing high-quality GaN-on-Si epilayers. Four-inch wafer-scale uniform green GaN epilayer is first grown on silicon substrate, which possesses a low dislocation density of 5.25 × 108 cm−2, small wafer bowing of 16.7 μm, and high wavelength uniformity (standard deviation STDEV < 1 nm), scalable to 6-inch sizes. Based on the high-quality GaN epilayers, green Micro-LEDs with 5 μm pixel sizes are designed with vertical non-alignment bonding technology. An atomic sidewall passivation method combined with wet treatment successfully addressed the Micro-LED sidewall damages and steadily produced nano-scale surface textures on the pixel top, which unlocked the internal quantum efficiency of the high-quality green GaN-on-Si epi-wafer. Ultra-high brightness exceeding 107 cd/m2 (nits) is thus achieved in the green Micro-LEDs, marking the highest reported results. Furthermore, integration of Micro-LEDs with Si-based CMOS circuits enables the realization of green Micro-LED displays with resolution up to 1080 × 780, realizing high-definition playback of movies and images. This work lays the foundation for the mass production of high-brightness Micro-LED displays on large-size GaN-on-Si epi-wafers.

Cu2O/Cu-MOF/Fe-MIL-88B with peroxidase activity for intelligent detection of p-aminophenol and barbituric acid and virtual reality display for detection mechanism

The bimetallic organic framework nanoenzyme Cu2O/Cu-MOF/Fe-MIL-88B is synthesized by the solvothermal method. The characterizations of transition electron microscopy (TEM), X-ray photoelectron spectroscopy (XPS) and cyclic voltammetry prove that the amounts of oxygen vacancies are existed on the surface of Cu2O/Cu-MOF/Fe-MIL-88B and stronger electron transfer is the key factor for free radical •OH and O2·- production, thus exhibiting excellent peroxidase activity of Cu2O/Cu-MOF/Fe-MIL-88B. On the basis of the excellent peroxidase activity, versatile colourimetric and intelligent detection platforms for p-aminophenol and barbituric acid are successfully constructed on site and timely. The linear ranges of p-aminophenol and barbituric acid are 25–75 μM and 5–45 μM, and the lowest limits of detection (LOD) are 0.51 μM and 0.46 μM, severally. Moreover, the mechanisms for detecting p-aminophenol and barbituric acid were displayed by virtual reality technology. Therefore, the colourimetric and intelligent sensor of p-aminophenol and barbituric acid based on the peroxidase of Cu2O/Cu-MOF/Fe-MIL-88B have a hopeful prospect in the field of biomedicine.

The contribution of dynamic cues to externalization of virtual source in virtual auditory display

It is well known that static virtual auditory display with headphone is inclined to create the auditory perception of in-head-localization, especially for target virtual sources in the median plane. By introducing the dynamic variation of binaural pressures caused by head turning, dynamic virtual auditory display eliminates the in-head-localization effectively. The present work investigates the contributions of dynamic interaural time difference (ITD) and dynamic interaural level difference (ILD) in binaural pressures to the externalization of auditory perception. By modifying the head-related transfer functions used in a dynamic virtual auditory display, binaural signals with dynamic ITD or dynamic ILD alone are created and presented by headphone. A psychoacoustic experiment is conducted to evaluate the extent of perceived externalization. The experiment results prove that, for the target virtual source in the median plane, dynamic ITD and dynamic ILD contribute to the externalization of low and high frequency components of the stimuli respectively. For wideband stimulus with low and high frequency components, both dynamic ITD and dynamic ILD contribute externalization but dynamic ITD contributes more than that of dynamic ILD.

How does visual context of urban open spaces drive soundscape pleasantness and eventfulness: a laboratory experiment using a higher order Ambisonics speaker system and immersive virtual reality

Perception of sound environments is influenced by the context we experience them in. A big portion of the contextual information comes from the visual domain, including our understanding of what a place is but also from its visual features. A laboratory study combining questionnaire responses and eye-tracking tools was designed to investigate if the soundscape outcomes and participants' behaviour inside the simulation can be explained by the perceptual outcomes defined by visual information. 360 degree videos and First Order Ambisonics audio recordings of 27 different urban open spaces taken from the International Soundscape Database were used as the stimuli delivered via a Virtual Reality Head-Mounted Display and a Higher Order Ambisonics speaker array, while a neutral grey environment without sounds being reproduced represented the baseline scenario. A questionnaire tool was deployed within the IVR simulation to collect participants' responses describing their perception of the reproduced environments corresponding to the circumplex model featured in the Method A of the ISO/TS 12913-2. The results revealed a good coverage of the two-dimensional perceptual circumplex space and significant differences between perceptual outcomes driven by sound and those driven by visual stimuli.

Development and Initial Testing of an Artificial Intelligence-Based Virtual Reality Companion for People Living with Dementia in Long-Term Care.

Background/Objectives: Feelings of loneliness are common in people living with dementia (PLWD) in long-term care (LTC). The goals of this study were to describe the development of a novel virtual companion for PLWD living in LTC and assess its feasibility and acceptability. Methods: The computer-generated virtual companion, presented using a head-mounted virtual reality display, was developed in two stages. In Stage 1, the virtual companion asked questions designed to encourage conversation and reminiscence. In Stage 2, more powerful artificial intelligence tools allowed the virtual companion to engage users in nuanced discussions on any topic. PLWD in LTC tested the application at each stage to assess feasibility and acceptability. Results: Ten PLWD living in LTC participated in Stage 1 (4 men and 6 women; average 82 years old) and Stage 2 (2 men and 8 women; average 87 years old). Session lengths ranged from 0:00 to 5:30 min in Stage 1 and 0:00 to 53:50 min in Stage 2. Speech recognition issues and a limited repertoire of questions limited acceptance in Stage 1. Enhanced conversational ability in Stage 2 led to intimate and meaningful conversations with many participants. Many users found the head-mounted display heavy. There were no complaints of simulator sickness. The virtual companion was best suited to PLWD who could engage in reciprocal conversation. After Stage 2, response latency was identified as an opportunity for improvement in future versions. Conclusions: Virtual reality and artificial intelligence can be used to create a virtual companion that is acceptable and enjoyable to some PLWD living in LTC. Ongoing innovations in hardware and software will allow future iterations to provide more natural conversational interaction and an enhanced social experience.

A Digital Twin System for Adaptive Aligning of Large Cylindrical Components

Most large aerospace cylindrical components still adopt a manual aligning method with low automation, large manual intervention, and heavy dependence on operator workers, resulting in the low quality and efficiency of large component aligning, which seriously prolongs the manufacturing time of aerospace products. To cope with this issue, based on closed-loop adaptive control and digital twin (DT) technologies, an adaptive aligning system for large cylindrical components, i.e., the DT aligning system, is proposed in this study. For the DT aligning system, through the DT multi-dimensional modeling, i.e., geometric modeling, physical modeling, functional modeling, and data modeling, it can be divided into a physical space, a virtue space, and twin data. Note that the association, mapping, and interaction between physical space and virtual space of the aligning system can be realized via the twin data, thereby realizing real-time virtual display, monitoring, and control of the large component aligning. In addition, based on the measured pose data, aligning stress, and predicted aligning error, an adaptive force/position control method for large component aligning is proposed, and it can achieve real-time decision-making and precise execution of the aligning process. Finally, through application validation, the DT process system can realize the real-time status perception and process execution decision during the large component aligning. Finally, through experimental validation, it is found that the proposed system, i.e., the DT aligning system, can improve the quality and efficiency of the large aerospace cylindrical component aligning, as well as the automation and intelligent level of the aligning system.

AR/VR assisted integrated framework of autonomous disassembly system for industrial products

An immersive Augmented Reality (AR) technology enhances productivity and flexibility in manufacturing by providing real-time instructions within workers’ field of view, aiding in complex tasks. According to cognitive load and attention-based theories, AR guidance can reduce cognitive strain, enhancing task performance. The affordance of AR in industrial processes like assembly or disassembly for repair/maintenance has been exploited broadly. However, the prevailing studies on automated user interfaces for AR instruction development, specifically virtual scene creation and target generation for object mapping rendering, still need to be explored. Notably, production industries currently rely on outsourcing skilled professionals to create AR overlaying instructions using manufacturer input. However, the exactness of the disassembly repository is substantial to avoid defects in AR guidance particulars. This underscores the significance of generalized automated AR instruction development and emphasizes the demands for further research. In this work, the autonomous AR-integrated framework is developed for the disassembly context, empowering manufacturers to acquire desired outcomes across various modules. These modules include the validation of the manual user disassembly sequence repository during the prefatory phase of AR instruction creation and the provision of Disassembly Sequence Planning (DSP) input. Additionally, the framework supports the simulation/rendering of disassembly efforts in a virtual platform and the generation of rendering virtual display scenes for appropriate AR-guided object mapping. Furthermore, it enables the generation of essential model targets for registering the AR scenes. Subsequently, the extracted repositories (virtual scenes and model targets) are input into the Unity AR tool to create AR mapping guidance for disassembly operations. The performance of the autonomous disassembly module is evaluated through three case studies with different AR visualizations. Furthermore, the potential of the proposed framework is exploited by extracting feedback from experts regarding flexibility, compatibility, completion time, and cognitive level.

Decline in Sensory Integration in Old Age and Its Related Functional Brain Connectivity Correlates Observed during a Virtual Reality Task.

Sensory integration is an essential human function whose decline impacts quality of life, particularly in older adults. Herein, we propose an arm-reaching task based on a virtual reality head-mounted display system to assess sensory integration in daily life, and we examined whether reaching task performance was associated with resting-state functional connectivity (rsFC) between the brain regions involved in sensory integration. We hypothesized that declining sensory integration would affect performance during a reaching task with multiple cognitive loads. Using a task in which a young/middle-aged group showed only small individual differences, older adults showed large individual differences in the gap angle between the reaching hand and the target position, which was used to assess sensory integration function. Additionally, rsfMRI data were used to identify correlations between rsFC and performance in older adults, showing that performance was correlated with connectivity between the primary motor area and the left inferior temporal gyrus and temporo-occipital region. Connectivity between areas is related to visuomotor integration; thus, the results suggest the involvement of visuomotor integration in the decline of sensory integration function and the validity of the gap angle during this VR reaching task as an index of functional decline.

Virtual Exhibition Platform Display and Works Presentation Experience Based on Computer Vision

Virtual Exhibition Platform Display and Works Presentation Experience Based on Computer Vision

Virtual Reality Head-Mounted Display (HMD) and Preoperative Patient-Specific Simulation: Impact on Decision-Making in Pediatric Urology: Preliminary Data.

To assess how virtual reality (VR) patient-specific simulations can support decision-making processes and improve care in pediatric urology, ultimately improving patient outcomes. Children diagnosed with urological conditions necessitating complex procedures were retrospectively reviewed and enrolled in the study. Patient-specific VR simulations were developed with medical imaging specialists and VR technology experts. Routine CT images were utilized to create a VR environment using advanced software platforms. The accuracy and fidelity of the VR simulations was validated through a multi-step process. This involved comparing the virtual anatomical models to the original medical imaging data and conducting feedback sessions with pediatric urology experts to assess VR simulations' realism and clinical relevance. A total of six pediatric patients were reviewed. The median age of the participants was 5.5 years (IQR: 3.5-8.5 years), with an equal distribution of males and females across both groups. A minimally invasive laparoscopic approach was performed for adrenal lesions (n = 3), Wilms' tumor (n = 1), bilateral nephroblastomatosis (n = 1), and abdominal trauma in complex vascular and renal malformation (ptotic and hypoplastic kidney) (n = 1). Key benefits included enhanced visualization of the segmental arteries and the deep vascularization of the kidney and adrenal glands in all cases. The high depth perception and precision in the orientation of the arteries and veins to the parenchyma changed the intraoperative decision-making process in five patients. Preoperative VR patient-specific simulation did not offer accuracy in studying the pelvic and calyceal anatomy. VR patient-specific simulations represent an empowering tool in pediatric urology. By leveraging the immersive capabilities of VR technology, preoperative planning and intraoperative navigation can greatly impact surgical decision-making. As we continue to advance in medical simulation, VR holds promise in educational programs to include even surgical treatment of more complex urogenital malformations.

The Evolution of Cultural Relics: Environmental Sculpture of Huai'an in Metaverse

In recent years, with the rapid development of digital technology, cultural have gradually introduced the concept of digital system. Taking the environmental sculpture of Huai'an city as an example, this paper discusses the application of the meta-cosmic digital system in the cultural relics records, and analyzes the significance of this technology for transnational exchanges and mutual learning among civilizations. Through the in-depth study of the management of Huaian environmental sculpture, this paper finds that the digital system of the universe can also realize the real-time recording, virtual display and global sharing of cultural relics, thus providing a new way and method for the protection and inheritance of cultural relics. At the same time, the use of metaso digital system can also promote the innovation and artistic exchange of urban sculpture, and promote the process of mutual learning among civilizations. Therefore, the application of metaso digital system in cultural records has great theoretical and practical significance.

Advances in personalized modelling and virtual display of ethnic clothing for intelligent customization

Abstract To enhance the intelligence of the ethnic clothing customization industry and provide online solutions for clothing ordering and fashion shows, this study analyses the latest developments in the fields of digital human models, 3D clothing models, and virtual clothing displays in recent years. This study focuses on analysing the categories and processes of three related technologies in clothing virtual display and compares the advantages and disadvantages of four human body modelling methods. Reshaping the facial features of a 3D body scan model and binding the bones of each part is key to providing a personalized body model. In addition, 3D clothing modelling technology is classified into three methods. The 3D fabric scan data are used to establish realistic seam information and process details simulation and use hybrid modelling methods to virtually sew 2D samples, which can enhance the realism of clothing models. The virtual display platform for ethnic clothing is constructed through layered design based on the Unity3D engine, matching versions and virtual display. The above research ideas can solve bottleneck problems in clothing customization. This has important guiding significance for opening up the ethnic clothing customization market, promoting the intelligence of the customization industry, and promoting ethnic clothing culture.

Viewpoint-sharing method with reduced motion sickness in object-based VR/AR collaborative virtual environment

We propose a viewpoint-sharing method with reduced motion sickness in an object-based remote collaborative virtual environment (CVE). The method is designed with an assumption of asymmetric, object-based CVE where collaborators use non-homogeneous devices, such as immersive virtual reality head-mounted display (VR HMD) and tablet-based augmented reality (AR), and simultaneously interact with 3D virtual objects. Therefore, collaborators interact with different interfaces such as virtual reality (VR) users relying on controllers for virtual locomotion and object manipulation, while AR users perform physical locomotion and multi-touch input for object manipulation. The proposed viewpoint-sharing method allows both users to observe and manipulate the objects in interest from the shared point of view, enabling participants to interact with the objects without the need for virtual/physical locomotion. While viewpoint-sharing, instead of changing point of view, the proposed method performs seamless object transformation to provide a shared point of view, reducing motion sickness and associated discomfort. From our user experiment, the viewpoint-share condition resulted in a 35.47% faster task completion time than the baseline condition which is without proposed viewpoint-sharing. The advantage of viewpoint-sharing regarding system usability was significant, while task workloads were similar in the baseline and viewpoint-sharing conditions. We expect that the proposed viewpoint-sharing method allows users to quickly, efficiently, and collaboratively communicate in an object-based CVE, and represents a step forward in the development of effective remote, asymmetric CVE.

Effect of Self-Training Using Virtual Reality Head-Mounted Display Simulator on the Acquisition of Holmium Laser Enucleation of the Prostate Surgical Skills.

We aimed to evaluate the effect of self-training using a virtual reality head-mounted display simulator on the acquisition of surgical skills for holmium laser enucleation surgery. Thirteen medical students without surgical skills for holmium laser enucleation of the prostate were trained using multimedia to learn the technique via simulator manipulation. Thereafter, participants performed the technique on a virtual benign prostatic hyperplasia model A (test A). After a 1-week wash-out period, they underwent self-training using a simulator and performed the technique on model B (test B). Subsequently, participants were asked to respond to Training Satisfaction Questions. Video footage of hand movements and endoscope view were recorded during tests A and B for later review by 2 expert surgeons. A 20-step Assessment Checklist, 6-domain Global Rating Scale, and a Pass Rating were used to compare performance on tests A and B. Thirteen participants completed both tests A and B. The 20-step Assessment Checklist and 6-domain Global Rating Scale evaluation results showed significantly improved scores in test B than in test A (P<0.05). No evaluator rated participants as passed after test A, but 11 participants (84.6%) passed after test B. Ten participants (76.9%) indicated that the simulator was helpful in acquiring surgical skills for holmium laser enucleation of the prostate. The virtual reality head-mounted display holmium laser enucleation of the prostate simulator was effective for surgical skill training. This simulator may help to shorten the learning curve of this technique in real clinical practice in the future.

P‐177: Effect of metal BM as inducer electrode in liquid crystal display

In this paper, we propose a simulation method to evaluate the transmittance and response time when using metal BM in the Virtual Reality display. The influence of pixel design, the voltage of metal BM ; LC type are discussed in detail. Distinguishing the contribution of the above factors on display can further pave the way to Tr improvement of LCD‐based VR application.

Design and Implementation of Ceramic Virtual Display System

Using virtual reality technology to achieve the three-dimensional digitalization of ceramic artworks and accessing these 3D digital models through a web browser can provide a new approach for the promotion and dissemination of traditional ceramic culture. This paper aims to preserve traditional ceramic culture and craftsmanship as intangible cultural heritage, enhancing public understanding of traditional ceramic culture. Taking Jingdezhen ceramics as an example, various ceramics are three-dimensionally modeled using modeling software. A virtual exhibition hall is created in Unity3D, where the ceramic models are placed and interactive features are built, enabling functionalities such as virtual tours of the ceramic exhibition, ceramic viewing, and ceramic making. By leveraging WebGL, users can access the virtual exhibition system through a web browser. The virtual exhibition system promotes public understanding of ceramic culture and virtual reality technology, revitalizing traditional artifacts and promoting the inheritance and digital preservation of ceramic culture.

Design for a virtual reality head-mounted display with holographic pancake optics

A design for a head-mounted display using holographic optical elements (HOEs) and pancake optics is presented. The pancake optics assembly consists of a circular polarization, quarter-wave film, and a coated 50/50 (semi-transmissive and semi-reflective) aspherical lens. The HOE and pancake optics combination was employed to reduce the volume of the system. For this design, the effective focal length was 10.901 mm, the field of view was 30 deg, and the f-number was 3.633. In terms of image quality, the modulation transfer function (MTF) values for all fields exceeded 0.3 at 65 lp/mm. The MTF values for all fields under the tolerance error also exceeded 0.3 lp/mm. The maximum optical distortion was less than 2%. The impact of the efficiency of diffracted and non-diffracted light on the visibility was analyzed to select the appropriate reflectance for the coating of the 50/50 aspherical lens and the highest hologram diffraction efficiency to achieve the best visibility.

Virtual display of wooden furniture cultural relics based on laser and CT scanning technology

The 3D reconstruction and virtual display of wooden furniture cultural relics were investigated using laser scanning and CT scanning techniques. The suitability of different 3D reconstruction techniques and virtual display approaches were considered. The experiments demonstrated that digital models obtained from both laser scanning and CT scanning can be integrated seamlessly into virtual environments created with 3DMAX for exhibition purposes. Additionally, post-processing software, such as PR or AE, can be utilized to synthesize virtual display video. The resulting images exhibit self-adaptation capabilities, with clear and undistorted 3D model and texture image. Moreover, other types of scanned models are suitable for 3D micro-scale model printing, although CT-based models tend to achieve higher printing accuracy compared to those generated by laser scanning technology. However, the precision of 3D printing model is contingent upon factors such as the precision of digital model, the type of printer, and the printing material.

Recent Applications of Optical Elements in Augmented and Virtual Reality Displays: A Review

Recent Applications of Optical Elements in Augmented and Virtual Reality Displays: A Review