Objects In Virtual Environment Research Articles

Neural digital twins: reconstructing complex medical environments for spatial planning in virtual reality.

Specialized robotic and surgical tools are increasing the complexity of operating rooms (ORs), requiring elaborate preparation especially when techniques or devices are to be used for the first time. Spatial planning can improve efficiency and identify procedural obstacles ahead of time, but real ORs offer little availability to optimize space utilization. Methods for creating reconstructions of physical setups, i.e., digital twins, are needed to enable immersive spatial planning of such complex environments in virtual reality. We present a neural rendering-based method to create immersive digital twins of complex medical environments and devices from casual video capture that enables spatial planning of surgical scenarios. To evaluate our approach we recreate two operating rooms and ten objects through neural reconstruction, then conduct a user study with 21 graduate students carrying out planning tasks in the resulting virtual environment. We analyze task load, presence, perceived utility, plus exploration and interaction behavior compared to low visual complexity versions of the same environments. Results show significantly increased perceived utility and presence using the neural reconstruction-based environments, combined with higher perceived workload and exploratory behavior. There's no significant difference in interactivity. We explore the feasibility of using modern reconstruction techniques to create digital twins of complex medical environments and objects. Without requiring expert knowledge or specialized hardware, users can create, explore and interact with objects in virtual environments. Results indicate benefits like high perceived utility while being technically approachable, which may indicate promise of this approach for spatial planning and beyond.

Application of Neural Radiance Fields (NeRFs) for 3D Model Representation in the Industrial Metaverse

This study explores the utilization of Neural Radiance Fields (NeRFs), with a specific focus on the Instant NeRFs technique. The objective is to represent three-dimensional (3D) models within the context of the industrial metaverse, aiming to achieve a high-fidelity reconstruction of objects in virtual environments. NeRFs, renowned for their innovative approach, enable comprehensive model reconstructions by integrating diverse viewpoints and lighting conditions. The study employs tools such as Unity, Photon Pun2, and Oculus Interaction SDK to develop an immersive metaverse. Within this virtual industrial environment, users encounter numerous interactive six-dimensional (6D) models, fostering active engagement and enriching the overall experience. While initial implementations showcase promising results, they also introduce computational complexities. Nevertheless, this integration forms the basis for immersive comprehension and collaborative interactions within the industrial metaverse. The evolving potential of NeRF technology promises even more exciting prospects in the future.

Perception of Soft Objects in Virtual Environments Under Conflicting Visual and Haptic Cues.

In virtual/augmented/mixed reality (VR/AR/MR) applications, rendering soft virtual objects using a hand-held haptic device is challenging due to the anatomical restrictions of the hand and the ungrounded nature of the design, which affect the selection of actuators and sensors and hence limit the resolution and range of forces displayed by the device. We developed a cable-driven haptic device for rendering the net forces involved in grasping and squeezing 3D virtual compliant (soft) objects being held between the index finger and thumb only. Using the proposed device, we investigate the perception of soft objects in virtual environments. We show that the range of object stiffness that can be effectively conveyed to a user in virtual environments (VEs) can be significantly expanded by controlling the relationship between the visual and haptic cues. We propose that a single variable, named Apparent Stiffness Difference, can predict the pattern of human stiffness perception under manipulated conflict, which can be used for rendering a range of soft objects in VEs larger than what is achievable by a haptic device alone due to its physical limits.

VR-HandNet: A Visually and Physically Plausible Hand Manipulation System in Virtual Reality.

This study aims to allow users to perform dexterous hand manipulation of objects in virtual environments with hand-held VR controllers. To this end, the VR controller is mapped to the virtual hand and the hand motions are dynamically synthesized when the virtual hand approaches an object. At each frame, given the information about the virtual hand, VR controller input, and hand-object spatial relations, the deep neural network determines the desired joint orientations of the virtual hand model in the next frame. The desired orientations are then converted into a set of torques acting on hand joints and applied to a physics simulation to determine the hand pose at the next frame. The deep neural network, named VR-HandNet, is trained with a reinforcement learning-based approach. Therefore, it can produce physically plausible hand motion since the trial-and-error training process can learn how the interaction between hand and object is performed under the environment that is simulated by a physics engine. Furthermore, we adopted an imitation learning paradigm to increase visual plausibility by mimicking the reference motion datasets. Through the ablation studies, we validated the proposed method is effectively constructed and successfully serves our design goal. A live demo is demonstrated in the supplementary video.

Comparison of the perception of space in virtual reality and in the real world

The subject of the study is investigation the accuracy and identify the features of spatial information perception in virtual reality. During the experiment, the subjects were asked to estimate the distances to objects and between objects in virtual environment in metric units without the presence of reference objects with known dimensions in the virtual environment. It was found that in the complete absence of metric support in the action space in the real world, subjects estimated metric distances with an error of about 2 meters at 8 meters, 3 meters at 13-15 meters, and 2 meters at 30 meters. In contrast, in a virtual environment, the subjects estimated a distance of 8 meters with an error of 4 meters, a distance of 13-15 meters with an error of 7-8 meters, and a distance of 30 meters with an error of 18 meters. Introducing metric support is likely to significantly improve the result, and abandoning the metric distance measurement scale, which is not natural for humans, will considerably increase the accuracy of spatial information perception. It should be noted that in the field of long-range vision (30 meters or more) in virtual reality, the accuracy of information perception is extremely low.

Locomotion Techniques for Dynamic Environments: Effects on Spatial Knowledge and User Experiences.

Various locomotion techniques are used to navigate and find way through space in virtual environments (VE), and each technique provides different experiences and performances to users. Previous studies have primarily focused on static environments, whereas there is a need for research from a different perspective of dynamic environments because there are many moving objects in VE, such as other users. In this study, we compare the effects of different locomotion techniques on the user's spatial knowledge and experience, depending on whether the virtual objects are moving or not. The investigated locomotion techniques include joystick, teleportation, and redirected walking (RDW), all commonly used for VR navigation. The results showed that the differences in spatial knowledge and user experience provided by different locomotion techniques can vary depending on whether the environment is static or dynamic. Our results also showed that for a given VE, there are different locomotion techniques that induce fewer collisions between the user and other objects, or reduce the time it takes the user to perform a given task. This study suggests that when designing a locomotion interface for a specific VR application, it is possible to improve the user's spatial knowledge and experience by recommending different locomotion techniques depending on the degree of environment dynamism and and type of task.

PLACEMENT AND CONSTRUCTION OF 3D OBJECTS IN THE SYSTEM OF THE NATIONAL VIRTUAL UNIVERSITY BASED ON 3D TECHNOLOGIES

The article discusses the fields of application and content of virtual reality systems, the results of a comparative analysis of virtual educational environment platforms, and their role in presenting educational content in a virtual electronic format. The subject of research is geometric modeling, visualization methods and software tools of 3D models of objects in virtual environments. The purpose of the research is to develop algorithms for the representation and visualization of 3D models of exterior and interior objects in the organization of a virtual university, as well as a software tool for creating 3D models in the organization of a virtual university. Scientific novelty of the research, we offer new methods for geometric modeling of 3D objects in visualization, i.e. constructive logic-algebraic R-function method (RFM). The practical results of the research are aimed at increasing the activity of learners in modern virtual 3D environments in educational institutions based on the software tool, and thereby supporting the teaching process in the virtual three-dimensional environment designed on the basis of modern national landscape design in the Uzbek language “vAcademia.uz” software was developed.

Physical Activity in Immersive Virtual Reality: A Scoping Review.

Physical activity has benefits for health, but many adolescents are inactive. However, video games such as Immersive Virtual Reality (IVR) have grown in popularity as a leisure activity among young people, allowing them to manipulate objects in virtual environments increasing the practice of physical activity. The evidence indicates that the interest in physical activity through IVR is greater than in traditional methods, and different experiences have been reported. However, few studies indicate the sample evaluated, the effects found, or the IVR instruments used. Due to this, is the aim of this study is to identify the publications referring to IVR and physical activity, characterize them, and present the obtained main findings. For this, the guidelines described in the PRISMA-ScR for scoping reviews were applied. After the use of the inclusion and exclusion criteria, eight articles were included. Results show evidence regarding physiological outcomes, perceptual variables, interest and enjoyment, and psychological effects regarding physical activity through IVR. Additionally, the use of different devices and their prescriptions are explored. It is concluded that there is interest from the scientific community for the practice of physical activity through IVR, as well as for its application for the maintenance of active habits. This is important as it positions IVR as a method that can be a more experiential and effective way to develop and maintain a healthy lifestyle.

I Can't See That! Considering the Readability of Small Objects in Virtual Environments.

Though virtual reality has repeatedly seen usability improvements through higher fidelity headsets, interacting with small objects has remained an issue due to a reduction in visual acuity. Given the current uptake of virtual reality platforms and the range of real world applications that they may be used for, it is worth considering how such interactions can be accounted for. We propose three techniques for improving the usability of small objects in virtual environments: i) expanding them in place, ii) showing a zoomed-in twin above the original object, and iii) showing a large readout of the object's current state. We conducted a study comparing each technique's usability, induced presence, and effect on short-term knowledge retention in a VR training scenario that simulated the common geoscience exercise of measuring strike and dip. Participant feedback highlighted the need for this research, however simply scaling the area of interest may not be enough to improve the usability of information-bearing objects, while displaying this information in large text format can make tasks faster to complete at the cost of reducing the user's ability to transfer knowledge they've learned to the real world. We discuss these results and their implications for the design of future virtual reality experiences.

Using the Unity 3D Environment and the Vuforia Plugin to Develop an AR Application

Currently, more and more interactive media environments are offered to users. Augmented reality (AR) technologies are one of the environments where interactive interaction is provided and which can integrate objects in virtual environments with real objects. Augmented reality technologies are used in many areas, including in the field of education. Some aspects of the development of augmented reality applications are discussed in the article. It was revealed that the Unity platform and the Vuforia plugin are used as the most popular tools by AR application developers, which have a number of advantages. The process of developing an AR application for the education sector using the Unity environment, the C# programming language and Vuforia is considered. The Unity 3D environment and the Vuforia plugin are in practice an effective combination of flexible mobile application development tools with augmented reality elements.

Design, Control, and Psychophysics of Tasbi: A Force-Controlled Multimodal Haptic Bracelet

Haptic feedback is known to enhance the realism of an individual’s interactions with objects in virtual environments. Wearable haptic devices, such as vibrotactile sleeves or armbands, can provide haptic feedback in a smaller and more lightweight form factor than haptic gloves that can be bulky and cumbersome to the wearer. In this article, we present tactile and squeeze bracelet interface (Tasbi), a multimodal haptic wristband that can provide radial squeeze forces around the wrist along with vibrotactile feedback at six discrete locations around the band. Tasbi implements a squeezing mechanism that minimizes tangential forces between the band’s points of contact with the skin, instead of focusing the motor actuation to predominantly normal forces. Force sensing capacitors enable closed-loop control of the squeeze force, while vibration is achieved with linear resonant actuators. A detailed description of the design and experimental results demonstrating closed-loop control of squeeze cues provided by Tasbi is presented. Additionally, we present the results of psychophysical experiments that quantify user perception of the vibration and squeeze cues, including vibrotactile identification accuracy in the presence of varying squeeze forces, discrimination thresholds for the squeeze force, and an analysis of user preferences for squeeze actuation magnitudes.

Two-In-One: A Design Space for Mapping Unimanual Input into Bimanual Interactions in VR for Users with Limited Movement

Virtual Reality (VR) applications often require users to perform actions with two hands when performing tasks and interacting with objects in virtual environments. Although bimanual interactions in VR can resemble real-world interactions—thus increasing realism and improving immersion—they can also pose significant accessibility challenges to people with limited mobility, such as for people who have full use of only one hand. An opportunity exists to create accessible techniques that take advantage of users’ abilities, but designers currently lack structured tools to consider alternative approaches. To begin filling this gap, we propose Two-In-One, a design space that facilitates the creation of accessible methods for bimanual interactions in VR from unimanual input. Our design space comprises two dimensions, bimanual interactions and computer assistance, and we provide a detailed examination of issues to consider when creating new unimanual input techniques that map to bimanual interactions in VR. We used our design space to create three interaction techniques that we subsequently implemented for a subset of bimanual interactions and received user feedback through a video elicitation study with 17 people with limited mobility. Our findings explore complex tradeoffs associated with autonomy and agency and highlight the need for additional settings and methods to make VR accessible to people with limited mobility.

Size Perception Bias and Reach-to-Grasp Kinematics: An Exploratory Study on the Virtual Hand With a Consumer Immersive Virtual-Reality Device

While studies have increasingly used virtual hands and objects in virtual environments to investigate various processes of psychological phenomena, conflicting findings have been reported even at the most basic level of perception and action. To reconcile this situation, the present study aimed 1) to assess biases in size perception of a virtual hand using a strict psychophysical method and 2) to provide firm and conclusive evidence of the kinematic characteristics of reach-to-grasp movements with various virtual effectors (whole hand or fingertips only, with or without tactile feedback of a target object). Experiments were conducted using a consumer immersive virtual reality device. In a size judgment task, participants judged whether a presented virtual hand or an everyday object was larger than the remembered size. The results showed the same amplitude of underestimation (approximately 5%) for the virtual hand and the object, and no influence of object location, visuo-proprioceptive congruency, or short-term experience of controlling the virtual hand. Furthermore, there was a moderate positive correlation between actual hand size and perception bias. Analyses of reach-to-grasp movements revealed longer movement times and larger maximum grip aperture (MGA) for a virtual, as opposed to a physical, environment, but the MGA did not change when grasping was performed without tactile feedback. The MGA appeared earlier in the time course of grasping movements in all virtual reality conditions, regardless of the type of virtual effector. These findings confirm and corroborate previous evidence and may contribute to the field of virtual hand interfaces for interactions with virtual worlds.

Object manipulation in immersive virtual environments: Hand Motion tracking technology and snap-to-fit function

There have been recent efforts to use virtual reality and manipulation to train construction workers and inspection. However, there is a lack of research efforts comparing and evaluating virtual manipulation hardware for construction tasks. Moreover, the current practice of virtual manipulation has limited functionality to guide users with the placement of objects in virtual environments. To address these issues, this paper presents 1) a detailed case study that compares three types of manipulation hardware (image-based, infrared-based, and magnetic-based) for construction applications and 2) a snap-to-fit method that improves the VM through solving limitations of the advanced virtual reality interaction metaphors. The latter enhances the placement process for manipulation by comparing two models (e.g., scan and BIM models) for proper placement in assembly scenarios. The case study results show that magnetic-based systems outperform others in construction scenarios. Lastly, the snap-to-fit function was validated in terms of accuracy and time performance.

Interactive 3D Modeling with Virtual Reality

The immersive Virtual Reality (VR) environment transforms the way people learn, work, and communicate. While the traditional modeling tools such as Blender and AutoCAD are commonly used for industrial design today, handling 3D objects on a two-dimensional screen with a keyboard and mouse is very challenging. In this work, we introduce a VR modeling system named The Virtual Workshop supporting the design and manipulation of various 3D objects in virtual environments. The proposed system was developed for the Oculus Rift platform allowing a user to efficiently and precisely design 3D objects using two hands directly. The friendly system GUI supports the user to create new 3D objects from scratch using premade “basic objects” or alternatively to import an existing 3D object made in other applications. Meanwhile, the finished 3D models are stored in the standard OBJ file format and exported later for the development of 3D scenarios in other applications such as the Unity engine. From concept to design, the VR modeling system provides an open platform where the designers and the clients can better share their ideas as well as interactively refine the rendered virtual models in collaboration.

3D Mirrored Object Selection for Occluded Objects in Virtual Environments

There is an increasing demand for the manipulation of virtual objects in 3D virtual reality (VR) space, which begins with the user selecting a desired object. Existing selection methods aim to provide an intuitive and natural experience that minimizes user fatigue. Although typical ray-casting methods are effective when objects in the virtual space are sparsely or evenly placed, the selection becomes challenging when these objects are smaller or farther away, particularly when they overlap. In this study, we propose a method to address these challenges via an effective selection process. Based on 3D mirroring, the proposed method provides a new interaction metaphor that allows for a more efficient selection of occluded virtual objects. We conduct systematic experiments with 21 subjects and use various levels of target visibility determined by object size, spatial density, and degrees of occlusion. The results indicate that, in occluded 3D environments, the proposed 3D mirrored selection process outperforms traditional gaze-supported selection processes and offers better user satisfaction.

Sign Language Recognition System for Disabled People

The aim is to present a real time system for hand gesture recognition on the basis of detection of some meaningful shape based feature like orientation, center of mass, status of fingers in term of raised or folded fingers of hand and their respective location in image. Hand gesture Recognition System has various real time applications in natural, innovative, user friendly way of how to interact with the computer which has more facilities that are familiar to us. Gesture recognition has a wide area of application including Human machine interaction, sign language, game technology robotics etc are some of the areas where Gesture recognition can be applied. More specifically hand gesture is used as a signal or input means given to the computer especially by disabled person. Being an interesting part of the human and computer interaction hand gesture recognition is needed for real life application, but complex of structures presents in human hand has a lot of challenges for being tracked and extracted. Making use of computer vision algorithms and gesture recognition techniques will result in developing low-cost interface devices using hand gestures for interacting with objects in virtual environment. SVM (support vector machine) and efficient feature extraction technique is presented for hand gesture recognition. This method deals with the dynamic aspects of hand gesture recognition system.

The Effects of Augmented Reality Interaction Techniques on Egocentric Distance Estimation Accuracy

Recent developments in virtual environment applications allow users to interact with three-dimensional (3D) objects in virtual environments. As interaction with 3D objects in virtual environments becomes more established, it is important to investigate user performance with such interaction techniques within a specific task. This study investigated two interaction modes, direct and indirect, depending on how the users interacted with the 3D objects, by measuring the accuracy of egocentric distance estimation in a stereoscopic environment. Fourteen participants were recruited to perform an acquisition task with both direct pointing and indirect cursor techniques at three egocentric distances and three task difficulty levels. The accuracy of the egocentric distance estimation, throughput, and task completion time were analyzed for each interaction technique. The indirect cursor technique was found to be more accurate than the direct pointing one. On the other hand, a higher throughput was observed with the direct pointing technique than with the indirect cursor technique. However, there were no significant differences in task completion time between the two interaction techniques. The results also showed accuracy to be higher at the greatest distance (150 cm from the participant) than at the closer distances of 90 cm and 120 cm. Furthermore, the difficulty of the task also significantly affected the accuracy, with accuracy lower in the highest difficulty condition than in the medium and low difficulty conditions. The findings of this study contribute to the understanding of user-interaction techniques in a stereoscopic environment. Furthermore, developers of virtual environments may refer to these findings in designing effective user interactions, especially those in which performance relies on accuracy.

Simple and effective deep hand shape and pose regression from a single depth image

Simultaneously estimating the 3D shape and pose of a hand in real time is a new and challenging computer graphics problem, which is important for animation and interactions with 3D objects in virtual environments with personalized hand shapes. CNN-based direct hand pose estimation methods are the state-of-the-art approaches, but they can only regress a 3D hand pose from a single depth image. In this study, we developed a simple and effective real-time CNN-based direct regression approach for simultaneously estimating the 3D hand shape and pose, as well as structure constraints for both egocentric and third person viewpoints by learning from the synthetic depth. In addition, we produced the first million-scale egocentric synthetic dataset called SynHandEgo, which contains egocentric depth images with accurate shape and pose annotations, as well as color segmentation of the hand parts. Our network is trained based on combined real and synthetic datasets with full supervision of the hand pose and structure constraints, and semi-supervision of the hand mesh. Our approach performed better than the state-of-the-art methods based on the SynHand5M synthetic dataset in terms of both the 3D shape and pose recovery. By learning simultaneously using real and synthetic data, we demonstrated the feasibility of hand mesh recovery from two real hand pose datasets, i.e., BigHand2.2M and NYU. Moreover, our method obtained more accurate estimates of the 3D hand poses based on the NYU dataset compared with the existing methods that output more than joint positions. The SynHandEgo dataset has been made publicly available to promote further research in the emerging domain of hand shape and pose recovery from egocentric viewpoints (https://bit.ly/2WMWM5u).

Impending Collision Judgment from an Egocentric Perspective in Real and Virtual Environments: A Review.

The human egocentric perception of approaching objects and the related perceptual processes have been of interest to researchers for several decades. This article gives a literature review on numerous studies that investigated the phenomenon when an object approaches an observer (or the other way around) with the goal to single out factors that influence the perceptual process. A taxonomy of metrics is followed by a breakdown of different experimental measurement methods. Thereinafter, potential factors affecting the judgment of approaching objects are compiled and debated while divided into human factors (e.g., gender, age, and driving experience), compositional factors (e.g., approaching velocity, spatial distance, and observation time), and technical factors (e.g., field of view, stereoscopy, and display contrast). Experimental findings are collated, juxtaposed, and critically discussed. With virtual-reality devices having taken a tremendous developmental leap forward in the past few years, they have been able to gain ground in experimental research. Therefore, special attention in this article is also given to the perception of approaching objects in virtual environments and put in contrast to the perception in reality.