Large Virtual Environments Research Articles

The metaverse in the world of health: The present future. Challenges and opportunities.

The World Health Organization has defined "digital health" as the use of information and communication technologies to improve health. In recent years, there has been a strong acceleration in the adoption of these digital tools, which has had a major impact on traditional healthcare models. We are currently witnessing the emergence of a large immersive virtual environment called the "metaverse." Its emergence creates new and challenging opportunities in health care. This article explores some metaverse-related concepts, provides specific examples of its use in pediatrics, describes experiences in the hospital setting, and finally delves into the resulting challenges and opportunities.

An Approach to Recreate Large Virtual Environments for Use in Road Traffic Research

An Approach to Recreate Large Virtual Environments for Use in Road Traffic Research

Redirected walking for exploration of unknown environments

Real walking is the most natural locomotion method for exploring Virtual Environments (VE), enhancing the immersion of Virtual Reality (VR). Redirected Walking (RDW) is employed to enable real walking within limited tracking spaces in large VEs by subtly manipulating the mapping between the virtual and real environments. However, the effectiveness of RDW is greatly influenced by the convex shape and size of the manually defined physical tracking space, subsequently impacting the user’s immersive experience. To improve performance, one strategy is to integrate exploration methods from mobile robotics with RDW. This will expand the usable tracking space, facilitating dynamic environments and rapid exploration. For this, we adapted a Unity framework for an RDW algorithm to facilitate simulations for such an exploration. We conducted a simulation with artificially created non-convex explorable tracking spaces and pre-recorded path elements, simulating two adapted RDW artificial potential field (APF) concepts. Three conceptualized modes were applied: repulsive APF, exploration APF, and exploration APF with a distance threshold. Additionally, one APF was extended with a frontier-based exploration approach that utilized the path between the user’s position and a targeted frontier. The analysis revealed a significant trade-off between exploration and immersion. APF combined with frontier-based the exploration technique showed the fastest exploration speed, but - however - resulted in the lowest distance between resets.

A Segmented Redirection Mapping Method for Roadmaps of Large Constrained Virtual Environments.

Redirected walking (RDW) enables users to explore large virtual spaces by real walking in small real spaces. How to effectively reduce physical collisions and decrease user perceptions of redirection are important for most RDW methods. This paper proposes a segmented redirection mapping method to calculate and map the roadmap of a large virtual space with inner obstacles to a mapped roadmap within a small real space. We adopt a Voronoi-based pruning method to extract the roadmap of the virtual space and design an RDW platform to interactively modify the virtual roadmap. We propose a roadmap mapping method based on divide-and-conquer and dynamic planning strategies to subdivide the virtual roadmap into several sub-virtual roads that are mapped individually. By recording connections of different sub-virtual roads, our method is applicable to virtual roadmaps with loop structures. During mapping, we apply the reset and redirection gains of the RDW technique as optimal aims and restrict conditions to obtain the mapped roadmap, which has small path curving and contains as few resets as possible. By real walking along the mapped roadmap, users perceive moving along the virtual roadmap to explore the entire virtual space. The experiment shows that our method works effectively for various virtual spaces with or without inner obstacles. Furthermore, our method is flexible in obtaining mapped roadmaps of different real spaces when the virtual space is fixed. Compared to prevalent RDW methods, our method can significantly reduce physical boundary collisions and maintain user experience of virtual roaming.

Encountering an emotional landmark on the route for a better spatial memory: What matters, valence or arousal?

This study explores the impact of the interplay between the valence and arousal dimensions of landmarks on spatial memory. Participants were asked to watch a movie of a walk in a large urban virtual environment with landmarks that differed in terms of arousal and valence across conditions (high-arousal-positive; low-arousal-positive; high-arousal-negative; low-arousal-negative). Their route knowledge was assessed through a direction recall task, and their survey knowledge by asking them to plot the position of each landmark encountered during the route on a map. Results showed that the presence of positive landmarks benefited route knowledge, whereas high arousing ones impaired it. However, no clear effect of the landmarks’ valence and arousal was observed on survey knowledge. In addition to supporting some of the previous evidence while extending it to more complex and real-like environments, these new results call into question the relevance of the dimensional approach to emotions classically adopted in this type of studies.

ManiLoco: A VR-Based Locomotion Method for Concurrent Object Manipulation.

The use of virtual reality (VR) in laboratory skill training is rapidly increasing. In such applications, users often need to explore a large virtual environment within a limited physical space while completing a series of hand-based tasks (e.g., object manipulation). However, the most widely used controller-based teleport methods may conflict with the users' hand operation and result in a higher cognitive load, negatively affecting their training experiences. To alleviate these limitations, we designed and implemented a locomotion method called ManiLoco to enable hands-free interaction and thus avoid conflicts and interruptions from other tasks. Users can teleport to a remote object's position by taking a step toward the object while looking at it. We evaluated ManiLoco and compared it with state-of-the-art Point & Teleport in a within-subject experiment with 16 participants. The results confirmed the viability of our foot- and head-based approach and better support concurrent object manipulation in VR training tasks. Furthermore, our locomotion method does not require any additional hardware. It solely relies on the VR head-mounted display (HMD) and our implementation of detecting the user's stepping activity, and it can be easily applied to any VR application as a plugin.

An empirical evaluation of enhanced teleportation for navigating large urban immersive virtual environments

Navigation is the most prevalent interaction in large urban virtual environments (VEs). Any Metaverse application that foresees navigating or exploring virtual cities requires an effective and efficient navigation technique. These environments, however, have distinct characteristics that make the navigation more challenging and the design of the interactions more critical. We have conducted an empirical study to assess how enhancing the teleportation technique with additional capabilities affects the performance of navigating large urban VEs. For this purpose, three interactions have been developed that extend the conventional point-and-click teleportation. The first one is named Mini-Map and provides a top-down view of the user’s surroundings. Portal Preview provides a preview of the selected target location and allows users to choose their desired orientation at that location. The last technique, called X-Ray Vision, makes the buildings around the user translucent and allows teleporting to locations that would otherwise be obscured. A within-subject controlled lab study with twenty five participants has been conducted, where each extension is evaluated individually as well as in combination with others. Our results show that extending the teleportation can significantly improve its performance when navigating large urban VEs. Overall, the X-Ray Vision was the most successful extension with respect to both task metrics and usability measures. Mini-Map was able to improve some of the task metrics, but did not have a significant effect on most self-reported measures. Portal Preview was the least effective extension, however, multiple participants liked the fact that they could define their desired orientation with the controller. Combining all interactions together performed well with respect to the task metrics, but this option was not favored by the participants. Extending the teleportation with X-Ray Vision and Mini-Map was by far the most favored option.

UrbanRama: Navigating Cities in Virtual Reality.

Exploring large virtual environments, such as cities, is a central task in several domains, such as gaming and urban planning. VR systems can greatly help this task by providing an immersive experience; however, a common issue with viewing and navigating a city in the traditional sense is that users can either obtain a local or a global view, but not both at the same time, requiring them to continuously switch between perspectives, losing context and distracting them from their analysis. In this article, our goal is to allow users to navigate to points of interest without changing perspectives. To accomplish this, we design an intuitive navigation interface that takes advantage of the strong sense of spatial presence provided by VR. We supplement this interface with a perspective that warps the environment, called UrbanRama, based on a cylindrical projection, providing a mix of local and global views. The design of this interface was performed as an iterative process in collaboration with architects and urban planners. We conducted a qualitative and a quantitative pilot user study to evaluate UrbanRama and the results indicate the effectiveness of our system in reducing perspective changes, while ensuring that the warping doesn't affect distance and orientation perception.

Making Resets away from Targets: POI aware Redirected Walking.

Rapidly developing Redirected Walking (ROW) technologies have enabled VR applications to immerse users in large virtual environments (VE) while actually walking in relatively small physical environments (PE). When an unavoidable collision emerges in a PE, the ROW controller suspends the user's immersive experience and resets the user to a new direction in PE. Existing ROW methods mainly aim to reduce the number of resets. However, from the perspective of the user experience, when users are about to reach a point of interest (POI) in a VE, reset interruptions are more likely to have an impact on user experience. In this paper, we propose a new ROW method, aiming to keep resets occurring at a longer distance from the virtual target, as well as to reduce the number of resets. Simulation experiments and real user studies demonstrate that our method outperforms state-of-the-art ROW methods in the number of resets and dramatically increases the distance between the reset locations and the virtual targets.

Modelling group dynamics for crowd simulations

This paper investigates a new method to simulate pedestrian crowd movement in a large and complex virtual environment, representing a public space such as a shopping mall. To demonstrate pedestrian dynamics, we consider groups of pedestrians of different size, sharing a crowded environment. A pedestrian has its own characteristics, such as gender, age, position, velocity, and energy. The proposed method uses a multi-group microscopic model to generate real-time trajectories for all people moving in the defined virtual environment. Additionally, a dynamic model is introduced for modelling group behaviour. Based on the proposed method, all pedestrians in each group can continuously adjust their attributes and optimize their path towards the desired visiting targets, while avoiding obstacles and other pedestrians. Simulation results show that the proposed method can describe a realistic simulation of dynamic behaviour.

Adaptive Redirection: A Context-Aware Redirected Walking Meta-Strategy.

Previous research has established redirected walking as a potential answer to exploring large virtual environments via natural locomotion within a limited physical space. However, much of the previous work has either focused on investigating human perception of redirected walking illusions or developing novel redirection techniques. In this paper, we take a broader look at the problem and formalize the concept of a complete redirected walking system. This work establishes the theoretical foundations for combining multiple redirection strategies into a unified framework known as adaptive redirection. This meta-strategy adapts based on the context, switching between a suite of strategies with a priori knowledge of their performance under the various circumstances. This paper also introduces a novel static planning strategy that optimizes gain parameters for a predetermined virtual path, known as the Combinatorially Optimized Pre-Planned Exploration Redirector (COPPER). We conducted a simulation-based experiment that demonstrates how adaptation rules can be determined empirically using machine learning, which involves partitioning the spectrum of contexts into regions according to the redirection strategy that performs best. Adaptive redirection provides a foundation for making redirected walking work in practice and can be extended to improve performance in the future as new techniques are integrated into the framework.

Impact Sound Generation for Audiovisual Interaction with Real-World Movable Objects in Building-Scale Virtual Reality

In building-scale VR, where the entire interior of a large-scale building is a virtual space that users can walk around in, it is very important to handle movable objects that actually exist in the real world and not in the virtual space. We propose a mechanism to dynamically detect such objects (that are not embedded in the virtual space) in advance, and then generate a sound when one is hit with a virtual stick. Moreover, in a large indoor virtual environment, there may be multiple users at the same time, and their presence may be perceived by hearing, as well as by sight, e.g., by hearing sounds such as footsteps. We, therefore, use a GAN deep learning generation system to generate the impact sound from any object. First, in order to visually display a real-world object in virtual space, its 3D data is generated using an RGB-D camera and saved, along with its position information. At the same time, we take the image of the object and break it down into parts, estimate its material, generate the sound, and associate the sound with that part. When a VR user hits the object virtually (e.g., hits it with a virtual stick), a sound is generated. We demonstrate that users can judge the material from the sound, thus confirming the effectiveness of the proposed method.

ARC: Alignment-based Redirection Controller for Redirected Walking in Complex Environments.

We present a novel redirected walking controller based on alignment that allows the user to explore large and complex virtual environments, while minimizing the number of collisions with obstacles in the physical environment. Our alignment-based redirection controller, ARC, steers the user such that their proximity to obstacles in the physical environment matches the proximity to obstacles in the virtual environment as closely as possible. To quantify a controller's performance in complex environments, we introduce a new metric, Complexity Ratio (CR), to measure the relative environment complexity and characterize the difference in navigational complexity between the physical and virtual environments. Through extensive simulation-based experiments, we show that ARC significantly outperforms current state-of-the-art controllers in its ability to steer the user on a collision-free path. We also show through quantitative and qualitative measures of performance that our controller is robust in complex environments with many obstacles. Our method is applicable to arbitrary environments and operates without any user input or parameter tweaking, aside from the layout of the environments. We have implemented our algorithm on the Oculus Quest head-mounted display and evaluated its performance in environments with varying complexity. Our project website is available at https://ganuna.umd.edu/arc/.

The Statistical Structure of the Hippocampal Code for Space as a Function of Time, Context, and Value

Hippocampal activity represents many behaviorally important variables, including context, an animal's location within a given environmental context, time, and reward. Using longitudinal calcium imaging in mice, multiple large virtual environments, and differing reward contingencies, we derived a unified probabilistic model of CA1 representations centered on a single feature-the field propensity. Each cell's propensity governs how many place fields it has per unit space, predicts its reward-related activity, and is preserved across distinct environments and over months. Propensity is broadly distributed-with many low, and some very high, propensity cells-and thus strongly shapes hippocampal representations. This results in a range of spatial codes, from sparse to dense. Propensity varied ∼10-fold between adjacent cells in salt-and-pepper fashion, indicating substantial functional differences within a presumed cell type. Intracellular recordings linked propensity to cell excitability. The stability of each cell's propensity across conditions suggests this fundamental property has anatomical, transcriptional, and/or developmental origins.

A flexible technique to select objects via convolutional neural network in VR space

Most studies on the selection techniques of projection-based VR systems are dependent on users wearing complex or expensive input devices, however there are lack of more convenient selection techniques. In this paper, we propose a flexible 3D selection technique in a large display projection-based virtual environment. Herein, we present a body tracking method using convolutional neural network (CNN) to estimate 3D skeletons of multi-users, and propose a region-based selection method to effectively select virtual objects using only the tracked fingertips of multi-users. Additionally, a multi-user merge method is introduced to enable users actions and perception to realign when multiple users observe a single stereoscopic display. By comparing with state-of-the-art CNN-based pose estimation methods, the proposed CNN-based body tracking method enables considerable estimation accuracy with the guarantee of real-time performance. In addition, we evaluate our selection technique against three prevalent selection techniques and test the performance of our selection technique in a multi-user scenario. The results show that our selection technique significantly increases the efficiency and effectiveness, and is of comparable stability to support multi-user interaction.

Multi-Projector Calibration Based on Virtual Viewing Space

In large-scale maritime maneuvering simulators, the multi-channel systems are usually used to tile multiple projectors on a cylindrical display to create a large virtual environment, which brings more realistic immersion to the users. In the non-planar multi-projector display system, to finish the work of geometric and photometric corrections, there exist the problems of difficult projector calibration, more manual intervention and poor adaptability. In order to solve these problems, a multi-projector calibration based on virtual viewing space is proposed in this paper. First, a virtual environment of multiple projectors is created, and by using the structured light and re-projection model on a cylindrical wall, the calibration of internal and external parameters of a projector are realized separately. Finally, the geometric correction is completed by the mapping relationship between the target images and the calibrated re-projected images. This method is free of limitation by the angle of view of a single camera and can calibrate any of the projectors in the multi-channel systems. The example results show that the projector’s parameters can be estimated more accurately, and the efficiency and accuracy of geometric correction can be improved. Good results are achieved when applying this method in the maritime maneuvering simulator system.

Redirected Smooth Mappings for Multiuser Real Walking in Virtual Reality

We propose a novel technique to provide multiuser real walking experiences with physical interactions in virtual reality (VR) applications. In our system, multiple users walk freely while navigating a large virtual environment within a smaller physical workspace. These users can interact with other real users or physical props in the same physical locations. The key of our method is a redirected smooth mapping that incorporates the redirected walking technique to warp the input virtual scene with small bends and low distance distortion. Users possess a wide field of view to explore the mapped virtual environment while being redirected in the real workspace. To keep multiple users away from the overlaps of the mapped virtual scenes, we present an automatic collision avoidance technique based on dynamic virtual avatars. These avatars naturally appear, move, and disappear, producing as little influence as possible on users’ walking experiences. We evaluate our multiuser real walking system through formative user studies, and demonstrate the capability and practicability of our technique in two multiuser applications.

Pathfinding in hierarchical representation of large realistic virtual terrains for simulation systems

A prominent pathfinding challenge is the fast computation of path plans in large virtual terrain environments used in simulation systems. Based on a hierarchical approach for pathfinding and terrain representation, this work details the design, the implementation and the test of a quadtree-based structure in the irregular grid navigation map representation of large realistic virtual terrain environments. To achieve this goal, it is described how a hierarchical global A* pathfinding algorithm searches for a path in a coarse initial irregular grid structure then proceeding with the search in refined regions of interest where obstacles are found. Thus, the proposed solution presents as major contribution the enhanced quadtree-based map representation for pathfinding in large virtual terrains. The pathfinding response time of this solution is statistically compared with different hierarchical/non-hierarchical and regular/irregular terrain representation structures instantiated in the modeling of “small” realistic terrain scenarios. After these experiments, similar ones are developed in a massive (large) virtual terrain inserted into a real-life simulation system for the development of military tactical training exercises. The results show that the response time required to generate pathfinding results can be reduced when the proposed global pathfinding technique over the quadtree-based hierarchical and irregular navigation map representation of the large virtual terrain is explored in the development of simulation systems.

Optimized wavelet-based texture representation and streaming for GPU texture mapping

Because of the ever increasing resolution of consumer displays, high quality real-time 3D rendering applications require large amounts of 2D texture data, which in turn require large amounts of storage, memory and bandwidth. One of the major advantages of using compressed texture data is the significant decrease in bandwidth requirements, decreasing texture loading times. Additionally, instead of preloading all the potentially required information, texture data can be streamed progressively at run-time, which is a very common scenario in web-based applications, games and large virtual environments. This paper proposes a new texture streaming system, utilizing both pre-computed and run-time scene analysis, camera prediction and wavelet-based texture compression to provide maximal visual quality within bandwidth and real-time constraints. Texture decoding is done on the GPU, saving both on streaming bandwidth and GPU memory. The pre-computed scene analysis system can perform the run-time analysis at virtually no performance cost in multi-threaded systems. The proposed solution can easily be plugged into existing classical texture mapping solutions, as it features drop-in replacement shaders and re-uses existing render facilities. The wavelet-based streaming system results in PSNR improvements of up to 2 dB when compared to a DXT1-based streaming system.

Bending the Curve: Sensitivity to Bending of Curved Paths and Application in Room-Scale VR.

Redirected walking (RDW) promises to allow near-natural walking in an infinitely large virtual environment (VE) by subtle manipulations of the virtual camera. Previous experiments analyzed the human sensitivity to RDW manipulations by focusing on the worst-case scenario, in which users walk perfectly straight ahead in the VE, whereas they are redirected on a circular path in the real world. The results showed that a physical radius of at least 22 meters is required for undetectable RDW. However, users do not always walk exactly straight in a VE. So far, it has not been investigated how much a physical path can be bent in situations in which users walk a virtual curved path instead of a straight one. Such curved walking paths can be often observed, for example, when users walk on virtual trails, through bent corridors, or when circling around obstacles. In such situations the question is not, whether or not the physical path can be bent, but how much the bending of the physical path may vary from the bending of the virtual path. In this article, we analyze this question and present redirection by means of bending gains that describe the discrepancy between the bending of curved paths in the real and virtual environment. Furthermore, we report the psychophysical experiments in which we analyzed the human sensitivity to these gains. The results reveal encouragingly wider detection thresholds than for straightforward walking. Based on our findings, we discuss the potential of curved walking and present a first approach to leverage bent paths in a way that can provide undetectable RDW manipulations even in room-scale VR.