Immersive Interface Research Articles

Redefining Spectral Data Analysis with Immersive Analytics: Exploring Domain-Shifted Model Spaces for Optimal Model Selection.

Modern developments in autonomous chemometric machine learning technology strive to relinquish the need for human intervention. However, such algorithms developed and used in chemometric multivariate calibration and classification applications exclude crucial expert insight when difficult and safety-critical analysis situations arise, e.g., spectral-based medical decisions such as noninvasively determining if a biopsy is cancerous. The prediction accuracy and interpolation capabilities of autonomous methods for new samples depend on the quality and scope of their training (calibration) data. Specifically, analysis patterns within target data not captured by the training data will produce undesirable outcomes. Alternatively, using an immersive analytic approach allows insertion of human expert judgment at key machine learning algorithm junctures forming a sensemaking process performed in cooperation with a computer. The capacity of immersive virtual reality (IVR) environments to render human comprehensible three-dimensional space simulating real-world encounters, suggests its suitability as a hybrid immersive human-computer interface for data analysis tasks. Using IVR maximizes human senses to capitalize on our instinctual perception of the physical environment, thereby leveraging our innate ability to recognize patterns and visualize thresholds crucial to reducing erroneous outcomes. In this first use of IVR as an immersive analytic tool for spectral data, we examine an integrated IVR real-time model selection algorithm for a recent model updating method that adapts a model from the original calibration domain to predict samples from shifted target domains. Using near-infrared data, analyte prediction errors from IVR-selected models are reduced compared to errors using an established autonomous model selection approach. Results demonstrate the viability of IVR as a human data analysis interface for spectral data analysis including classification problems.

Virtual Simulation and Experiment of Quality Inspection Robot Workstation

(1) Background: Quality inspection robots are widely used in automated production lines. However, the design cycle is long, iteration costs are high, and algorithm development is challenging. It is difficult to perform effective validation during the design phase. Applying virtual reality technology to simulate quality inspection robot workstations offers a new approach to addressing the issues. (2) Methods: The research creates a simulation platform for quality inspection robot workstations based on a virtual reality architecture. The platform creates an immersive quality inspection robot workstation operation interface and conducts testing of the inspection process, thereby validating the rationality of the quality inspection robot workstation design. Building upon this foundation, we conducted experimental comparisons of various defect detection algorithms. (3) Results: Compared to the traditional YOLOv7 algorithm, the improved YOLOv7 algorithm achieved an 18.1% increase in recognition precision. Experimental results demonstrate that the quality inspection robot workstation simulation platform can be applied to validating workstation design proposals. (4) Conclusions: It has a positive impact on reducing the research and development costs of quality inspection robot workstations and shortening the defect recognition algorithm development cycle.

Examining presence in immersive virtual reality and website interfaces through the cognitive fit and cognitive load theories

PurposeThe purpose of this paper is to assess the effects of presence on users’ information processing in mediated visualizations using immersive virtual reality (VR) and websites, focusing on information understanding and cognitive load. The impact of user interfaces on users’ perception of presence is also tested.Design/methodology/approachA between-subjects laboratory experiment using two user interfaces – immersive VR and website – is carried out. It is complemented by a self-administered survey on the participants.FindingsThe current results indicate that an immersive VR interface improves tourists’ information processing by producing a heightened sense of presence. The results confirm that presence facilitates tourists’ information processing by increasing information understanding and decreasing cognitive load. Finally, a negative relationship between cognitive load and information understanding is obtained.Practical implicationsVR user interfaces are not only visually appealing but also cognitively optimized. Immersive VR devices can be introduced in the specific sale process of cruise trips through both offline channels (physical stores and travel agencies) and online channels to facilitate tourists’ information processing during the prepurchase phase.Originality/valueThe primary contribution lies in the integration of the theory of presence with two influential information processing frameworks, namely, cognitive load theory and cognitive fit theory. This integration provides a holistic perspective on how user interfaces influence consumer information processing in the unique context of cruise tourism, particularly during the stage of a prepurchase information search.

Online Gambling Addiction: A Study among College Students of Kerala State, India

Online Gambling may increase rates of disordered gambling due to its high degree of accessibility, immersive interface and simply spending money for these activities. The problem of college students developing an addiction to internet gambling has received increased attention recently in India. The availability of online gambling sites and the pervasive use of mobile devices puts a lot of college students at risk of developing a gambling addiction. Teenagers engage in both regulated and non-regulated forms of gambling. Different forms of non-regulated gambling activities include skill-based games, dice, sports betting, and card games amongst peers. Online gambling has opened the doors for entrepreneurs to be involved in fraud, theft, extortion, and money laundering in the gambling sites. This study through a questionnaire survey investigates the exponential growth of online gambling. The objective of the study is to bring out the demographic variables’ influence on online gambling addiction among college Students.

Maintenance study on WCLL-TBS ancillary systems in process room

The Water-Cooled Lithium Lead Test Blanket System (WCLL-TBS) is one of the European TBS candidates for being installed and operated in ITER. The ongoing activities toward the Preliminary Design Review of the WCLL-TBS Ancillary Systems include the reallocation of their units in the reserved space, also in the view of the proper execution of the maintenance activities. Starting from the available documentation (System Design Description, Process Flow Diagrams, Process and Instrumentation Diagrams), detailed 3D CAD models have been developed for two Ancillary Systems to be installed in the Process room: the Tritium Extraction System (TES) and the Tritium Accountancy System (TAS). In the Process room (PR), the largest part of the TES process units is placed inside a Glove-box, as enclosure to avoid the contamination of the room in case of radioactive release due to failure of units. Based on the expected radioactivity content, all the TAS units are placed inside the Glove-box. Maintainability studies have been performed, focusing on the preventive (i.e. planned) and corrective (i.e. required by failures) maintenance tasks on the units installed inside the Glove-box. They were based on the 3D CAD models and supported using Virtual Reality as intuitive and immersive human-computer interface. They provided insights for the iterative improvement and consolidation of the Systems design and information for the evaluation of their availability.

Pragmatic and hedonic aspects of user experience in Virtual Reality: analysis of novice users’ information mediation during their first interaction with a Metaverse platform

The topic of virtual and extended reality has increasingly captured the interest of developers who aim to widely adopt it among the general population. Over the last five years, significant investments in this field have brought the term “Metaverse” back into discussions as a potential ideal for social interaction within virtual environments. This paper aims to contextualize the Metaverse and its potentialities by investigating its relation to new users from a UX (user experience) design perspective. This work aims to understand the pragmatic and hedonic aspects of user experience and information mediation of novice users during their first interaction with a Metaverse application. To achieve this, we selected six users and analyzed their first interactions within the popular digital platform called “VRchat”. For data analysis, we used a method inspired by Marc Hassenzahl’s AttrakDiff tool and investigated the interactions based on three main aspects: interactivity, immersion, and self-identification. As a result, we noticed a positive impression of virtual reality (VR) on user experience after immersion. We also observed some UI (user interface) solutions that are functional, and others that are not, in an immersive interface. Based on these findings, we propose some hypotheses for solution paths to enhance these three aspects of the Metaverse experience.

Multi-user immersive environment for excavator teleoperation in construction

Excavation on jobsites is a collaborative effort, with a spotter serving as an extra set of eyes for the operator to ensure safety. However, current training methods using simulators are limited in immersion and primarily designed for individual operators to practice basic skills. This paper presents a multi-user teleoperation system featured with hybrid-immersive interface and between-user communication. The effectiveness of the system is evaluated through human subject experiments. The study's findings indicate that operators experienced greater immersion, a stronger sense of presence, and improved user interaction when using an immersive interface. This paper has important implications for the future of excavation training, with potential benefits including reduced utility strike damage and the opportunity to investigate human-robot collaboration in jobsites with multiple workers performing various roles within a highly immersive operational environment.

Augmented reality-enabled human-robot collaboration to balance construction waste sorting efficiency and occupational safety and health

Construction waste sorting (CWS) is highly recommended as a key step for construction waste management. However, current CWS involves humans’ manual hand-picking, which poses significant threats to their occupational safety and health (OSH). Robotic sorting promises to change the situation by adopting modern artificial intelligence and automation technologies. However, in practice, it is usually challenging for robots to do an efficient job (e.g., measured by quickness and accuracy) owing to the difficulties in precisely recognizing compositions of the mixed and heterogeneous waste stream. Leveraging augmented reality (AR) as a communication interface, this research aims to develop a human-robot collaboration (HRC) approach to address the dilemmatic balance between CWS efficiency and OSH. Firstly, a model for human-robot collaborative sorting using AR is established. Then, a prototype for the AR-enable collaborative sorting system is developed and evaluated. The experimental results demonstrate that the proposed AR-enabled HRC method can improve the accuracy rate of CWS by 10% and 15% for sorting isolated waste and obscured waste, respectively, when compared to the method without human involvement. Interview results indicate a significant improvement in OSH, especially the reduction of contamination risks and machinery risks. The research lays out a human-robot collaborative paradigm for productive and safe CWS via an immersive and interactive interface like AR.

I want to travel to the past! The role of creative style and historical reconstructions as antecedents of informativeness in a virtual visit to a heritage tourist destination

ABSTRACT Virtual visits are a particularly useful way to learn about heritage tourism destinations. This article examines three design variables (i.e. technology interface: web browser vs. virtual reality device; creative style of information: textual vs. pictorial; and presence of historical reconstructions: yes vs. no) of a virtual visit to identify their influence on perceived informativeness and intention to recommend a destination. We performed a 2 × 2 × 2 between-subjects experimental factorial design with 116 Spanish tourists. We found that the presence of historical reconstructions especially improves informativeness when combined with extra textual information; and that if the visit is mediated by an immersive VR device, textual information cues influence intention to recommend more than do pictorial cues, but when the virtual visit is carried out using a less immersive technological interface, pictorial content has more influence on intention to recommend. Theoretical and managerial contributions are discussed.

Meditating in a neurofeedback virtual reality: effects on sense of presence, meditation depth and brain oscillations

ABSTRACT With the advent of consumer-grade electroencephalography (EEG) and virtual reality (VR) devices, the use of human cognitive processes directly as means for user-adapted interaction in immersive virtual environments has become increasingly relevant. In this study (N = 43), we investigate electroencephalography-based neurofeedback interaction in virtual reality (VR). Particularly, we investigate this phenomenon in the context of meditation which enables the studying of cognitive processes (attention, sense of presence, meditation depth) when using an immersive interface that is adaptive based on neural responses. A prototype virtual reality environment was built that employs head-mounted display (HMD), and the neurofeedback functionality guided the user to increase the neural indices of meditation-related concentration and relaxation. The observed findings provide evidence for the effectiveness of neurofeedback functionality and VR in evoking desired types of neural activation and subjective experiences.

Participant Perceptions of Augmented Reality Simulation for Cardiac Anesthesiology Training: A Prospective, Mixed-Methods Study.

Simulations are a critical component of anesthesia education, and ways to broaden their delivery and accessibility should be studied. The primary aim was to characterize anesthesiology resident, fellow, and faculty experience with augmented reality (AR) simulations. The secondary aim was to explore the feasibility of quantifying performance using integrated eye-tracking technology. This was a prospective, mixed-methods study using qualitative thematic analysis of user feedback and quantitative analysis of gaze patterns. The study was conducted at a large academic medical center in Northern California. Participants included 7 anesthesiology residents, 6 cardiac anesthesiology fellows, and 5 cardiac anesthesiology attendings. Each subject participated in an AR simulation involving resuscitation of a patient with pericardial tamponade. Postsimulation interviews elicited user feedback, and eye-tracking data were analyzed for gaze duration and latency. Thematic analysis revealed 5 domains of user experience: global assessment, spectrum of immersion, comparative assessment, operational potential, and human-technology interface. Participants reported a positive learning experience and cited AR technology's portability, flexibility, and cost-efficiency as qualities that may expand access to simulation training. Exploratory analyses of gaze patterns suggested that trainees had increased gaze duration of vital signs and gaze latency of malignant arrythmias compared with attendings. Limitations of the study include lack of a control group and underpowered statistical analyses of gaze data. This study suggests positive user perception of AR as a novel modality for medical simulation training. AR technology may increase exposure to simulation education and offer eye-tracking analyses of learner performance.

Mixed-reality for quadruped-robotic guidance in SAR tasks

Abstract In recent years, exploration tasks in disaster environments, victim localization and primary assistance have been the main focuses of Search and Rescue (SAR) Robotics. Developing new technologies in Mixed Reality (M-R) and legged robotics has taken a big step in developing robust field applications in the Robotics field. This article presents MR-RAS (Mixed-Reality for Robotic Assistance), which aims to assist rescuers and protect their integrity when exploring post-disaster areas (against collapse, electrical, and toxic risks) by facilitating the robot’s gesture guidance and allowing them to manage interest visual information of the environment. Thus, ARTU-R (A1 Rescue Tasks UPM Robot) quadruped robot has been equipped with a sensory system (lidar, thermal, and RGB-D cameras) to validate this proof of concept. On the other hand, Human-Robot interaction is executed by using the Hololens glasses. This work’s main contribution is the implementation and evaluation of a Mixed-Reality system based on a ROS-Unity solution, capable of managing at a high level the guidance of a complex legged robot through different interest zones (defined by a Neural Network and a vision system) of a post-disaster environment (PDE). The robot’s main tasks at each point visited involve detecting victims through thermal, RGB imaging, and neural networks and assisting victims with medical equipment. Tests have been carried out in scenarios that recreate the conditions of PDE (debris, simulation of victims, etc.). An average efficiency improvement of 48% has been obtained when using the immersive interface and a time optimization of 21.4% compared to conventional interfaces. The proposed method has proven to improve rescuers’ immersive experience of controlling a complex robotic system.

VR OR NOT? INVESTIGATING INTERFACE TYPE AND USER STRATEGIES FOR INTERACTIVE DESIGN SPACE EXPLORATION

AbstractComputational design tools allow the generation of vast numbers of possible designs, entrusting the human designer with describing constraints or specifications to guide exploration of the design space. Designers can have many different decision considerations when conducting this type of exploration, including form, function, users, or context. In this work, we investigate strategies that emerge when people are tasked with exploring a large design space within either a non-immersive (2D) or immersive (VR) interface and equipped with action-based interactions to set or envision specifications related to their considerations. Results from a 28 participant user study uncovers that people have varying strategies to enact their decision considerations that are not unique to the type of interface. However, the interfaces differ in perceptions of enabling breadth or depth of exploration holistically, with preference towards 2D interfaces to compare options, and VR to understand single designs. These results have implications for the user experience of systems that allow designers to explore the outputs of large design spaces, both at the interaction and interface levels.

A Sketch-Based Interface for Facial Animation in Immersive Virtual Reality

Creating facial animations using 3D computer graphics represents a very laborious and time-consuming task. Among the numberless approaches for animating faces, the use of blendshapes remains the most common solution because of their simplicity and the ability to produce high-quality results. This approach, however, is also characterized by important drawbacks. With the traditional animation suites, to select the blendshapes to be activated animators are generally requested to memorize the mapping between the blendshapes and the influenced mesh vertices; alternatively, they need to adopt a trial-and-error search within the whole library of available blendshapes. Moreover, the level of expressiveness that can be reached may be lower than expected; this is due to the fact that the possibility to apply transformations to vertices different than just linear translations and mechanisms for adding, e.g., exaggerations, are typically not integrated into the same animation environment. To tackle these issues, this article proposes an immersive virtual reality-based interface that leverages sketches for the direct manipulation of blendshapes. Animators can draw both linear and curved strokes, which are used to automatically extract information about the blendshape to be activated and its weight, the trajectories that associated vertices have to follow, as well as the timing of the overall animation. A user study was carried out with the aim of evaluating the proposed approach on several representative animations tasks. Both objective and subjective measurements were collected. Experimental results showed the benefits of the devised interface in terms of task completion time, animation accuracy, and usability.

The Presentation of Self in Virtual Reality: A Cognitive Load Study

The use of virtual reality (VR) in interior design has increased dramatically. Its interactive and visualization benefits are undeniable. Designers, clients, developers, and stakeholders can immerse themselves in future or existing design projects without the need to be physically there. Thanks to more immersive and realistic experiences, the boundaries that separate the physical and the virtual world are becoming nonexistent. Nonetheless, research has focused on the visual characteristics of the virtual space, undermining the consequences for individuals engaged within it. In this study, we assessed the effects on mental workload caused by how individuals visualize themselves in VR using a virtual body (VB). The VB is typically represented by the use of avatars. An experimental setup was carried out with a convenience sample of 72 individuals. Participants interacted in an immersive VR interface with three different conditions of the VB. They were randomly assigned to one of the three conditions and engaged for a period of approximately 20 minutes in tackling a design-like activity. Data were collected through self-report questionnaires in addition to a psychophysiological device accounting for the cognitive load (CL) and task difficulty. The statistical analysis supported differences in CL between conditions. A more detailed visual representation of the VB increased the sensation of being there but contributed extraneous CL that can hinder the task at hand. The findings of this study can guide interior designers in selecting the type of VB they should use for their immersive VR experiences.

Labeling Out-of-View Objects in Immersive Analytics to Support Situated Visual Searching.

Augmented Reality (AR) embeds digital information into objects of the physical world. Data can be shown in-situ, thereby enabling real-time visual comparisons and object search in real-life user tasks, such as comparing products and looking up scores in a sports game. While there have been studies on designing AR interfaces for situated information retrieval, there has only been limited research on AR object labeling for visual search tasks in the spatial environment. In this article, we identify and categorize different design aspects in AR label design and report on a formal user study on labels for out-of-view objects to support visual search tasks in AR. We design three visualization techniques for out-of-view object labeling in AR, which respectively encode the relative physical position (height-encoded), the rotational direction (angle-encoded), and the label values (value-encoded) of the objects. We further implement two traditional in-view object labeling techniques, where labels are placed either next to the respective objects (situated) or at the edge of the AR FoV (boundary). We evaluate these five different label conditions in three visual search tasks for static objects. Our study shows that out-of-view object labels are beneficial when searching for objects outside the FoV, spatial orientation, and when comparing multiple spatially sparse objects. Angle-encoded labels with directional cues of the surrounding objects have the overall best performance with the highest user satisfaction. We discuss the implications of our findings for future immersive AR interface design.

Evaluation of Response Time of AIRR with Immersive Aerial Interface by 3D Motion Capture

Evaluation of Response Time of AIRR with Immersive Aerial Interface by 3D Motion Capture

A Cybersecurity Model for a Roblox-based Metaverse Architecture Framework

The adoption of virtual reality (VR) and augmented reality (AR) headsets in futuristic and science fiction has made it possible for the Metaverse to exist as a single, universal, immersive virtual universe. By extending technology outside of our physical reality, the Metaverse alters the human experience. The four categories we use to categorize metaverse definitions are environment, interface, interaction, and social value. Currently, it is unclear what the metaverse's structure and elements are. A cybersecurity framework for these devices is necessary as the world grows more interconnected and immersive technologies are increasingly widely used in business, government, and consumer markets. Used was a literature review. The goal of the study is to create a cybersecurity model for a Roblox-based Metaverse architecture framework that can be applied to internationalization, the value chain of education, and the delivery of online and e-learning education. The research is conducted using the Interpretivist Paradigm, which is characterized by a subjectivist epistemology, a relativist ontology, a naturalist methodology, and a balanced axiology. Both the qualitative methodology and the quantitative methodology with an experimental research design were applied. A systematic literature review was conducted on the metaverse, AR and VR. By describing each aspect of the metaverse, we categorized definitions of the metaverse into four categories: environment, interface, interaction, and social value. The interface is based on the level of immersion and there are physical, immersive, and 3D interface options available. There are realistic, unreal, and blended environments in the metaverse. Social networking, teamwork, and persona discourse are the three categories used to describe interaction in the metaverse. A major advantage of 6G is that it facilitates instant communication in phones, computers, wearable devices, robotics, and more. The Cybersecurity model for a Roblox-based Metaverse architecture framework developed is a Bayesian Network, which is a directed acyclic graph that has an associated probability distribution function that can be used for multivariate analysis.

Empirical Research on the Metaverse User Experience of Digital Natives

The metaverse has been settled as a platform that is widely beloved by digital natives that are familiar with mobile devices and immersive contents. Thanks to the protocol enabling hedonic interaction, the user experience provides significant value from its communication, enabling learning experiences anytime and anywhere. However, the research topics are focused on the promotions of technology development, marketing effects, and relevant investment consensus. Surprisingly, the biggest problem was the lack of research from the perspective of the young generation, who mainly use the metaverse. This paper intends to examine the usability of digital native participants in detail and suggest how immersive contents, usage environment, and interface aspects should be designed from their point of view. As a result, the significant engagement factors and improvements, through heuristic usability evaluation considering content and user control, were discovered from individual interviews. Conversely, the elements to be supplemented in user experience were derived from information architecture and usage environment categories. In conclusion, the theoretical basis of the empirical usability evaluation on metaverse platforms and following recommendations with practical implications could gain more importance from this research.

A lower limb rehabilitation platform with mirror therapy, electrical stimulation and virtual reality for people with limited dorsiflexion movement

This paper presents a low-cost portable system for applying a rehabilitation therapy, based on virtual reality, functional electrical stimulation and a training device. This open-hardware system, intended to support a home-made therapy process in people with reduced dorsiflexion ability, contains three main building blocks: an electromechanical platform, an immersive virtual reality interface and the firmware. The electromechanical training platform is the core of the system, which acts as a gateway interface of sensors-actuators for a closed loop system, through an immersive virtual reality application, the patient closes the loop by acting as the controller by means of a “virtual dummy” preforming in mirror configuration.The platform measures and process the dorsiflexion angle described by both ankles, real-time transmitting the information to the mobile phone virtual application to be presented as a mirror. The onboard microprocessor drives the operation of the electric stimulation interface, which is sent through two channels to both, healthy and affected lower limbs. The platform communicates continuously through a USB bidirectional interface with a personal computer, in which the physician enters the rehabilitation protocol and follows the patient performance. A first case study by using this device has been reported in [1].