Augmented Reality Environment Research Articles

Metaverse-based cardiac magnetic resonance imaging simulation application for overcoming claustrophobia: a preliminary feasibility trial.

Introduction: Cardiac magnetic resonance imaging (CMR) is vital, but claustrophobia affects 10% of patients. The metaverse, an immersivevirtual and augmented reality environment, has healthcare potential. We present a metaverse-based CMR simulation for claustrophobic patients. Methods: Three cardiomyopathy patients, initially CMR-refusing due to claustrophobia, received training via a virtual reality headset in a metaverse-based virtual hospital. Training efficacy was assessed through questionnaires and anxiety scales. Results: The patients successfully completed metaverse-based training, adapting to the CMR simulation. On CMR day, all entered the machine without issues and with reduced anxiety. Patients found the training useful, suggesting platform familiarization. Discussion: Our study demonstrates the metaverse's potential in alleviating CMR-related claustrophobia. The immersive nature enhances patient preparation, although usability improvements are needed. Further research should compare this approach with alternatives.

착용형 증강현실 기반 체험형 콘텐츠 연구

This paper proposes AR stone tower content, an experiential content based on wearable augmented reality (AR). Although wearable augmented reality is gaining attention, the acceptance of the technology is still focused on specialized applications such as industrial sites. On the other hand, the proposed AR stone tower content is based on the material of 'stone tower' so that general users can relate to it and easily participate in it, and it is organized to utilize space in a moving environment and find and stack stones based on natural hand gestures. The proposed AR stone tower content was implemented in the HoloLens 2 environment and evaluated by general users through a pilot exhibition in a small art museum. The evaluation results showed that the overall satisfaction with the content averaged 3.85, and the content appropriateness for the stone tower material was very high at 4.15. In particular, users were highly satisfied with content comprehension and sound, but somewhat less satisfied with object recognition, body adaptation, and object control. The above user evaluations confirm the resonance and positive response to the material, but also highlight the difficulties of the average user in experiencing and interacting with the wearable AR environment.

Rock Slope Stability Analysis Using Terrestrial Photogrammetry and Virtual Reality on Ignimbritic Deposits

Puerto de Cajas serves as a vital high-altitude passage in Ecuador, connecting the coastal region to the city of Cuenca. The stability of this rocky massif is carefully managed through the assessment of blocks and discontinuities, ensuring safe travel. This study presents a novel approach, employing rapid and cost-effective methods to evaluate an unexplored area within the protected expanse of Cajas. Using terrestrial photogrammetry and strategically positioned geomechanical stations along the slopes, we generated a detailed point cloud capturing elusive terrain features. We have used terrestrial photogrammetry for digitalization of the slope. Validation of the collected data was achieved by comparing directional data from Cloud Compare software with manual readings using a digital compass integrated in a phone at control points. The analysis encompasses three slopes, employing the SMR, Q-slope, and kinematic methodologies. Results from the SMR system closely align with kinematic analysis, indicating satisfactory slope quality. Nonetheless, continued vigilance in stability control remains imperative for ensuring road safety and preserving the site’s integrity. Moreover, this research lays the groundwork for the creation of a publicly accessible 3D repository, enhancing visualization capabilities through Google Virtual Reality. This initiative not only aids in replicating the findings but also facilitates access to an augmented reality environment, thereby fostering collaborative research endeavors.

Acceptance of augmented reality for laboratory safety training: methodology and an evaluation study

This paper presents a proof-of-concept Augmented Reality (AR) system known as “SMARTLab” for safety training in hazardous material science laboratories. The paper contains an overview of the design rationale, development, methodology, and user study. Participants of the user study were domain experts (i.e., actual lab users in a material science research department, n = 13), and the evaluation used a questionnaire and free-form interview responses. The participants undertook a virtual lab experiment, designed in collaboration with a domain expert. While using the AR environment, they were accompanied by a virtual assistant. The user study provides preliminary findings about the impact of multiple dimensions, such as Performance Expectancy, Emotional Reactivity, and Spatial Presence, on SMARTLab acceptance by analyzing their influence on the Behavioral Intention dimension. The findings indicate that users find the approach useful and that they would consider using such a system. Quantitative and qualitative analysis of the SMARTLab assessment data suggests that a) AR-based training is a potential solution for laboratory safety training without the risk of real-world hazards, b) realism remains an important property for some aspects such as fluid dynamics and experimental procedure, and c) use of a virtual assistant is welcome and provides no sense of discomfort or unease. Furthermore, the study recommends the use of AR assistance tools (a virtual assistant, an attention funnel, and an in-situ arrow) to improve usability and make the training experience more user-friendly.

Elevating metaverse virtual reality experiences through network‐integrated neuro‐fuzzy emotion recognition and adaptive content generation algorithms

AbstractInteractions between individuals and digital material have completely changed with the advent of the Metaverse. Due to this, there is an immediate need to construct cutting‐edge technology that can recognize the emotions of users and continuously provide material that is relevant to their psychological states, improving their overall experience. An inventive method that combines natural language processing adaptive content generation algorithms and neuro‐fuzzy‐based support vector machines natural language processing (SVM‐NLP) is proposed by researchers to meet this demand. With this merging, the Metaverse will be able to offer highly tailored and engaging experiences. Initially, a neuro‐fuzzy algorithm was developed to identify people's emotional moods from their physiological reactions and other biometric information. Fuzzy Logic and Support Vector Machine work together to manage the inherent ambiguity and unpredictability, which results in a more exact and accurate categorization of emotions. A key component of the ACGA is NLP technology, which uses real‐time emotional data to dynamically modify and personalize characters, stories, and interactive features in the Metaverse. The novelty of the proposed approach lies in the innovative integration of neuro‐fuzzy‐based SVM‐NLP algorithms to accurately recognize and adapt to users' emotional states, enhancing the Metaverse experience across various applications. The proposed method is implemented using Python software. This adaptive approach significantly enhances users' immersion, emotional involvement, and overall satisfaction within the augmented reality environment by tailoring information to their responses. The findings show that the SVM‐NLP emotion identification algorithm based on neuro‐fuzzy, has a high degree of accuracy in recognizing emotional states, which holds promise for creating a Metaverse that is more emotionally compelling and immersive. Stronger human–computer interactions and a wider range of applications, including virtual therapy, educational resources, entertainment, and social media networking, might be made possible by integrating SVM‐NLP. These sophisticated systems are around 92% accurate in interpreting the emotions.

Design and development of a mobile augmented reality‐based learning environment for teaching the lives of scientists

AbstractBackgroundScience history is a discipline that teaches the development of scientific thought and the discoveries of scientists. However, these topics can sometimes be abstract and difficult to understand. The use of technology can make the teaching of the history of science more effective, engaging, and accessible. By providing students with visual and interactive experiences, it allows them to better understand abstract concepts and closely follow the discoveries of scientists. In this way, history of science lessons can become more appealing and conducive to learning for students.ObjectivesThis research aims to design and develop a Mobile Augmented Reality (MAR) environment that can be used for teaching science history.MethodsThe research was conducted using a design‐based research method consisting of four stages: Analysis, design, development‐implementation‐evaluation, and reporting. The research involved 10 science teachers and five secondary school students. Data were collected through focus group interviews, individual interviews, notes, audio recordings, and images, and then analysed using content analysis.Results and ConclusionsAs a result of the research, a MAR application named TISAR‐3D was developed, which allows students to interactively and visually explore scientists and enrich their learning experience.ImplicationsThe widespread use of the TISAR‐3D application in science history education is recommended, as well as the development of similar MAR applications for other topics.

Hand drawing-based daylight analysis using deep learning and augmented reality

The utilization of artificial intelligence (AI) in daylight analysis has recently experienced an increase. Its capabilities have been demonstrated through different studies. While using AI for daytime simulation can reduce time, it also presents significant challenges. Whether we utilize the conventional approach of simulating daylight or employ artificial intelligence, the process of going from hand drawing to obtaining the daylight analysis result involves intricate procedures associated with modeling, programming, and integrating AI models. This paper introduces an efficient and simple approach to rapidly predict daylight performance directly from the user's hand-drawn sketches. The core idea is to leverage a deep learning model for predicting directly from hand drawing and augmented reality (AR) to seamlessly display results in an AR environment, overlaying them on the original drawings. This method not only increases user engagement and accessibility but also eliminates the dependency on traditional daylight simulation software and workflow. Artificial neural network (ANN) was exploited using data derived from 3D building models and Useful Daylight Illuminance simulations created by Rhino3D and Honeybee. The color code method, various imaging, and data processing techniques were implemented to ensure compatibility between the hand-drawn input and the simulation model. Unity3D is used as the development platform with the AR engine Vuforia. Notably, the ANN model yields promising results, with R2 ranging from 0.872 to 0.985. This is achieved by efficiently capturing and converting grid-based hand-drawn images into suitable input for the ANN. The obtained results demonstrate the feasibility and emphasize the concept's practicality.

GPS-Induced Disparity Correction for Accurate Object Placement in Augmented Reality

The use of augmented reality (AR) continues to increase, particularly in marketing and advertising, where virtual objects are showcased in the AR world, thereby expanding its various applications. In this paper, a method of linking coordinate systems to connect the metaverse with the real world is proposed and a system for correcting and displaying virtual objects in the AR environment is implemented. The proposed method calculates errors to accurately represent virtual objects in AR and presents a method to show these objects without errors. The proposed method was verified through experiments to successfully display virtual objects in AR. To minimize localization errors, semantic segmentation was used to recognize objects and estimate buildings, thereby correcting the device location. An error correction expression is also presented. The proposed system is designed to correct and display virtual objects in AR, with confirmed functionality for location correction.

An end-to-end approach for blindly rendering a virtual sound source in an audio augmented reality environment

Audio augmented reality (AAR), a prominent topic in the field of audio, requires understanding the listening environment of the user for rendering an authentic virtual auditory object. Reverberation time (RT60\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$RT_{60}$$\\end{document}) is a predominant metric for the characterization of room acoustics and numerous approaches have been proposed to estimate it blindly from a reverberant speech signal. However, a single RT60\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$RT_{60}$$\\end{document} value may not be sufficient to correctly describe and render the acoustics of a room. This contribution presents a method for the estimation of multiple room acoustic parameters required to render close-to-accurate room acoustics in an unknown environment. It is shown how these parameters can be estimated blindly using an audio transformer that can be deployed on a mobile device. Furthermore, the paper also discusses the use of the estimated room acoustic parameters to find a similar room from a dataset of real BRIRs that can be further used for rendering the virtual audio source. Additionally, a novel binaural room impulse response (BRIR) augmentation technique to overcome the limitation of inadequate data is proposed. Finally, the proposed method is validated perceptually by means of a listening test.

Towards sensory substitution and augmentation: Mapping visual distance to audio and tactile frequency.

Multimodal perception is the predominant means by which individuals experience and interact with the world. However, sensory dysfunction or loss can significantly impede this process. In such cases, cross-modality research offers valuable insight into how we can compensate for these sensory deficits through sensory substitution. Although sight and hearing are both used to estimate the distance to an object (e.g., by visual size and sound volume) and the perception of distance is an important element in navigation and guidance, it is not widely studied in cross-modal research. We investigate the relationship between audio and vibrotactile frequencies (in the ranges 47-2,764 Hz and 10-99 Hz, respectively) and distances uniformly distributed in the range 1-12 m. In our experiments participants mapped the distance (represented by an image of a model at that distance) to a frequency via adjusting a virtual tuning knob. The results revealed that the majority (more than 76%) of participants demonstrated a strong negative monotonic relationship between frequency and distance, across both vibrotactile (represented by a natural log function) and auditory domains (represented by an exponential function). However, a subgroup of participants showed the opposite positive linear relationship between frequency and distance. The strong cross-modal sensory correlation could contribute to the development of assistive robotic technologies and devices to augment human perception. This work provides the fundamental foundation for future assisted HRI applications where a mapping between distance and frequency is needed, for example for people with vision or hearing loss, drivers with loss of focus or response delay, doctors undertaking teleoperation surgery, and users in augmented reality (AR) or virtual reality (VR) environments.

Non-deterministic method for semi-automatic calibration of smartphone-based OST HMDs

An Augmented Reality (AR) system must show real and virtual elements as if they coexisted in the same environment. The tridimensional aligment (registration) is particularly challenging on specific hardware configurations such as Head Mounted Displays (HMDs) that use Optical See-Through (OST) technology. In general, the calibration of HMDs uses deterministic optimization methods. However, non-deterministic methods have been proposed in the literature with promising results in distinct research areas. In this work, we developed a non-deterministic optimization method for the semi-automatic calibration of smartphone-based OST HMDs. We tested simulated annealing, evolutionary strategy, and particle swarm algorithms. We also developed a system for calibration and evaluated it through an application that aligned a virtual object in an AR environment. We evaluated our method using the Mean Squared Error (MSE) at each calibration step, considering the difference between the ideal/observed positions of a set of reference points and those estimated from the values determined for the calibration parameters. Our results show an accurate OST HMD calibration for the peripersonal space, with similar MSEs for the three tested algorithms.

Projection mapping technologies: A review of current trends and future directions.

This study summarizes current trends and future directions in projection mapping technologies. Projection mapping seamlessly merges the virtual and real worlds through projected imagery onto physical surfaces, creating an augmented reality environment. Beyond traditional applications in advertising, art, and entertainment, various fields, including medical surgery, product design, and telecommunications, have embraced projection mapping. This study categorizes recent techniques that address technical challenges in accurately replicating desired appearances on physical surfaces through projected imagery into four groups: geometric registration, radiometric compensation, defocus compensation, and shadow removal. It subsequently introduces unconventional projectors developed to resolve specific technical issues and discusses two approaches for overcoming the inherent limitations of projector hardware, such as the inability to display images floating above physical surfaces. Finally, this study concludes the discussion with possible future directions for projection mapping technologies.

Genesis of legal regulation of the web and the model of the Metaverse electronic jurisdiction

The study examines the transformation of scientific views and approaches to the problem of expediency and necessity of legal regulation of public relations, emerging from the evolution of the world system of public electronic resources in the transmission of information and Internet data from Web 1.0, Web 2.0 to Web 3.0. The stages of formation of the role and place of electronic jurisdiction in public relations are also investigated. Legal regulation of modern relations in virtual and augmented reality environments with the use of Web 3.0 technologies is not available today. At the same time, there are precedents for the application of certain provisions of analogue law to address legal uncertainties in the virtual environment, such as establishing ownership of virtual non-property assets, buying/selling of virtual non-property assets, liability for misappropriation of virtual non-property assets, etc. Obviously, the problem of legal regulation by the rules of analogue law in the virtual environment cannot be fully addressed. The solution to this problem is possible by creating a comprehensive e-jurisdiction and developing the Metaverse Grand Charter of Laws to regulate public relations in the meta-universe and to establish new branch of e-law. Given the urgency of the problem, the model of e-jurisdiction Grand Charter of Laws Metaverse is proposed. The model of complex electronic jurisdiction of Metaverse will allow to create basic conceptual apparatus, doctrinal and regulatory and legal concepts, to define objects and subjects of legal relations in Metaverse, to establish the basic forms of legal relations and mutual relations in Metaverse. This, in turn, will be the basis for reforming analogue legislation, partial interoperability in the digital environment and the development of new regulations in various areas of law and will stimulate the establishment of new e-jurisdiction. The study proposes the construction and basic elements of electronic jurisdiction, mechanisms for the separation of electronic offences and interaction with analogue jurisdictions. E-jurisdiction of the Metaverse Grand Charter of Laws will provide legal regulation of public relations both directly in Metaverse and in public relations related to the analogue and electronic world.

A review of the application of virtual and augmented reality in physical and occupational therapy

AbstractThis paper includes a research review in five bibliographic databases on using the application of virtual reality (VR) and augmented reality (AR) in physical and occupational therapy (POT). This literature review addresses five research questions and two sub‐research questions. A total of 36 relevant studies were selected in the review based on the defined keywords and inclusion‐exclusion criteria. The primary motivation for using the application of VR and AR in POT is that it is accurate, involves higher patient participation, and requires less therapy recovery time. The standard software tool used is the Unity 3D game engine, and the common device used is the Oculus Rift HMD. Various applications of VR and AR consist of different VR environments and AR contents used in POT. Post‐stroke rehabilitation, rehabilitation exercises, pain management, mental and behavioral disorders, and autism in children are the main aspects addressed through the VR and AR environments. Literature review indicates that questionnaires, interviews, and observation are the primary metrics for measuring therapy's effectiveness. The study's findings show positive results such as reduced treatment time, nervousness, pain, hospitalization period, making therapy enjoyable and encouraging, improved quality of life, and focus on using the application of VR and AR in POT. This review will be relevant to researchers, VR and AR application designers, doctors, and patients using the application of VR and AR in POT. Further research addressing multiple participants with clinical trials, adding new VR environments and AR content in VR and AR applications, including follow‐up sessions, and increasing training sessions while using the application of VR and AR in POT are recommended.

PCQD‐AR: Subjective quality assessment of compressed point clouds with head‐mounted augmented reality

AbstractThis letter fully studies the coloured point cloud quality assessment in augmented reality (AR) environment through subjective test. Firstly, a point cloud dataset, named point cloud quality dataset‐AR, including ten reference point clouds and their 90 distorted versions is presented, which were generated using the reference software of video‐based point cloud compression across various combinations of geometry and texture quantization parameters. Then, the impact of geometry and texture distortions on perceived quality of point clouds in AR environment was discussed in detail. Moreover, the performance of existing objective point cloud quality assessment metrics on the proposed dataset is evaluated. The subjective dataset including the values of mean opinion score were released to public.

The Course of the Trochlear Nerve Presented via a 3-Dimensional Photorealistic Anatomic Model

Several factors contribute to the anatomical complexity of the trochlear nerve including small diameter, complex and longest intracranial course, deep location, and numerous neurovascular relationships. 3-D photorealistic model of cranial nerves provides detailed and immersive representation of the anatomy, enabling to improve surgical planning, advanced surgical research and training. The purpose of this paper is to present a 3-dimentional photogrammetric study for a more intuitive and interactive way to explore and describe the entire course of trochlear nerve. Two injected-fixed head human specimens (4 sides) were examined. The dissection protocol was divided into: step 1) brain hemisphere exposure, step 2) hemispherectomy dissecting all cranial nerves and partial removal of the free edge of the tentorium, step 3) middle fossa and lateral wall of cavernous sinus exposure, step 4) orbital exposure. A detailed 3-D photogrammetric model was generated for each dissection step. Four main volumetric models were generated during a step-by-step layered dissection of the entire nerve pathway highlighting its different segments. Finally, a full and integrated model of the entire course of the nerve was created. The models are available for visualization on monoscopic display, virtual, and augmented reality environment. The present photogrammetric model provides a more comprehensive understanding of the nerve's anatomy in its different segments, allows for customizable views thus simulating different perspectives, and can be a valuable alternative to traditional dissections. It is an advanced tool for surgical planning and surgical simulation as well as virtual reality representation of the anatomy.

Enhancing Calculus Learning through Interactive VR and AR Technologies: A Study on Immersive Educational Tools

In the realm of collegiate education, calculus can be quite challenging for students. Many students struggle to visualize abstract concepts, as mathematics often moves into strict arithmetic rather than geometric understanding. Our study presents an innovative solution to this problem: an immersive, interactive VR graphing tool capable of standard 2D graphs, solids of revolution, and a series of visualizations deemed potentially useful to struggling students. This tool was developed within the Unity 3D engine, and while interaction and expression parsing rely on existing libraries, core functionalities were developed independently. As a pilot study, it includes qualitative information from a survey of students currently or previously enrolled in Calculus II/III courses, revealing its potential effectiveness. This survey primarily aims to determine the tool’s viability in future endeavors. The positive response suggests the tool’s immediate usefulness and its promising future in educational settings, prompting further exploration and consideration for adaptation into an Augmented Reality (AR) environment.

Augmented Reality-Based Real-Time Visualization for Structural Modal Identification.

In the era of aging civil infrastructure and growing concerns about rapid structural deterioration due to climate change, the demand for real-time structural health monitoring (SHM) techniques has been predominant worldwide. Traditional SHM methods face challenges, including delays in processing acquired data from large structures, time-intensive dense instrumentation, and visualization of real-time structural information. To address these issues, this paper develops a novel real-time visualization method using Augmented Reality (AR) to enhance vibration-based onsite structural inspections. The proposed approach presents a visualization system designed for real-time fieldwork, enabling detailed multi-sensor analyses within the immersive environment of AR. Leveraging the remote connectivity of the AR device, real-time communication is established with an external database and Python library through a web server, expanding the analytical capabilities of data acquisition, and data processing, such as modal identification, and the resulting visualization of SHM information. The proposed system allows live visualization of time-domain, frequency-domain, and system identification information through AR. This paper provides an overview of the proposed technology and presents the results of a lab-scale experimental model. It is concluded that the proposed approach yields accurate processing of real-time data and visualization of system identification information by highlighting its potential to enhance efficiency and safety in SHM by integrating AR technology with real-world fieldwork.

Digital Apparel Try-Ons: A Technological Odyssey into the World of Virtual Dressing Rooms and Consumer Engagement

This individual can now enjoy internet shopping whenever they want and from anywhere in the world thanks to the invention of smartphones and tablets. Online shopping has unquestionably taken the place of conventional methods for purchasing apparel and daily necessities. People benefit from a reputation as they pick internet buying. Almost all online retailers offer payment on delivery, free shipment, and special rates these days. These internet merchants have eliminated the hassles of parking, stuck in traffic, and standing in line to make purchases. They have also helped folks who consistently lament their lack of time. This is the primary justification for why most consumers now shop online. They may easily access a wide choice of affordable prices here, and they also get quick customer service and free home delivery. There is no denying that these are a few of the alluring aspects that draw customers in. Even though there is a minor problem that can make people lose interest in online buying, they might not be able to try on garments in such situations. By establishing a virtual changing room setting, the aim is to improve accessibility and time efficiency for trying on clothing. Our proposed methodology consists of three main components: skin color recognition, model alignment, and user image extraction from the video stream. Through user extraction, we may create an augmented reality environment by taking the user area out of the video stream and superimposing it on top of a simulated world within the user interface. To match the 2D cloth models with the use of scaling, rotation, and positioning using the 3D locations of the joints.

Research on the technology of intelligent vehicle sensor positioning system in autonomous driving

In this comprehensive exploration of positioning and navigation technologies, we have delved into the intricacies of GPS (Global Positioning System), IMU (Inertial Measurement Unit), and VPS (Visual Positioning Systems) systems, shedding light on their unique attributes and applications. GPS, a global satellite-based system, offers precise positioning on a worldwide scale, although it encounters challenges in complex urban and environmental conditions. Its role in navigation and tracking is undeniable. IMU, characterized by accelerometers and gyroscopes, excels in delivering high-precision positioning over short durations, making it invaluable for applications in aviation, robotics, and virtual reality. However, it is susceptible to drift over time. Visual Positioning Systems (VPS), harnessing computer vision and visual sensors, provide remarkable sub-meter to centimeter-level accuracy when conditions are optimal. Their significance is particularly pronounced in indoor navigation, augmented reality, and robotics, although they may face challenges in less favorable environments. These technologies are not isolated but can synergize to enhance accuracy and reliability. GPS and IMU collaborate to compensate for signal disruptions, while GPS and VPS join forces to tackle urban complexities. IMU and VPS integration offer precise indoor navigation and augmented reality experiences, delivering impeccable positioning and orientation data. Ultimately, the choice of technology hinges on specific application requirements, encompassing accuracy, environmental considerations, cost factors, and the need for complementary systems. As these technologies advance, they hold the promise of revolutionizing navigation across various domains, from autonomous vehicles to immersive augmented reality environments.