Initial Evaluation of an Intelligent Virtual Museum Prototype Powered by AI, XR and Robots

Several of the next generation nuclear reactors being considered will require autonomous maintenance capabilities, due to the hostile radiation and thermal environments that exclude human operators from supporting equipment in close proximity to the reactors. Research is ongoing to use virtual training environments to train robots to perform maintenance and emergency response operations. Since future designs are still under development, there is no testbed for physical experiments. To overcome this obstacle, we have developed a generalized methodology to train robots in a virtual environment. In this regard, the virtual environment can be created from either 3-D mapping of existing physical components or from components that only exist as engineering specifications, as long as their dimensions and behaviors can be represented in the virtual world. Our approach creates an integrated software and algorithm architecture with the ability to “teach” automated systems successful tasks from simulated data sets with minimal human oversight. In this work, we develop a digital-twin system for training and real-time control of a robotic arm. The digital twin comprises a virtual training environment and a real-time control interface. The virtual training environment allows simulation of maintenance tasks under diverse domain randomizations. After training, the physical robot is controlled while monitoring anticipated behavior of the digital twin in real time. This approach increases operator confidence and can decrease risk compared to deployment of a model without real-time supervision of the digital-twin alongside the physical machine. It is clear that this approach has extensive applications to terrestrial and space-based tasks. Autonomous systems including robots, will support future maintenance and security protocols and this method provides one possible approach to address future uncertainties or responses to accidents. This technology is in its infancy but expected to mature rapidly and undertake complex and dynamic tasks using reinforcement learning derived algorithms.

The Medical Metaverse, Part 1: Introduction, Definitions, and New Horizons for Neuropsychiatry.

Digital twins represent real objects or subjects with their data, functions, and communication capabilities in the digital world. As nodes within the internet of things, they enable networking and thus the automation of complex value-added chains. The application of simulation techniques brings digital twins to life and makes them experimentable; digital twins become experimentable digital twins (EDTs). Initially, these EDTs communicate with each other purely in the virtual world. The resulting networks of interacting EDTs model different application scenarios and are simulated in virtual testbeds, providing new foundations for comprehensive simulation-based systems engineering. Its focus is on EDTs, which become more detailed with every single application. Thus, complete digital representations of the respective real assets and their behaviors are created successively. The networking of EDTs with real assets leads to hybrid application scenarios in which EDTs are used in combination with real hardware, thus realizing complex control algorithms, innovative user interfaces, or mental models for intelligent systems.

Tourism is one of the largest industries and one of the fastest growing sectors in the world. At present and due to accessibility and improvement of technology, the tourism industry has been using experiences based on Virtual Reality, because they offer immersive experiences that absorb the user through sensory stimulation like sight and sound. The game, plays an essential role in the imaginary constructive process. By engaging fascination through interactivity, digital games introduce a sense of experimentation that facilitates immersion. In this context, the present preliminary study sought to assess the user experience through development of an immersive virtual reality game, to share the cultural aspects of Sao Tome island and engaging the participants to visit the island. The goal of this study is to evaluate usability and user experience in a three-dimensional virtual reality environment prototype of a beach place, in a traditional hostel. Concerning usability, we evaluated the participants interaction in virtual world through their behavior, and responses to questionnaires of presence and simulator sickness. To evaluate the user experience, we evaluated emotions through use of a scale of valence and arousal (SAM) and categories of emotions elicited by the VR simulation using the Geneva Emotion Wheel. The results reveal that the three-dimensional virtual reality environment prototype provided high level of immersion, the control and sensory factors also present high value, while the distraction factors recorded reveal that they were not significant, and participant also felt that the experience had a high level of realism. No significant simulator sickness problems were reported. User experience was emotionally positive with high values of pleasure and contentment. In conclusion, the 3D virtual reality environment prototype was able to engage the participants and provide interactive experiences capable to seducing and arousing interest in the Sao Tome and Principe islands.

The modern development of computer technologies is accompanied by an increasing immersion of a person into the virtual world, where he often spends many hours. The longer a person uses a computer and various electronic gadgets, the more traces he leaves. In fact, there is a situation in which a digital copy or digital twin of a person appears in the virtual world. This phenomenon is new to legal science and requires its own research. The purpose of the article is to analyze the concept of «digital twin» and its development, to study the approaches of foreign and Russian authors from various fields of knowledge to the definition of this term, to identify the possibilities of using this concept in the legal sphere. When writing the article, the following methods are used: comparative legal, analysis and synthesis. The following results are obtained: every computer and gadget user has his «digital copy», a kind of «digital twin», on behalf of which a person performs various, including criminal actions in the virtual space. Public relations in the cyberspace are largely not regulated by law, and when developing a framework for regulating such specific legal relations, it is necessary to begin with the definition of the subject. The authors pay special attention to the question of equating a digital identity with a real one, as well as the lack of legislative regulation of the legal status of a digital twin. The discrepancy between such categories as «digital identity» and «digital twin» is shown. A more correct definition of a digital twin is proposed for legal regulation, in particular for identifying the account with which the Internet crime was committed – a digital identity twin. The paper also proposes the basic principles for the subsequent legal regulation of public relations arising and developing in the cyberspace and lays the foundation for further research on the topic of legal regulation of the digital twin.

<div class="section abstract"><div class="htmlview paragraph">With the increasing connectivity and complexity of modern automobiles, cybersecurity has become one of the most important properties of a vehicle. Various strategies have been proposed to enhance automotive cybersecurity. Digital twin (DT), regarded as one of the top 10 strategic technology trends by Gartner in 2018 and 2019, establishes digital representations in a virtual world and raises new ideas to benefit real-life objects. In this paper, we explored the possibility of using digital twin technology to improve automotive cybersecurity. We designed two kinds of digital twin models, named mirror DT and autonomous DT, and corresponding environments to support cybersecurity design, development, and maintenance in an auto’s lifecycle, as well as technique training. The mirror DT, which displays the external behaviors of the physical object, collects, displays, and analyzes real-time data, for specific purposes like security analysis, anomaly detection, and so on. The autonomous DT models the internal logic of the real-world object, and is applicable for virtual design and debugging, system evaluation, and training in a purely virtual world. We proposed approaches for building the DT models, established a real-virtual interaction environment, and explored several feasible applications. Prototypes were developed to verify the effectiveness and expansibility of the proposed approaches and applications. In summary, DT technology helps engineers to improve work efficiency and effectiveness of cybersecurity design and management activities during the whole product lifecycle. Furthermore, the digital world is also ideal for professional training to help learners to better understand corresponding technics and do more exercises in the virtual world at a low cost.</div></div>

Non-Verbal Cues (NVCs) add to communication effectiveness among individuals in both real and virtual world. Thus, NVCs transference between the two receives increased attention from both the industry and research community. Their efforts lead to sophisticated technological solutions which allow high fidelity NVCs to be transferred from real individuals to virtual characters/avatars, in real-time, adding to the communication effectiveness of both standalone Immersive Virtual Reality (IVR) applications and Social Virtual Reality (SVR) platforms. That combination triggered the development of Virtual Reality Learning Environments (VRLEs), aiming at assisting students towards learning sign language and communicating through it in both virtual and real world. Still, technological limitations affect the quality of such learning and communication experience. This work presents the design, development, and evaluation of an SVR compatible system for sign language learning and communication in virtual worlds. The system considers lessons learned from previous studies and was used to create a VRLE mimicking learning and examination processes students of sign language follow in real life. The results of our study show that despite the appreciation of technological solutions efficiency towards NVCs transference, there are still several issues to be investigated to increase the effectiveness of communication through sign language in virtual worlds.

A wide variety of virtual worlds exists today. They embed the user into a virtual reality allowing interaction with the virtual environment and, in case of multiplayer worlds, other users. Our goal is to support these interactions. Therefore we are using virtual parameters to describe the user's virtual context and provide additional services depending on this context. These services can either be included into the virtual world or can be realized using a community portal. In this paper we will introduce the concept of virtual context based services and its fundamentals like the classification of interaction in virtual worlds, a generic interaction model and an extended model including additional services.

Chapter 9 - Digital twin based computerized numerical control machine tool virtual prototype design

Virtual worlds (VWs) have become increasingly popular to support students' foreign language learning, especially beyond the classroom. Unfortunately students' interaction in VWs is not always available for the supervisor and thus is not easy to analyse. Nonetheless, it provides interesting information not only in terms of assessment, but also to detect general learner profiles and trends in their use. Therefore we propose a computer system that accepts queries in a simple but specific language for VWs, allowing the supervisor to design assessment formulas, generate reports on students' interaction in VWs and refine the formulas from these results until obtaining a valid indicator for analysing and assessing the retrieved data. In this paper, we describe a case study which has been carried out in a German foreign language course, using a VW implemented video-game and assessing students' foreign language skills. The study has been deployed in a scalable way through our computer system. Preliminary results provide a set of indicators for individual and group behaviour which can be used to assess student's ability to communicate in the target language.

In order to realize the collaborative operation between the physical prototype and the virtual prototype of industrial robot, a digital twin model of a six-degree-of-freedom industrial robot was established. The structure and motion parameters of the 6-DOF industrial robot are introduced. The virtual prototype model of the industrial robot was established based on Visual One; the digital twin modeling of the industrial robot was completed by the three methods communication between the virtual prototype of the industrial robot and the physical prototype was realized, and the virtual and real collaborative operation of the twin robot and the physical robot was realized. Experiments show that when manipulating the movement of the robot on the physical terminal, the digital twin was consistent with its behavior. The research results provide references for the application of digital twin technology in the field of industrial robots.

Digital twins are rapidly establishing themselves as a necessary step in digital change also in the design of automated and robotic manufacturing systems. The digital twin serves as an interface between physical industrial products and digital worlds. This allows you to map the continuous availability of data throughout the lifecycle from product planning and development, production and commissioning to use and recycling. An important step towards this goal is to create digital twins of the components in order to create a database usable for the design of new production systems in a virtual environment. One of the possibilities of using the digital twins is testing and virtual commissioning of production systems. Instead of the classic CAD model, the digital twin replaces a detailed digital image of individual components, including their full functionality. The Digital Twin comprises the CAD model, kinematic behavior model a most important is the logical behavior model. Our goal is to create a digital twin component of the production system, integrate a CAD model, create a kinematic model and a logical behavior model. This digital twin will have signals assigned to it for logical behavior. The result is the verification and testing of the information exchange between the virtual control system and the virtual model, using the Software-in-the-Loop method and the Tecnomatix Process Simulate software environment on the digital twin side and the Siemens TIA Portal with PLC SIM Advanced on the controller side. The result of the knowledge is that the exchange of information is functional and usable for virtual commissioning.

The industry use cases for the Digital Twin idea

Increasing industrial development and digital transformations have given rise to a technology called Digital Twin (DT) that has the potential to break the barrier between physical and cyberspace. DT is a virtual and dynamic model enabled through a bidirectional data flow that creates high-reliability models with interconnection and fusion between the physical and digital systems for full integration. In smart manufacturing, this technology is increasingly used in research and industry. However, the studies conducted do not provide a definition or a single integrally connected model. To develop the Digital Twin shown in this research, the literature was reviewed to learn about the enabling technologies and architectures used at the industrial level. Then, a methodology was used to obtain the physical process information, create the digital environment, communicate the physical environment, apply simulation models in the digital environment, and parameterize the simulation environment with the physical process in real-time to obtain the digital twin supported with augmented reality. The system was implemented in the MPS-500 modular production station that has industrial sensors and actuators. The virtual environment was designed with Blender and Vuforia to create the augmented reality environment. In the proposed methodology, robust devices (field and control level) and low-cost embedded systems were used for the creation and communication of the virtual environment (monitoring and control); for the joint work of these technologies, they were carried out through the use of the following protocols: Open Platform Communications United Architecture (OPC UA), Ethernet, and machine to machine (M2M), with which a communication was achieved between the different levels of the automation pyramid. The results show that the proposed methodology for the implementation of the DT allows bidirectional communication between the physical and virtual environment and can also be visualized with the support of AR, thus providing its characteristics to the proposed DT. Digital Twin is an essential factor in creating virtual environments and improving applications between the real and digital world, establishing a bidirectional communication through the Ethernet protocol, with a communication time of approximately 100 ms. This technology interacts with the virtual environment and performs mappings, thus achieving timely and dynamic adjustment. This improves data management and production and incorporates process simulation and physical control in real-time, allowing to execute and trigger actions in the physical equipment simultaneously.

This paper examines youth storytelling during co-design of an identity-expressive educational game for learning data science.Using interaction analysis of co-design interviews with middle school students, we explore how storytelling allows participants to (a) use existing game elements to position themselves and (b) express their interests and identities through imagined game futures.We argue that analyzing youth's interactive storytelling and identity enactments during co-design can inform the development of game narratives that represent diverse youth.Our findings contribute to the design of inclusive virtual worlds for STEM learning that celebrate youth identities and experiences.We discuss implications for engaging youth voices in the co-design process and creating educational games that resonate with diverse learners. PurposeVideo games are important contexts for inclusive STEM learning, allowing players to explore and enact identities (Gaydos & Devane, 2019; Gee, 2014).However, designing educational games that connect to diverse players' lives and identities is a significant challenge.Our project employs youth co-design to create "The Isles of Ilkmaar," a virtual world that supports identity-expressive data science learning among diverse youth, particularly genderexpansive youth, cis-girls, and youth of color.In this paper, we explore storytelling as a method for engaging diverse youth in co-design.An initial analysis showed how storytelling within co-design activates youth expertise and imaginations, facilitating a transition from game players to game designers (Radke, et al., 2023).We now look more closely at storytelling episodes to understand youth's shifting positionalities within the co-design context, illustrating how storytelling allows youth to (a) refer to and take up existing game elements as resources for positioning themselves as designers and (b) author and animate imagined new game futures in which they can projectively identify (Gee, 2014) with the player character.By analyzing youth narratives through the lens of positioning, we aim to understand how youth contributions to co-design arise from and reflect the dual positionalities of game designer and future player, contributing to the understanding of how to design educational games that connect to diverse players' lives and identities.