Scene Simulation Research Articles

A comprehensive experimental framework based on analysis of the pilot's EEG and NASA-TLX questionnaire in a VR environment.

The relationship framework between electroencephalogram (EEG) signals and subjective perception during pilots' missions was established to improve pilots' training efficiency and flight safety. Primarily, this study constructs a real flight scene through virtual reality (VR) and then obtains EEG data in simulated scenes. Researchers use VR technology to build a mission simulation room and then acquire EEG data from participants wearing EEG acquisition devices in the simulated room. The experimental process is divided into flight simulation and a questionnaire survey. Based on the participants' EEG analysis, the researchers verified the changes of β rhythm under a high-difficulty operation mission. In addition, this study infers the mechanism of affecting pilots' mental workload under high-difficulty operation by analyzing the correlation between subjective questionnaire results and β rhythms. The results showed that in the context of pilots performing flight missions in the aircraft space environment, the pilots' mental load had the most excellent rhythmic relationship with the regions representing β rhythm. Therefore, a comprehensive experimental framework constructed in this study based on virtual simulation space to analyze the relationship between EEG and NASA-TLX subjective questionnaire provides a set of more accurate reference information for the design of a pilot training system based on pilot training efficiency and flight safety.

Robot trajectory planning for autonomous 3D reconstruction of cockpit in aircraft final assembly testing

The trend towards automation and intelligence in aircraft final assembly testing has led to a new demand for autonomous perception of unknown cockpit operation scenes in robotic collaborative airborne system testing. To address this demand, a robotic automated 3D reconstruction cell which enables to autonomously plan the robot end-camera's trajectory is developed for image acquisition and 3D modeling of the cockpit operation scene. A continuous viewpoint path planning algorithm is proposed that incorporates both 3D reconstruction quality and robot path quality into optimization process. Smoothness metrics for viewpoint position paths and orientation paths are introduced together for the first time in 3D reconstruction. To ensure safe and effective movement, two spatial constraints, Domain of View Admissible Position (DVAP) and Domain of View Admissible Orientation (DVAO), are implemented to account for robot reachability and collision avoidance. By using diffeomorphism mapping, the orientation path is transformed into 3D, consistent with the position path. Both orientation and position paths can be optimized in a unified framework to maximize the gain of reconstruction quality and path smoothness within DVAP and DVAO. The reconstruction cell is capable of automatic data acquisition and fine scene modeling, using the generated robot C-space trajectory. Simulation and physical scene experiments have confirmed the effectiveness of the proposed method to achieve high-precision 3D reconstruction while optimizing robot motion quality.

Accurate and Efficient Self-Localization of AGV Relying on Trusted Area Information in Dynamic Industrial Scene

Automatic Guided Vehicles (AGVs) are crucial for improving the operational efficiency of industrial scenes, however the self-localization of AGVs faces severe challenges in a dynamic environment due to a large number of observation noises. To address the above issue, this paper presents a trusted area information based self-localization method to achieve efficient and accurate localization in a highly dynamic scene, where the proportion of dynamic obstacles is close to 50%. First of all, the “coarse-noise filter-fine” self-localization framework is proposed to establish the foundation of accurate and efficient localization. Then, the morphology-based trusted area selection is adopted to efficaciously extract the trusted area, in which the probability of appearing dynamic objects is low. After that, based on dynamic angular resolution filtering considering the trusted area, a robust point cloud generation is designed to generate trusted area information. Trusted area information contains dense reliable point cloud in the statice area and spares observation noise in the dynamic area, thus improving the performance of observation models and scan matching. Finally, the self-developed AGV is adopted to verify the effectiveness of the proposed self-localization system in real and simulated dynamic scenes.

Remote Respiratory and Cardiac Motion Patterns Separation with 4D Imaging Radars.

Radar-based noncontact physiological signals monitoring is meaningful for daily health monitoring, post-disaster rescue, and public security. This paper focuses on the theoretical and experimental study of noncontact respiratory and cardiac motion signals separation employing remote sensing by a four-dimensional (4D) imaging radar. To adaptively separate respiratory and cardiac motion patterns, we propose a variational mode separation (VMS) algorithm. VMS is established on optimizing a variational problem to separate different modes. It minimizes the energy overlap of the heartbeat and respiration signals as well as their harmonics with an equality constraint. Both simulation and real scene data results show that the proposed VMS algorithm is suitable for separating the weaker cardiac motion pattern from the strong respiratory motion pattern, restraining the influence of respiration harmonics on the heartbeat component. Also, the continuous remote monitoring of respiratory rate (RR) and heart rate (HR) by employing the proposed method in a real scene shows a strong consistency with the reference respiration belt and electrocardiogram (ECG). The root mean square error (RMSE) of RR and HR for the noncontact measurement are 0.13 breaths per minute (brpm) and 1.7 beats per minute (bpm), respectively.

Exploration With Intrinsic Motivation Using Object–Action–Outcome Latent Space

One effective approach for equipping artificial agents with sensorimotor skills is to use self-exploration. To do this efficiently is critical, as time and data collection are costly. In this study, we propose an exploration mechanism that blends action, object, and action outcome representations into a latent space, where local regions are formed to host forward model learning. The agent uses intrinsic motivation to select the forward model with the highest learning progress to adopt at a given exploration step. This parallels how infants learn, as high learning progress indicates that the learning problem is neither too easy nor too difficult in the selected region. The proposed approach is validated with a simulated robot in a table-top environment. The simulation scene comprises a robot and various objects, where the robot interacts with one of them each time using a set of parameterized actions and learns the outcomes of these interactions. With the proposed approach, the robot organizes its curriculum of learning as in existing intrinsic motivation approaches and outperforms them in learning speed. Moreover, the learning regime demonstrates features that partially match infant development; in particular, the proposed system learns to predict the outcomes of different skills in a staged manner.

Evaluation index system and evaluation method of energy storage and regional power grid coordinated peak regulation ability

With the development of energy storage technology, energy storage technology began to be put into the peak regulation of power grid. But at present, the lack of scientific evaluation means for coordinated peak regulation ability of energy storage and regional power grid (ESRPG) hinders the large-scale participation of energy storage devices in peak regulation. To solve this problem, this paper proposes an evaluation system and evaluation method to comprehensively and accurately evaluate the coordinated peak regulation ability of ESRPG. Firstly, an evaluation system for the coordinated peak regulation ability of ESRPG is established from the perspective of energy storage and generator units. The indicator system comprehensively considers the impact of technology, economy, reliability, and other factors on the peak regulation ability of the ESRPG. At the same time, a multi-level evaluation method composed of an analytic hierarchy process and weighted rank sum ration is proposed to comprehensively evaluate the proposed indicator system. This method not only takes subjective and objective factors into account, but also eliminates the interference of dimensions, orders of magnitude and other factors on the evaluation results. Finally, the evaluation indicator system and evaluation method are applied to the simulation scene for evaluation, and the results show that the evaluation system and method make up for the shortcomings of the current mainstream methods and effectively evaluate the peak shaving capability.

Research on practical teaching of Ideological and political education based on VR technology in the information age

Abstract With the popularity and maturity of VR technology, maximizing the utility advantages of relevant technologies and teaching projects has become a new topic. Because of the current era of science and technology’s best ideas, relevant methods of Ideological and political education learning have become richer in form and connotation. In order to make relevant methods better, true adoption experimental teaching of Ideological and political education here explores the true adoption of relevant methods in Ideological and political education college students’ learning. Through questionnaire survey to study the effect of relevant methods used by college students’ learning Ideological and political education and fuzzy analysis judging method to argue the teaching effect of integrating VR in Ideological and political education classroom, using comparative analysis method to compare and reference the traditional means of communication of Ideological and political education college students’ learning VR (virtual reality) The advantages of VR over traditional means of communication are summarized. Finally, the reasonableness and Probability of possible realization of relevant methods of college Ideological and political education are demonstrated by considering the needs and feasibility of the presented audience. The results show that most students’ applicability of the confirmation method and optimistic ideas and expressions for future development prospect of applying VR, a real simulated scenes relevant methods, to the practical college students’ learning for patriotic learning at the ideological level of college students learning. During the implementation of learning activities, about 89% of the students think that it can increase immersion, 80% think that it can be interactive, and most think that VR can make learning emotions high, highly concentrate on learning, and improve learning efficiency. Therefore, adopting relevant methods improves children’s enthusiasm for Ideological and political Education and transforms passive education into active participation.

Scalable modular synthetic data generation for advancing aerial autonomy

One major barrier to advancing aerial autonomy has been collecting large-scale aerial datasets for training machine learning models. Due to costly and time-consuming real-world data collection through deploying drones, there has been an increasing shift towards using synthetic data for training models in drone applications. However, to increase widespread generalization and transferring models to real-world, increasing the diversity of simulation environments to train a model over all the varieties and augmenting the training data, has been proved to be essential. Current synthetic aerial data generation tools either lack data augmentation or rely heavily on manual workload or real samples for configuring and generating diverse realistic simulation scenes for data collection. These dependencies limit scalability of the data generation workflow. Accordingly, there is a major challenge in balancing generalizability and scalability in synthetic data generation. To address these gaps, we introduce a scalable Aerial Synthetic Data Augmentation (ASDA) framework tailored to aerial autonomy applications. ASDA extends a central data collection engine with two scriptable pipelines that automatically perform scene and data augmentations to generate diverse aerial datasets for different training tasks. ASDA improves data generation workflow efficiency by providing a unified prompt-based interface over integrated pipelines for flexible control. The procedural generative approach of our data augmentation is performant and adaptable to different simulation environments, training tasks and data collection needs. We demonstrate the effectiveness of our method in automatically generating diverse datasets and show its potential for downstream performance optimization. Our work contributes to generating enhanced benchmark datasets for training models that can generalize better to real-world situations.Video:youtube.com/watch?v=eKpOh-K-NfQ

The Uncanny Valley Effect and Induced Compensatory Consumption Behavior in Language Conditions

With the gradually widespread use of personification marketing, there is the phenomenon with extensive attention, the uncanny valley effect, that people ' s familiarity for robots decreases sharply with the increase of personification degree, further leading to compensatory consumption behavior based on the self-discrepancy theory and cause the illusion of 'marketing success'. This paper theoretically explores the essence of violation expectations as the uncanny valley effect, and designs a simulation scene experiment in which subjects make consumption decisions in different personification situations in the stimulation of language materials, so as to study the familiarity and compensatory consumption curves under different violation expectations. The results show that under the stimulation of language conditions, there is a familiarity curve similar to the uncanny valley effect and complementary compensatory consumption curve, and the threshold is calculated by fitting curve, within which the familiarity is the lowest and the compensatory consumption is the highest. The results provide empirical evidence to prove the violation of expectation hypothesis, and prove that there is also the uncanny valley effect in the language channel, and the threshold provides guidance for the development of artificial intelligence enterprises.

Visualization Simulation of Branch Fractures Based on Internal Structure Reconstruction

This paper presents a visualization algorithm for wood fracture simulation based on wood science and wood internal structure reconstruction. The algorithm can simulate a reasonable and realistic wood fracture effect. First, the 3D point-cloud data of the bark structure are obtained using a laser scanner, and the cross-section of the branch is obtained by voxelization of the surface mesh model. Then, the outer contour of the cross-section is shrunk inward to reconstruct the annual rings and wood fiber bundles, and reasonable internal structures of branch 3D models are generated. The internal structure consists of a hierarchical model composed of several ring-like annual rings, and each annual ring is divided into a series of continuous fan rings. On the basis of the reconstruction results, the wood fracture surface model generated by the parameter control can be mapped to the irregularly shaped 3D branch model. In this research, the internal structure of branches and the shape of annual rings on the fracture surface of branches are analyzed to provide a reliable fracture model for different branch fractures of trees. In addition, the realistic fractured tree branch model generated by this algorithm can be widely applied in fields such as animation film special effects, game scene simulation, virtual reality scene construction, and mechanical research on broken tree branches.

Application of acquiring region of interest based on the YOLOv5 model in terahertz digital holography.

The imaging process of terahertz in-line digital holography is susceptible to environmental interference, and it is difficult to obtain high-quality images and image segmentation results. The challenge can be better handled by using the region of interest (ROI) condition to improve the image quality of the object region and reduce the interference of peripheral noise. In this paper, for two-dimensional 2.52THz in-line digital holography, a method to add a variety of real backgrounds in the simulation scene is proposed, and through this method, a sufficient amount of close-to-real-scene images are produced to train the YOLOv5 model. The object ROI is extracted by the YOLOv5 model (YOLOv5-ROI). Based on the region, a new, to the best of our knowledge, support-domain-constrained phase retrieval algorithm (YOLOv5-ROI-SPRA) and an image segmentation method combined with the Otsu method (YOLOv5-ROI-Otsu) are proposed, respectively. The better results show that the expected purpose is achieved.

A Step Towards Conditional Autonomy - Robotic Appendectomy

In recent years, Robot-Assisted Minimally Invasive Surgery (RAMIS) has been widely adopted worldwide due to its high precision, improved ergonomics and intuitive control. With the advances in artificial intelligence and surgical robot technologies, it is anticipated that the cognitive load on the surgeons could be relieved with a higher level of autonomy on surgical robots as well as simultaneously improving the precision of surgical robots' operations, minimising the risk of human errors and leading to safer operations. However, much research is still focused on task autonomy due to the complexity of the operations, and the limited clinical data on robotic surgery in some specific scenarios. Therefore, recent research has widely used simulated surgery scenarios to collect data to train artificial intelligence (AI) models. In this paper, we propose a method as a step towards conditional autonomy for robotic appendectomy. To achieve conditional autonomy, demonstration data was collected from human operators performing the appendectomy in a simulated robotic scene. The robotic instrument movements and trajectories for performing the appendectomy were learned from the demonstrated data. Once the surgeon specifies the location of the appendix, the proposed framework can carry out the appendectomy autonomously by using the dynamic motion primitives (DMP) generated from the learned surgical movements. Through the extensive validations in a simulated environment and on the da Vinci Research Kit (dVRK), we have shown that the proposed method can carry out the appendectomy procedure semi-automatically. Based on five metrics, it shows that the framework can reduce the total task path length, completion time and appendix stump length while maintaining a high degree of similarity to the demonstrated trajectory. This work demonstrated the feasibility of conditional autonomy in robotic surgeries.

A real-time dynamic simulation platform for 6R manipulators based on an explicit modeling method

Dynamic simulation is critical for robotic design and control. In this study, a modular simulation platform is developed using a novel explicit dynamic modeling method. Combined with a PD controller and a globally linearized compensator, the dynamic modeling method presents an explicit and canonical form for tree-chain and closed-chain multi-body systems. Paired with the new approach, the platform makes it convenient for modeling, and it performs well in real-time with a high update frequency. The platform also offers a thorough parameter setting and simulation scene presentation interface through the construction of several modules. A simulation experiment using a 6-DOF manipulator demonstrates the platform’s capacity for real-time, stable, and practical simulations, which is also applicable for high-DOF multi-body systems of future uses.

Evaluation of single-stage vision models for pose estimation of surgical instruments.

Multiple applications in open surgical environments may benefit from adoption of markerless computer vision depending on associated speed and accuracy requirements. The current work evaluates vision models for 6-degree of freedom pose estimation of surgical instruments in RGB scenes. Potential use cases are discussed based on observed performance. Convolutional neural nets were developed with simulated training data for 6-degree of freedom pose estimation of a representative surgical instrument in RGB scenes. Trained models were evaluated with simulated and real-world scenes. Real-world scenes were produced by using a robotic manipulator to procedurally generate a wide range of object poses. CNNs trained in simulation transferred to real-world evaluation scenes with a mild decrease in pose accuracy. Model performance was sensitive to input image resolution and orientation prediction format. The model with highest accuracy demonstrated mean in-plane translation error of 13mm and mean long axis orientation error of 5[Formula: see text] in simulated evaluation scenes. Similar errors of 29mm and 8[Formula: see text] were observed in real-world scenes. 6-DoF pose estimators can predict object pose in RGB scenes with real-time inference speed. Observed pose accuracy suggests that applications such as coarse-grained guidance, surgical skill evaluation, or instrument tracking for tray optimization may benefit from markerless pose estimation.

Adaptive fusion of multi-exposure images based on perceptron model

Abstract Multi-exposure image fusion as a technical means to bridge the dynamic range gap between real scenes and image acquisition devices, which makes the fused images better quality and more realistic and vivid simulation of real scenes, has been widely concerned by scholars from various countries. In order to improve the adaptive fusion effect of multi-exposure images, this paper proposes a fusion algorithm based on multilayer perceptron (MLP) based on the perceptron model and verifies the feasibility of the algorithm by the peak signal-to-noise ratio (PSNR), correlation coefficient (PCC), structural similarity (SSMI) and HDR-VDR-2, an evaluation index of HDR image quality. Comparison with other algorithms revealed that the average PSNR of the MLP algorithm improved by 4.43% over the Ma algorithm, 7.88% over the Vanmail algorithm, 10.30% over the FMMR algorithm, 11.19% over the PMF algorithm, and 11.19% over the PMF algorithm. For PCC, the MLP algorithm improves by 20.14%, 17.46%, 2.31%, 11.24%, and 15.36% over the other algorithms in that order. For SSMI, the MLP algorithm improved by 16.99%, 8.96%, 17.17%, 14.41%, and 4.85% over the other algorithms, in that order. For HDR-VDR-2, the MLP algorithm improved by 3.02%, 2.79%, 6.84%, 4.90%, and 6.55% over the other algorithms, in that order. The results show that the MLP algorithm can avoid image artifacts while retaining more details. The MLP-based adaptive fusion method is a step further in the theoretical study of multi-exposure image fusion, which is of great significance for subsequent research and practical application by related technology vendors.

Lightweight and thermal insulation fabric-based composite foam for high-performance electromagnetic interference shielding

With the rapid development of electronic devices, electromagnetic interference (EMI) has posed serious threats to human health. Therefore, wearable materials with high EMI shielding performance are urgently required for protecting people from electromagnetic radiation. Herein, electroless plating and compression molding methods were utilized to fabricate sandwich structured composite foam. Aramid-carbon blend fabrics with Co–Ni coatings were selected as the surfaces, and the porous polyurethane (PU) foam doped with diverse content of carbon nanotubes (CNTs) served as the core lightweight layer. With the CNTs doping content of 3 wt%, a 3D conductive network is precisely constructed in the core foam with a tested 13.82 GPa tensile strength. Through a “reflection-absorption-reflection” triple-loss mechanism, the resulting composite foam possesses a favorable average EMI shielding effectiveness (SE) of 73.9 dB in the X band (8–12 GHz), which was far more than the requirement of 30 dB for common commercial EMI SE. In addition, the porous structure exhibits excellent thermal insulation properties within a wide range of temperature (0–150 °C) while presenting a low thermal conductivity of 0.0695 W/(m·K). Finally, the prepared composites were applied to the simulation scene of electromagnetic protection to explore its reliability and practicability as a wearable electromagnetic wave proof material. With the good combination of mechanical properties, thermal properties and excellent electromagnetic shielding properties, the prepared composites have a great application prospect in electromagnetic protective materials.

Effect of driving distractions on driver mental workload in work zone’s warning area

Traffic accidents are common in advanced warning areas of a maintenance work zone. One of the reasons for the accident is that the complex driving environment of the warning area makes drivers need to change lanes and speed frequently. Another reason is that driving distractions shift the driver's mental workload and make the drivers unable to complete the driving task well. The paper aimed to examine the impact of visual and cognitive distractions on the driver’s mental workload, eye movements, and driving behaviour when driving in advanced warning areas of the maintenance work zones. Thirty-two participants (mean age = 23.97 years) participated in the experiment and completed 3 advanced warning area simulation scenes. The road initial speed limit of simulation scenes was different: 60 km/h, 80 km/h and 100 km/h. The drivers needed to pass three scenes under three types of distraction tasks (no task, visual task and cognitive task). Six indices were used to evaluate the effect of the advanced warning area and the distracting secondary tasks on the driver's mental workload: two related to eye movements, two related to driver behavior, and two related to the electroencephalogram. The TOPSIS analysis was used to solve the problem of multi-evaluation indices comparison, sorting and selection. The results show that the driver has higher vehicle control ability under the distraction-free task condition than in other scenes. Furthermore, the impact of visual distraction on drivers is more significant than that of cognitive distraction, suggesting more visual attention is required for drivers when passing the complex working zone.

UPG: 3D vision-based prediction framework for robotic grasping in multi-object scenes

Robotic grasping has the challenge of accurately extracting the graspable target from a complicated scenario. To address the issue, we propose a 3D vision prediction framework including visual observation and pose estimation. Firstly, we exploit the continuity characteristics in the U-disparity map to identify the isolated objects and occluded objects which can quickly partition the grasping scene and produce valid candidate regions for grasping. Secondly, an end-to-end approach based on PointNet＋＋ is improved to obtain the topmost target if there is a pile of stacked objects. We also provide a robust labeling method for generating the datasets comprising the multi-object scenes. Moreover, a designed evaluation criterion is presented to assist with estimating the 6-DOF (degree of freedom) pose. Our method UPG (U-disparity and PointNet＋＋ grasping) simplifies the segmentation task and makes the training model lightweight in order to apply in practical bin-picking and assembly. To validate the feasibility, UPG is evaluated on simulation and real-world scenes, respectively. The extensive results indicate that UPG can achieve better segmentation accuracy and grasping success rates against other state-of-the-arts.

A Metaverse-Based Teaching Building Evacuation Training System With Deep Reinforcement Learning

With the development of IoT, virtual reality, cloud computing, and digital twin technologies, the advent of metaverse has attracted increasing world attention. Metaverse integrates and applies multiple emerging technologies to cloud education, smart health, digital government, and emergency evacuation. Evacuation systems are of great importance to ensure life safety. Due to panic, people in a building may not be able to make the right judgment to choose an optimal path to leave the building in case of an emergency event such as a fire. As a branch of machine learning, deep reinforcement learning (DRL) can model an evacuation scene, collect real-time information, such as crowd distribution and disaster location, find the optimal escape path with a path-planning algorithm, induce the movement state of the crowd through dynamic guidance signs, and improve the evacuation efficiency. In this article, we apply DRL technology to solve the efficient emergency evacuation problem with the help of metaverse and show a training system built upon metaverse that would enable evacuees to choose the most efficient route and leave the building in the least amount of time. The information collected by various sensors, such as video cameras and smoke detectors, can give a whole picture of the status of the building in a real-time manner. The collected data are processed by cloud servers in which a DRL model is trained to dynamically guide evacuees. Experiments in different simulation scenes demonstrate that the proposed method is superior to the traditional static guidance method in saving evacuation time. It can effectively avoid major crowding along the evacuation route and improve evacuation efficiency.

Effect of mind mapping combined with video-based scene simulation on self-management ability and coping style of patients with autogenous arteriovenous fistula

Effect of mind mapping combined with video-based scene simulation on self-management ability and coping style of patients with autogenous arteriovenous fistula