Simulation Environment Research Articles

Phacoemulsification to MSICS: A transfer of skills study in a simulated environment.: Transfer of phacoemulsification surgical skills to MSICS.

To investigate if phacoemulsification experience impacts and transfers to the skill acquisition of novices in manual small incision cataract surgery (MSICS) within a simulation environment. Copenhagen Academy for Medical Education and Simulation, Denmark. Prospective controlled experimental study. The study included 29 residents or specialist in ophthalmology with no prior MSICS experience. Participants were required to either have 1) proficiency on the EyeSi surgical simulator (PG; phaco group) or 2) no prior extensive exposure to the EyeSi simulator or any phacoemulsification surgery experience as a primary surgeon (CG; control group). Possible skill transfer was assessed using a test on the HelpMeSee virtual-reality simulator, including nine modules and 30 steps (points). A pass/fail score was determined at 20 points (out of 30 points). Performance scores were analyzed using independent samples t-tests, pass rates using Fisher's exact test, and individual modules using Fisher-Freeman-Halton analysis. The PG performed significantly better in terms of overall performance score (mean SD, PG; 21.8 points 2.3 versus CG; 18.9 points 2.2, p = 0.002, t-value = -3.39) and pass rates (PG; 72 % versus CG; 22%, p = 0.018). The sub-analysis for individual modules demonstrated significant differences exclusively in the capsulorhexis and cortex removal modules. The results suggest a positive inter-procedural transfer of skills between phacoemulsification and MSICS. Pretraining and experience in phacoemulsification demonstrated a significant effect on the performance of MSICS within a simulation environment. These findings suggest that phacoemulsification experience provides an advantage before MSICS training in surgical residency programs.

Advancing urban electric vehicle charging stations: AI-driven day-ahead optimization of pricing and Nudge strategies utilizing multi-agent deep reinforcement learning

Public charging stations (CSs) serve for electric vehicles (EVs) to charge during urban travel. Optimizing the charging time, location distribution, and power of EVs can increase the revenue of charging system operators (CSOs) and provide flexible regulation resources for the power grid. However, the optimization scheduling of CSs involves the charging choices of various users, which are influenced by their autonomy and bounded rationality. To guide users and encourage their participation in the charging schedule, we introduce the Nudge method from behavioral economics. To achieve collaborative optimization of non-economic Nudges and economic incentive strategies applying to multiple charging stations in a complex nonlinear environment involving users, CSO, and the transportation network, we leverage multi-agent deep reinforcement learning (MADRL). We construct a simulation environment using historical and survey data tailored to real users. This environment facilitates the training of agent groups to enhance decision-making processes. Case studies in a metropolis demonstrate that the agent group aimed at revenue improvement yields significant improvements in the CSO's revenue compared to fixed service fees and pricing strategies without Nudges. Moreover, the agent group aimed at power curve tracking achieves a lower average relative error in aligning the total charging power with the desired curve of the power system. This paper integrates sociological methods into the optimization of physical systems by MADRL, providing a new approach for the scheduling of EV charging considering user behavior.

The Prisoner’s Dilemma game as a tool to investigate cooperation and undergraduate education in evolution

Evolutionary theory is based on the conflict that arises when certain heritable variants out-compete others. Given this foundational conflict, a central question for evolutionary biologists concerns the presence of cooperation found throughout all levels of biological organization; from biochemical pathways to complex animal societies. Human behavior is often distinguished from other animal behavior by the presence of acts of cooperative behavior called altruism. Altruism is a cooperative act that penalizes the actor for actions that benefit the recipient. Any other form of cooperation, one that does not penalize the actor, is formally not considered altruistic. How can costly altruistic behavior evolve? This question was the basis for development of a web application tool incorporating a game theory model to investigate conditions affecting cooperative behavior. The game theory model described as Prisoner’s Dilemma incorporates acts of cooperation and non-cooperation (defection). Computer simulations of Prisoner’s Dilemma were developed and online applications were administered for five semesters at Georgia Gwinnett College, using two simulation environments referred to as Random and Non-random. Data collected from simulation runs were used to evaluate the effect of environment on student cooperative behavior and actively engage students in concepts associated with the evolution of cooperation and game theory. Results from student game play suggest group simulation environment played a significant role in the likelihood of observing cooperative behavior. Educational content and attitudinal surveys suggested that PD game play in the undergraduate evolution class at Georgia Gwinnett College improved student knowledge and self-confidence.

A high-level simulator for Network-on-Chip

This study presents a high-level simulator for Network-on-Chip (NoC), designed for many-core architectures, and integrated with the PlatEMO platform. The motivation for developing this tool arose from the need to evaluate the behavior of application mapping algorithms and the routing, both aspects are essential in the implementation and design of NoC architectures. This analysis underscored the importance of having effective NoC simulators as tools that allow for studying and comparing various network technologies while ensuring a controlled simulation environment. During this investigation and evaluation, some simulators, such as Noxim, NoCTweak, and NoCmap, among others, offered configurable parameters for application traffic, options to synthetically define topology and packet traffic patterns. Additionally, they include mapping options that optimize latency and energy consumption, routing algorithms, technological settings such as the CMOS process, and measurement options for evaluating performance metrics such as throughput and power usage. However, while these simulators meet detailed technical demands, they are mostly restricted to analyzing the low-level elements of the architecture, thus hindering quick and easy under- standing for non-specialists. This insight underscored the challenge in developing a tool that balances detailed analysis with a comprehensive learning perspective, considering the specific restrictions of each simulator analyzed. Experiments demonstrated the proposed simulator efficacy in handling algorithms like GA, PSO, and SA variant, and, surprisingly, revealed limitations of the XY algorithm in Mesh topologies, indicating the need for further investigation to confirm these findings. Future work will expand the simulator functionalities, incorporating a broader range of algorithms and performance metrics, to establish it as an indispensable tool for research and development in NoCs.

Autonomous Soft-Tissue Needle Steering Using Reinforcement Learning Guided by Human Input

Soft-tissue needle steering, where a deformable needle is inserted into the tissue to guide its tip to a desired position, is a common minimally invasive surgery (MIS) procedure. The diverse types of needles and complex tissue dynamics limit the use of existing approaches that utilize models of the needle and the tissue for automating the task. In this work, we employ a data-driven approach using deep reinforcement learning (DRL) to achieve autonomous needle steering by viewing it as a multi-goal reinforcement learning problem. Human interventions are incorporated during training to accelerate learning and reduce catastrophic failures. Generative adversarial imitation learning (GAIL) is combined with regular DRL by utilizing a hindsight relabeling scheme for human interventions to encourage the agent to imitate human behavior. To emulate the sim-to-real process, an agent is first trained in a simplistic simulation environment for needle steering and then transferred to a sophisticated one considered as the real world with fine-tuning (sim-to-sim). Experimental results show that with human interventions, the proposed method outperforms the other compared DRL approaches and can achieve good performance with only 2,000 training steps in the complex simulation environment, achieving an average return comparable to that of a 55,000-step agent trained from scratch.

Dynamics and advanced active disturbance rejection control of tethered UAV

Due to its vast potential for applications in agricultural harvesting, the tethered unmanned aerial vehicle (UAV) has garnered considerable interest. However, the anti-disturbance control of this system remains fraught with obstacles, stemming primarily from its inherent nonlinearity, unpredictable airflow disturbances, unmeasurable tether tension, and the changes in the center of gravity during operation. We propose a comprehensive dynamic model for the operational tethered unmanned aerial vehicle with considering the center of gravity shift. Because of obstructions caused by branches and leaves, the tethered UAV can't reach fruits inside the tree. Therefore, two deformation functions of the tethered quadcopter are considered. Additionally, in order to enhance the effectiveness of traditional Active Disturbance Rejection Control, we present an optimized controller combining a phase compensation algorithm with a nonlinear function. The proposed Controller is validated in a high-fidelity simulation environment, considering internal interference, wind field interference and reactive force from produces. Besides, the simulation is specifically developed for picking operation, The results underscore the better observation capabilities of the enhanced controller proposed in this study. Moreover, when deployed in a simulated picking environment, the improved controller exhibits remarkable resistance to the interference and significantly mitigates chattering issues.

Experimental Assessment of a Vision-Based Obstacle Avoidance Strategy for Robot Manipulators: Off-line Trajectory Planning and On-line Motion Control

Human-Robot Interaction is an increasingly important topic in both research and industry fields. Since human safety must be always guaranteed and accidental contact with the operator avoided, it is necessary to investigate real-time obstacle avoidance strategies. The transfer from simulation environments, where algorithms are tested, to the real world is challenging from different points of view, e.g., the continuous tracking of the obstacle and the configuration of different manipulators. In this paper, the authors describe the implementation of a collision avoidance strategy based on the potential field method for off-line trajectory planning and on-line motion control, paired with the Motion Capture system Optitrack PrimeX 22 for obstacle tracking. Several experiments show the performance of the proposed strategy in the case of a fixed and dynamic obstacle, disturbing the robot’s trajectory from multiple directions. Two different avoidance modalities are adapted and tested for both standard and redundant robot manipulators. The results show the possibility of safely implementing the proposed avoidance strategy on real systems.

Learning Curves of Harvester Operators in a Simulator Environment

Simulator training helps provide safe and cost-effective training for operators of modern forestry machines that require high motor skills, constant concentration, and proper planning. The aim of the study was to analyze the learning curves of the trainees in order to determine the period during which most development takes place. In this study, 11 trainees were trained on a John Deere harvester simulator for approximately 15 h each. In each case, a clear learning curve could be identified, despite high inter- and intra-person variability. Effective time showed a steady decrease during training, with a group minimum at the end of training (1.25 min). Crane tip distance per tree dropped rapidly in the first 3–4 h, followed by a more gradual decrease to reach a minimum of 23.8 m. Crane control showed a significant increase from an initial 0.63 to a maximum of 0.8 by the 9th hour of training. A number of crane functions used simultaneously increased more rapidly to almost a maximum value (1.8) already in the 5th hour. The individual curves for each trainee were highly variable, showing a wide range of values and shapes. In conclusion, most personal development occurs during the first phase of simulator training, which typically takes approximately 9–10 h. It is important to consider significant inter-personal variability and tailor the duration of simulator training to individual needs.

[formula omitted] : An adaptive data aggregation algorithm for IoT infrastructure

In IoT infrastructure, high-frequency sensing and subsequent transmission of sensed data to computational facilities can lead to redundant data storage and processing, consuming significant storage and processing capacity. As a result, the IoT infrastructure needs more data transmission cycles, leading to data redundancy and low network up-time due to the drainage of limited battery capacity. Conversely, if the data is communicated at a lower rate, it may cause absolute data delivery to the processing unit, which is useless. As a result, a well-designed data aggregation algorithm is required. This paper proposes the ADA2−IoT, an Adaptive Data Aggregation Algorithm for IoT Infrastructure tailored to optimize parameters such as low data redundancy, limited data communication cycles, and high IoT infrastructure up times. The proposed algorithm consists of two key components: the Route Data Aggregator (RDA) performs aggregation during data transit towards the Edge node or gateway, and the Node Data Aggregator (NDA) performs data aggregation during capturing or sensing data. The algorithm employs metrics like Age of Information (AoI) and data freshness factor during the node and route data aggregation phase to capture and timely deliver data to the Edge node, where this data is processed for informed decision-making. The proposed algorithm was efficiently tested on a simulation and IoT hardware deployment environment. Both simulation and hardware results demonstrate a substantial improvement in QoS parameters, such as a decrease in data redundancy and packet exchanges, leading to considerable energy savings and prolonging the lifespan of IoT infrastructure.

Research on Isomorphic Task Transfer Algorithm Based on Knowledge Distillation in Multi-Agent Collaborative Systems.

In response to the increasing number of agents and changing task scenarios in multi-agent collaborative systems, existing collaborative strategies struggle to effectively adapt to new task scenarios. To address this challenge, this paper proposes a knowledge distillation method combined with a domain separation network (DSN-KD). This method leverages the well-performing policy network from a source task as the teacher model, utilizes a domain-separated neural network structure to correct the teacher model's outputs as supervision, and guides the learning of agents in new tasks. The proposed method does not require the pre-design or training of complex state-action mappings, thereby reducing the cost of transfer. Experimental results in scenarios such as UAV surveillance and UAV cooperative target occupation, robot cooperative box pushing, UAV cooperative target strike, and multi-agent cooperative resource recovery in a particle simulation environment demonstrate that the DSN-KD transfer method effectively enhances the learning speed of new task policies and improves the proximity of the policy model to the theoretically optimal policy in practical tasks.

A practical real-time control strategy for high-performance building HVAC operation concerning potential pandemic outbreaks in post-pandemic era

The lessons learned from the COVID-19 pandemic emphasize the need for building HVAC systems to be adaptable and prepared to adjust their operation for infection risk mitigation for future pandemics. Most guidelines suggest increasing outdoor air fraction in HVAC systems during pandemics. However, this approach is not energy-efficient due to the conditioning of additional outdoor air, and many practical systems face limitations in introducing more outdoor air than normal due to capacity or design constraints. Therefore, there is an urgent need for an alternative solution for HVAC system operation during pandemics that is energy-efficient, can effectively mitigate infection risks and does not require extensive retrofits to existing systems. To meet this need, this study proposes a practical real-time control strategy for high-performance HVAC operation in response to potential pandemic outbreaks. First, a supply air temperature (SAT) reset scheme for infection risk mitigation is proposed based on Mamdani fuzzy inference theory. Then, a coordinated control scheme is proposed to coordinate SAT reset with chilled water temperature adjustment for energy-efficiency enhancement. This strategy does not require system equipment retrofits but only changes in control logic to transition to pandemic operation. The effectiveness of the proposed strategy is validated in a high-fidelity Modelica-based simulation environment under various weather conditions. Results show that, compared to normal operation, the proposed strategy can significantly reduce new infection cases by 32∼46 % without increasing energy use, and may even achieve slight energy savings of 9∼13 %, while maintaining expected occupant comfort.

Interactive Path Editing and Simulation System for Motion Planning and Control of a Collaborative Robot

Robots in hazardous environments demand precise and advanced motion control, making extensive simulations crucial for verifying the safety of motion planning. This paper presents a simulation system that enables interactive path editing, allowing for motion planning in a simulated collaborative robot environment and its real-world application. The system includes a simulation host, a control board, and a robot. Unity 3D on a Windows platform provides the simulation environment, while a virtual Linux environment runs ROS2 for execution. Unity sends edited motion paths to ROS2 using the Unity ROS TCP Connector package. The ROS2 MoveIt framework generates trajectories, which are synchronized back to Unity for simulation and real-world validation. To control the six-axis Indy7 collaborative robot, we used the MIO5272 embedded board as an EtherCAT master. Verified trajectories are sent to the target board, synchronizing the robot with the simulation in position and speed. Data are relayed from the host to the MIO5272 using ROS2 and the Data Distribution Service (DDS) to control the robot via EtherCAT communication. The system enables direct simulation and control of various trajectories for robots in hazardous environments. It represents a major advancement by providing safe and optimized trajectories through efficient motion planning and repeated simulations, offering a clear improvement over traditional time-consuming and error-prone teach pendant methods.

Enhancing Food Safety and Sanitation Education: Developing an Immersive Learning Platform Using Virtual Simulation

ABSTRACT In hospitality management education, diminishing interest in food safety and sanitation education poses challenges compounded by limited awareness of its importance. An immersive learning platform with virtual simulation environments for food safety and sanitation education was developed and integrated into a foodservice management course on a Bachelor of Science in Hospitality Management curriculum. This paper describes the development and implementation of the immersive learning platform, highlighting its positive impact on learning outcomes to date. The paper underscores the practical use of immersive technology in food safety and sanitation education and suggests avenues for further research to optimize student learning and engagement.

Using SUMO for Test Automation and Demonstration of Digitalized Railway Concepts

The Train Dispatcher in the Cloud (ZLiC) is a cloud-based approach to digitalize the German Zugleitbetrieb (comparable to American track warrant control). The ZLiC aims to replace the train dispatcher with speech to text, natural-language understanding, a digital occupancy sheet, a prototype interlocking logic, and text to speech. For train conductors, who operate on the trains, the voice-based communication with the train dispatcher remains unchanged. Because the external interfaces of the ZLiC are either voice-based or graphic, understanding and testing the internal components from an integration level is a challenge. To address these challenges, we first injected the Simulation of Urban Mobility (SUMO) as a simulation environment. Since the ZLiC has been developed model-based, the integration requires minimal modifications. Afterward, we fetch the operation commands (e.g., registering trains, locating trains, drive requests for trains) between the components and send them to a SUMO instance for analyzing and visualizing the train operations in the railway network. Lastly, we insert additional planned trains to add simulated traffic. The reproducible operations enable test automation of the ZLiC while reusing the sophisticated models in SUMO. This prototype shows that SUMO can support the development of digitalized railway operating procedures.

RescuePY: Simulation-Based Rescue Response Impact Assessment

Mobility in metropolitan regions is changing. The distribution of space in cities, the design of transport modes, and the organization of mobility are being re-thought. However, no matter the changes and innovations on the way to a more sustainable future, essential constants must be upheld: In the event of minor, regionally limited emergencies, medical assistance must reach those in need quickly. When dealing with large-scale emergencies, the ability to evacuate the area promptly must be ensured. The impact analysis of mobility innovations on emergency services within urban areas so far has been based purely on empirical observations using existing data. Currently, it is only possible to analyze what-if considerations in a limited way. Nevertheless, due to the increasingly rapid changes in mobility, a comprehensive and interlinked analysis will be necessary. This is the key contribution of rescuePY: rescuePY is a simulation suite based on the mesoscopic and microscopic simulation environment hybridPY. It allows holistic and microscopic transport modeling of rescue infrastructure to quantify the impact of the mobility transition towards higher sustainability on the performance of rescue services. The main features of this software are: Rescue system assessment for strategic, long-term planning Mobility-influence studies for operative, mid-term planning Activity-based urban evacuation modeling The capabilities of rescuePY are demonstrated by two applications: a simulation- based, mesoscopic system analysis of emergency services in Munich compared to real-world data and microscopic modeling of emergency vehicles (EMVs) in different road architectures. Ongoing developments aim to improve the evaluation methodology for the aggregated impact analysis of mobility innovations on rescue response services.

An obstacle avoidance method for robotic arm based on reinforcement learning

Purpose Robotic arms play a crucial role in various industrial operations, such as sorting, assembly, handling and spraying. However, traditional robotic arm control algorithms often struggle to adapt when faced with the challenge of dynamic obstacles. This paper aims to propose a dynamic obstacle avoidance method based on reinforcement learning to address real-time processing of dynamic obstacles. Design/methodology/approach This paper introduces an innovative method that introduces a feature extraction network that integrates gating mechanisms on the basis of traditional reinforcement learning algorithms. Additionally, an adaptive dynamic reward mechanism is designed to optimize the obstacle avoidance strategy. Findings Validation through the CoppeliaSim simulation environment and on-site testing has demonstrated the method's capability to effectively evade randomly moving obstacles, with a significant improvement in the convergence speed compared to traditional algorithms. Originality/value The proposed dynamic obstacle avoidance method based on Reinforcement Learning not only accomplishes the task of dynamic obstacle avoidance efficiently but also offers a distinct advantage in terms of convergence speed. This approach provides a novel solution to the obstacle avoidance methods for robotic arms.

Tape-Shaped, Multiscale, and Continuous-Readable Fiducial Marker for Indoor Navigation and Localization Systems.

The present study proposes a fiducial marker for location systems that uses computer vision. The marker employs a set of tape-shaped markers that facilitate their positioning in the environment, allowing continuous reading to cover the entire perimeter of the environment and making it possible to minimize interruptions in the location service. Because the marker is present throughout the perimeter of the environment, it presents hierarchical coding patterns that allow it to be robust against multiple detection scales. We implemented an application to help the user generate the markers with a floor plan image. We conducted two types of tests, one in a 3D simulation environment and one in a real-life environment with a smartphone. The tests made it possible to measure the performance of the tape-shaped marker with readings at multiple distances compared to ArUco, QRCode, and STag with detections at distances of 10 to 0.5 m. The localization tests in the 3D environment analyzed the time of marker detection during the journey from one room to another in positioning conditions (A) with the markers positioned at the baseboard of the wall, (B) with the markers positioned at camera height, and (C) with the marker positioned on the floor. The localization tests in real conditions allowed us to measure the time of detections in favorable conditions of detections, demonstrating that the tape-shaped-marker-detection algorithm is not yet robust against blurring but is robust against lighting variations, difficult angle displays, and partial occlusions. In both test environments, the marker allowed for detection at multiple scales, confirming its functionality.

Virtual Reality Application for the Safety Improvement of Intralogistics Systems

Immersive technologies from the spectrum of Industry 4.0, such as Virtual Reality (VR), are increasingly used in research and safety analysis in industrial and intralogistics systems, including distribution warehouses and production plants. Safety in intralogistics systems is influenced by design and management processes, human behavior, and device performance. In all these areas, VR can serve as a supportive technology for visualization, testing, and employee training. However, this requires the development of principles for integrating VR into standard procedures for the design, modernization, and analysis of intralogistics and production systems. This article discusses the use of VR to analyze the occupational and functional safety of intralogistics systems. It reviews the literature and VR implementations aimed at examining and improving safety in industrial systems. The article explores the integration of VR into the design and analysis procedures for intralogistics and production systems. The authors present a five-dimensional decision space for assessing the use of VR, including identifying subjects of safety analysis, threats and hazards specific to intralogistics, countermeasures for these threats, factors affecting safety, and mechanisms by which VR can improve safety in intralogistics systems. As a subsequent step, the authors discuss using universal simulation environments that support VR technology to study and enhance safety in intralogistics systems, providing a framework example based on the FlexSim (2023 update 2) environment. Finally, this article addresses the threats and limitations of VR technology, along with the challenges and future prospects of VR in the context of Industry 4.0. The article concludes that VR can be an essential tool for increasing safety in the future, albeit with some reservations about certain features of this technology.

Design and implementation of an Autonomous Systems Training Environment framework for control algorithm evaluation in autonomous plant operation

The shortage of trained plant operators who can control complex systems in the process and energy industry is leading to an increasing need for more autonomy of such plants. In future, such systems will be controlled by autonomous agents executing intelligent control algorithms. Due to the critical nature of such plants and processes, new control strategies should not be introduced untested. In order to test suitable algorithms, robust and comprehensive simulation environments are required. In this paper, a framework, called Autonomous Systems Training Environment, is proposed for the evaluation of control algorithms in autonomous plant operations. Furthermore, an exemplary use case for a process engineering system is implemented in Matlab/Simulink, taking into account different levels of control (e.g., regulatory control, operator control, safety control). Faults, which represent a major challenge in the autonomous control, are also considered.

Real-Time Energy Management Strategy for Fuel Cell Vehicles Based on DP and Rule Extraction

Energy management strategy (EMS), as a core technology in fuel cell vehicles (FCVs), profoundly influences the lifespan of fuel cells and the economy of the vehicle. Aiming at the problem of the EMS of FCVs based on a global optimization algorithm not being applicable in real-time, a rule extraction-based EMS is proposed for fuel cell commercial vehicles. Based on the results of the dynamic programming (DP) algorithm in the CLTC-C cycle, the deep learning approach is employed to extract output power rules for fuel cell, leading to the establishment of a rule library. Using this library, a real-time applicable rule-based EMS is designed. The simulated driving platform is built in a CARLA, SUMO, and MATLAB/Simulink joint simulation environment. Simulation results indicate that the proposed strategy yields savings ranging from 3.64% to 8.96% in total costs when compared to the state machine-based strategy.