Simulation Learning Research Articles

Learning a Tracking Controller for Rolling μbots.

Micron-scale robots (μbots) have recently shown great promise for emerging medical applications. Accurate control of μbots, while critical to their successful deployment, is challenging. In this work, we consider the problem of tracking a reference trajectory using a μbot in the presence of disturbances and uncertainty. The disturbances primarily come from Brownian motion and other environmental phenomena, while the uncertainty originates from errors in the model parameters. We model the μbot as an uncertain unicycle that is controlled by a global magnetic field. To compensate for disturbances and uncertainties, we develop a nonlinear mismatch controller. We define the model mismatch error as the difference between our model's predicted velocity and the actual velocity of the μbot. We employ a Gaussian Process to learn the model mismatch error as a function of the applied control input. Then we use a least-squares minimization to select a control action that minimizes the difference between the actual velocity of the μbot and a reference velocity. We demonstrate the online performance of our joint learning and control algorithm in simulation, where our approach accurately learns the model mismatch and improves tracking performance. We also validate our approach in an experiment and show that certain error metrics are reduced by up to 40%.

Hydraulic conveying characteristics of particles in bend based on numerical simulation and explainable stacking machine learning model

As a hydraulic lifting pipeline structure widely used in deep-sea oil, gas transportation, and sediment dredging projects, the pipeline configuration is related to the improvement of transportation efficiency and pipeline safety. Particularly, the bending section consumes the most energy and withstands severe erosion. Understanding and predicting the conveying characteristics of two-phase flow in bends is therefore crucial. In this study, CFD-DEM (computational fluid dynamics-discrete element method) simulation method is employed to calculate various cases, considering five parameters: pipeline bending radius and angle, conveying velocity, particle diameter, and concentration, to explore the influence of these parameters on pressure drop and erosion rate of pipeline and result in a data set of hundreds of cases. Based on this data set, seven machine learning models are trained to predict pressure drop and erosion rate, respectively. To enhance model accuracy, the stacking method in ensemble learning is employed to combine multiple models with good performance. Additionally, the Optuna and SHAP (SHapley Additive exPlanation) methods are utilized to optimize hyperparameters and explain the degree to which parameters impact the predictions. The result demonstrates that pressure drop is almost unaffected by bending radius, while erosion rate initially decreases and then increases with bending angle, and both increase with other parameters. Among the evaluated models, artificial neural network, XGBoost, and random forest all demonstrate high prediction accuracy. The stacking model further improves the accuracy, with mean absolute error improving by 21.7% and 32.2%, and the SHAP method demonstrated good interpretability, which is basically consistent with CFD-DEM results.

Application of machine learning technique for dynamic analysis of confined geomaterial subjected to vibratory load

Computer programming-based numerical programs are firmly established in geotechnical engineering, with rapid growth of finite element modeling and machine learning techniques gaining much attention both in practice and academia. This study is intended to expedite the dissemination of advanced computer applications in terms of finite element simulation and machine learning models by investigating the dynamic response of geomaterials subjected to vibratory loads. Several trial models were built to perform the experimental investigations with a vibratory shaker, signal generator, several accelerometers, a data collection system, and other ancillary devices. The implicit integration techniques in commercialized software were adopted for numerical simulations. After data collection from numerical simulation, models were chosen, trained, and assessed to produce predictions that were then used in this study. Several technologies, including the ensemble boosted tree, squared exponential Gaussian Process Regression (GPR), Matern 5/2 GPR, exponential GPR, and decision tree architectures (fine and medium), were used to forecast the displacement of confined geomaterial. The displacement-depth ratio was found rising to 80% in the frequency range of 5 to 25 Hz, suggesting a considerable change in the behavior of the geomaterial. The Matern 5/2 GPR model showed better accuracy with an R2 value of 0.99, indicating an outstanding predictive ability. The Matern 5/2 GPR and boosted tree models could help better understand the links between displacement and its distribution along the direction of load application. The outcomes of this study based on computer-aided finite element programs can be effectively implemented in machine learning to develop computer programs. In conclusion, the computational machine learning models adopted in this study offer a new insight for uncovering hidden intrinsic laws and creating new knowledge for geotechnical researchers and practitioners.

ACERAC: Efficient Reinforcement Learning in Fine Time Discretization.

One of the main goals of reinforcement learning (RL) is to provide a way for physical machines to learn optimal behavior instead of being programmed. However, effective control of the machines usually requires fine time discretization. The most common RL methods apply independent random elements to each action, which is not suitable in that setting. It is not feasible because it causes the controlled system to jerk and does not ensure sufficient exploration since a single action is not long enough to create a significant experience that could be translated into policy improvement. In our view, these are the main obstacles that prevent the application of RL in contemporary control systems. To address these pitfalls, in this article, we introduce an RL framework and adequate analytical tools for actions that may be stochastically dependent in subsequent time instances. We also introduce an RL algorithm that approximately optimizes a policy that produces such actions. It applies experience replay (ER) to adjust the likelihood of sequences of previous actions to optimize expected n -step returns that the policy yields. The efficiency of this algorithm is verified against four other RL methods [continuous deep advantage updating (CDAU), proximal policy optimization (PPO), soft actor-critic (SAC), and actor-critic with ER (ACER)] in four simulated learning control problems (Ant, HalfCheetah, Hopper, and Walker2D) in diverse time discretization. The algorithm introduced here outperforms the competitors in most cases considered.

Application of self-learning enhancing Huff and Puff gas injection optimization in shale reservoirs through sequence-based proxy reservoir simulation and reinforcement learning

This study introduces a novel approach to optimize Huff and Puff (H-n-P) gas injection in complex shale reservoirs by integrating sequence-based Proxy reservoir simulation and reinforcement learning (RL). The aim is to drastically reduce simulation time while enhancing H-n-P gas injection project decisions. The method involves three main steps: (1) validating simulation outcomes using the GFREE-SIM in-house multi-porosity simulator against actual data; (2) training and validating a Proxy sequence-based machine learning (ML) model with simulation data; (3) implementing a customized RL model that interacts with the environment (e.g., sequence-based Proxy reservoir simulation) to learn optimal actions (e.g., injection/production periods, gas injection rate) based on cumulative rewards (net present value), discussed in the subsequent article.The Proxy reservoir simulator serves as a rapid numerical tool, generating multiple scenarios in minutes. Despite minor accuracy trade-offs, this innovative method accelerates simulations up to 20,000X. Operating as the environment for RL interactions, the Proxy model's reliability and speed play a crucial role.To address the complexities of cyclic enhanced oil recovery, the study employs a self-learning Sequential Data-Informed Degenerate Deep Reinforcement Learning Optimization (SDI-DgRL) framework. This framework leverages RL techniques to identify optimal gas injection strategies. An example is presented where the SDI-DgRL framework investigates injection/production periods and gas injection rate, achieving robust optimization in fewer than 1,000 scenarios. Results indicate that hyperparameter tuning significantly enhances the stability and performance of RL models. After several experiments, it is recommended that injecting 2.0–2.2 MMscfd of gas over 2–2.2 months, followed by production periods of 2.5–3 months, yields the highest net present value (NPV).

Simulation training in medicine: past, present and future

The research was carried out with financial support of Perm Scientific and Educational Center "Rational Subsoil Use" (2023), the project "New materials and technologies for medicine".
 The article provides a review of literature on the use of simulation learning in medical education. The review used articles on simulation in training at the specialty, residency and continuing postgraduate medical education levels. The research has demonstrated that simulation training leads to improved performance among future physicians. Simulation in the learning process allows improving teamwork and communication. With the introduction of medical accreditation and opening of simulation centers at medical universities throughout the country, simulation training is a basic principle of medical education in the Russian Federation. As medical simulation becomes more common, there is a need to develop new methods providing increase in effectiveness of training specialists including medical staff of the highest qualification.

Accelerated Pure Shift NMR Spectroscopy with Deep Learning.

Pure shift nuclear magnetic resonance (NMR) spectroscopy presents a promising solution to provide sufficient spectral resolution and has been increasingly applied in various branches of chemistry, but the optimal resolution is generally accompanied by long experimental times. We present a proof of concept of deep learning for fast, high-quality, and reliable pure shift NMR reconstruction. The deep learning (DL) protocol allows one to eliminate undersampling artifacts, distinguish peaks with close chemical shifts, and reconstruct high-resolution pure shift NMR spectroscopy along with accelerated acquisition. More meaningfully, the lightweight neural network delivers satisfactory reconstruction performance on personal computers by several hundred simulated data learning, which somewhat lifts the prohibiting demand for a large volume of real training samples and advanced computing hardware generally required in DL projects. Additionally, an M-to-S strategy applicable to common DL cases is further exploited to boost the network generalization capability. As a result, this study takes a meaningful step toward deep learning protocols for broad chemical applications.

Adolescent Simulation in a Health Assessment Course for Advanced Practice Nursing Students.

Advanced practice registered nurse (APRN) students face challenges transitioning to practice. They benefit greatly from simulated learning experiences that build confidence and increase competence in the skills needed to perform in an APRN role. The purpose of this quality improvement project was to evaluate the effectiveness of a simulated adolescent well visit in an advanced health assessment course. Using a pretest/posttest design, qualitative and quantitative data were collected from participants. Results included high learner satisfaction, an increase in both self-reported competence and confidence, and an appreciation for the "safe space" of learning.

A Reflection On: Simulated Learning in the Clinical Education of Novice Physiotherapy Students

A Reflection On: Simulated Learning in the Clinical Education of Novice Physiotherapy Students

Healthcare Students’ Anxiety, Cognitive Load, and Trait-mindfulness during Interprofessional obstetric Simulation Training

Background Simulation anxiety can hinder healthcare students' learning. Trait-mindfulness may help reduce learners' anxiety and improve their cognitive load during simulation training. However, the relationship between healthcare students' anxiety, trait mindfulness, and cognitive load has not been studied together during undergraduate interprofessional obstetric emergency simulation training. Methods This observational correlational research examined the relationship between healthcare students' anxiety, trait-mindfulness, and cognitive load. Furthermore, it examined whether healthcare students' trait-mindfulness can moderate the effect of anxiety on cognitive load during undergraduate interprofessional obstetric emergency simulation training. Forty-six undergraduate medical and nursing students were enrolled in this study. Each participant (in a team of 3 to 4) completed one postpartum hemorrhage scenario. The French versions of the State-Trait Anxiety Inventory (STAI-Y; Gauthier & Bouchard, 1993), the Mindfulness Attention Awareness Scale (Germann et al., 2009), and Paas’s cognitive load scale (Paas, 1992) were used to measure anxiety, trait-mindfulness, and cognitive load, respectively. Results The study results show a positive correlation between healthcare students’ state-anxiety and cognitive load. Additionally, healthcare students’ mindfulness moderated the effect of state-anxiety on cognitive load during simulation training. Discussion As a personality trait, healthcare students’ mindfulness is associated with reduced simulation anxiety and improved cognitive load during obstetric emergency simulation training. Besides the learning tasks and simulation environment characteristics, simulation instructors should consider healthcare students’ anxiety and mindfulness to optimize their cognitive load during simulation training. Conclusion The study demonstrated that healthcare students’ trait mindfulness mitigates the magnitude of simulation anxiety on cognitive load during emergency simulation training. It suggests further examining healthcare students’ personality traits to detect potential interactions of their traits with the simulation learning process.

Perception level among clinical officers’ students on the effectiveness of simulation models in the Department of Clinical Medicine

Introduction Clinical simulation learning as an educational tool is recommended in the clinical officers’ training. Upon entry into the workforce, clinicians are expected to respond to a range of clinical situations they may not have experienced during their program.Methodology. A cross-sectional survey was conducted at Two medical training colleges in Kakamega County, Kenya. This design also enabled testing of causeand-effect relationships of the variables under study. The study employed a systematic way of gathering information. The information gathered described simulation in clinical teaching, specifically a case study of medical training colleges in Kakamega County Kenya.Results. The results show that 215 (71.2%) of the respondents agreed that assessment in the skills laboratory improved the outcome of the early clerkship, 77 (25.5%) disagreed while 10 (3.3%) of the respondents were neutral. The mean score was 3.14 with a standard deviation of 1.14 which shows that most respondents were in agreement that assessment in skills laboratory improves the outcome of the performance and skills of students. The item mean was below the composite mean of 3.36 indicating a negative influence on the composite mean. The standard deviation for the item was above the composite standard deviation of 0.45 indicating a wider spread in response for the item than the variable.Conclusion. OSCE is a competency-based tool widely used in clinical teaching.

Challenge, integration, and change: ChatGPT and future anatomical education

ABSTRACT With the vigorous development of ChatGPT and its application in the field of education, a new era of the collaborative development of human and artificial intelligence and the symbiosis of education has come. Integrating artificial intelligence (AI) into medical education has the potential to revolutionize it. Large language models, such as ChatGPT, can be used as virtual teaching aids to provide students with individualized and immediate medical knowledge, and conduct interactive simulation learning and detection. In this paper, we discuss the application of ChatGPT in anatomy teaching and its various application levels based on our own teaching experiences, and discuss the advantages and disadvantages of ChatGPT in anatomy teaching. ChatGPT increases student engagement and strengthens students’ ability to learn independently. At the same time, ChatGPT faces many challenges and limitations in medical education. Medical educators must keep pace with the rapid changes in technology, taking into account ChatGPT’s impact on curriculum design, assessment strategies and teaching methods. Discussing the application of ChatGPT in medical education, especially anatomy teaching, is helpful to the effective integration and application of artificial intelligence tools in medical education.

Prediction method for response characteristics parameters of isolated-span overhead lines after ice-shedding based on finite element simulation and machine learning

In response to the problem of insufficient research on the prediction for response characteristics parameters of isolated-span transmission lines under various ice-shedding modes, study on the prediction for ice-shedding response parameters of isolated-span overhead lines is conducted in this paper based on finite element simulation and machine learning. Firstly, the finite element simulation model of isolated-span overhead line is established and verified. Seven model input parameters are determined by considering 12 typical ice-shedding modes and ice-shedding labels are introduced. Six key output parameters reflecting the electrical and mechanical characteristics of the ice-shedding transmission line are obtained by feature extraction. Secondly, the output parameters of the ice-shedding transmission line under different input parameter conditions are obtained by using the finite element simulation method. A total of 1400 sets sample data are constructed. Then, four typical regression prediction models of BP (Back Propagation Algorithm), RBF (Radial Basis Function Algorithm), SVM (Support Vector Machine Algorithm) and RF (Random Forest Algorithm) are established to predict the key output parameters of ice-shedding transmission lines. Finally, the optimal prediction model for ice-shedding response characteristics parameters of isolated-span transmission lines is selected and the application effect is analyzed by actual ice-shedding case. It is found that the RF prediction model is the optimal prediction model among the four regression prediction models. The prediction method for ice-shedding response characteristics parameters of isolated-span transmission line proposed in this paper can accurately predict the response characteristics parameters of ice-shedding transmission lines which can provide some references for the safe operation of isolated-span transmission lines in heavy ice areas.

Bridging the Gap Between Practicing Nurse Practitioners and Adult-Gerontology Acute Care Nurse Practitioner Post-Master’s Curriculum

The job growth for nurse practitioners (NPs) will exponentially grow by 2026 as registered nurses transition into this advanced practice role. Education is aligning with scope of practice. Although practice readiness has been looked at in the literature for new graduate nurses and new graduate NPs, the literature is lacking when describing practice readiness for NPs currently in practice who are back for a post-master’s certificate. This report describes the concept of practice readiness and how one school successfully prepares its students for clinical practice using a multimodal approach of didactic, simulation, and case-based learning.

An innovative gas management methodology based on PSA for efficient gas allocation and utilization in hybrid hydrogen network: Integrating process simulation, modeling, and machine learning

Pressure swing adsorption(PSA) based gas separation technology is widely applied in chemical industries, especially in hydrogen purification. Its complex mechanism and non-steady state feature restrict the separation performance adjustment and application on efficient gas utilization. Process simulation, machine learning and modelling combined innovating gas management methodology is proposed in this paper to perform accurate gas separation for natural gas based syngas mixture. VB script program is employed to drive Aspen adsorption simulation, and then classical and regression tree(CART), polynomial regression(PNR) and surrogate model are comprehensively used to perform machine learning. Consequently, the coefficient matrix is obtained to quantify the separation performance. This method holds the potential for broader application in refineries and SPCs hybrid hydrogen network, facilitating the comprehensive utilization of hydrogen containing gases. Case study results show this methodology can conserve natural gas and gas productivity by 21.3 %, as well as carbon emission by 8211.6 Nm3/h. This work highlights the promising application of digital twin technology within the chemical industry, showcasing how process simulation, machine learning, and modeling can collectively revolutionize gas management and resource conservation.

Feature detection in guided wave ultrasound measurements using simulated spectrograms and generative machine learning

Guided wave ultrasound field measurements capture reflections from aluminothermic welds in the rail track at unknown distances from a transducer. The measured guided wave signals are complex and challenging to interpret due to multiple dispersive modes propagating in the rail, changing environmental conditions, and noise. Data-driven machine learning techniques have been applied to complex signal-processing problems and have shown significant potential in learning and resolving complex problems. This study aims to understand the implications of using various simulated data sets for application within an appropriate learning framework to capture the underlying features of the field measurements and maximise the performance of these techniques. The use of simulated spectrograms, including variations of signal attributes (attenuation, positions of welds, noise, and mode reflection coefficients) generally observed in experimental measurements, allows the reflections of individual modes to be isolated or combined for training. This allows us to present the training data in three distinct forms. The first dataset has the highest mode reflection information density per sample and consists of simulated spectrogram data with multiple reflections of modes from multiple welds, like experimentally obtained spectrograms. The second training dataset consists of spectrograms with multiple mode reflections; however, only for a single weld reflection per spectrogram. The third training set contains a reflection of a single mode from a single weld in each spectrogram. The data-driven models applied are principal component autoregression and variational auto-encoders. The reconstruction error and latent space interpretability were considered as metrics for the algorithms’ ability to learn using test sets, i.e., unseen data. The results show that datasets with sufficient feature variation and higher mode reflection information density better construct the test set of simulated and experimental spectrograms. However, training using the third dataset shows more interpretable latent variables for an artificial growing defect attached to the rail. Furthermore, data-driven machine learning methods trained using simulated spectrogram data are useful for reconstructing and learning features from experimental measurements, provided that the training data have representative mode feature variation and noise.

Intelligent strategic bidding in competitive electricity markets using multi-agent simulation and deep reinforcement learning

Aiming at the lack of comprehension of agents in Multi-Agent Simulation (MAS) based on classic Reinforcement Learning algorithms of competitive electricity markets, an intelligent strategic bidding method using Deep Reinforcement Learning (DRL) and MAS is proposed in this paper, which not only can provide more intelligent strategies for market participants to maximize their profits, but can enhance the performance of simulation models dealing with high-dimensional continuous data in electricity markets. Firstly, a theoretical framework of intelligent strategic bidding in competitive electricity markets based on MAS and DRL is proposed, and the process of intelligent bidding in electricity markets based on MAS and DRL is described. Then, three MAS models of intelligent strategic bidding are built based on three classic DRL algorithms, including Deep Q-Network (DQN), Double Deep Q-Network (DDQN), and Asynchronous n-step Q-learning (Async n-step QL), and three algorithms’ convergence speed, computational efficiency, and response sensitivity are compared and analyzed. Finally, a novel Improved Async n-step QL (IAsync n-step QL) algorithm is proposed, the MAS model based on the IAsync n-step QL algorithm for intelligent strategic bidding is established. Simulation results show that the model using the novel DRL algorithm is more profitable and responsive than the classic DRL algorithms.

Unveiling the impact of axial ligands on Fe-N-C complexes through DFT simulation and machine learning analysis

Single-atom catalysts (SACs), featuring isolated metal atoms embedded in graphitic carbon materials, have attracted considerable research interest due to their cost-effectiveness, high catalytic activity, and customizable functionality across various catalytic reactions. Among SACs, the Fe-N4-C class has garnered significant attention. Tailoring the properties of Fe-N4 sites through localized chemical modifications stands as a key strategy for catalyst engineering. Recent experimental and computational investigations have underscored the distinct influence of axial ligands on Fe in modulating the oxygen reduction reaction (ORR) activity. However, the precise quantitative structure-property relationship between ligands and the catalytic properties of the Fe center remains elusive. In this study, we combined the density functional theory (DFT) simulations and machine learning (ML) models to unravel the relationship between the ligand properties and the oxo binding energy. This energy pertains to the binding of an oxygen atom to the Fe center, a fundamental step in ORR. Through the design of 33 ligands and 5 molecular complexes that accommodate the Fe-N4 moiety, we screened a total of 278 oxo binding energies across an array of ligands and host complexes. Harnessing the power of ML models, we achieved an accurate prediction of these oxo binding energies using features collected from DFT simulations. Notably, the predominant features contributing to the oxo binding energy prediction primarily derived from complexes with attached ligands, rather than isolated ligand properties. We formulated an approach that leverages these critical features and identified the isolated ligand properties capable of effectively predicting these features. This methodology can potentially be applied to investigate other ORR intermediates and a comprehensive understanding of the ligand effect for the ORR activity in SACs can be achieved.

The effect of escape room clinical evaluation method on satisfaction, learning, and preparedness to practice as interns of nursing students: A quasi-experimental quantitative study.

Clinical evaluation utilizing the escape room game is recognized as a novel method for assessing the team-oriented performance of learners. It is a tool for evaluating teamwork skills in clinical settings, which can boost student motivation and learning. This study aimed to investigate the effect of clinical evaluation through escape rooms and feedback provision through the PEARLS (promoting excellence and reflective learning in simulation) approach on pre-internship nursing students' satisfaction, learning, and preparedness to practice as interns. The current research is a quasi-experimental quantitative study conducted with a census sample of 42 sixth-semester undergraduate nursing students in 2022. The escape room method was utilized to evaluate entry preparedness into the clinical field. Reliable and valid researcher-made questionnaires were administered to assess the impact of the intervention on learning, satisfaction, and preparedness. The data were analyzed in SPSS version 26 using descriptive and inferential tests. The significance level was considered to be less than 0.05. Twenty-six males and 16 females constituted the 42 participants (mean age: 23.46 years). The clinical evaluation method of the escape room game was deemed satisfactory or highly satisfactory by 80% of students. From the perspective of 72% of students, escape rooms were definitely or highly effective in shaping their preparedness to enter the clinical field. Comparing the students' mean learning scores (self-assessment of clinical skills) before and after the test revealed that their post-test scores (55/16 ± 13/33) were significantly higher than their pre-test scores (45/58 ± 16/58) (P < 0.001). It appears that using the escape room evaluation method in conjunction with other student evaluation methods has helped improve students' interpersonal communication, problem-solving skills, critical thinking, and teamwork. The experience of working in a group not only improves these skills and is enjoyable to students but can also enhance their learning. Clinical education is dependent on teamwork. The escape room test can be viewed as a valuable tool for encouraging students to collaborate as a team. Therefore, it is suggested that students in all medical education groups be evaluated clinically using this test.

DeepTraderX: Challenging Conventional Trading Strategies with Deep Learning in Multi-Threaded Market Simulations

DeepTraderX: Challenging Conventional Trading Strategies with Deep Learning in Multi-Threaded Market Simulations