Reproducibility of A Model Driven Simulator and Sensitivity to Initial Conditions.

Abstract

BACKGROUND: An essential assumption to the use of model-driven simulators is that the devices generate consistent and reproducible results. While research has looked at improving the fidelity, validity, and clinical application of human patient simulators, few independent studies have rigorously tested the reliability of the embedded models. The METI-HPS was selected for its widespread use and as a standard model-driven simulator. We hypothesized that when the models were run on the workstation alone (disconnected from the tower and mannequin), the simulator should generate identical data each time the same initial state and exact sequence and timing of an action was followed. An “instantaneous” blood loss was selected as a perturbation with easily recognized effects on hemodynamics. METHODS: The METI HPS (Version 6.3,MAC OS 10.2.8) was used in the “disconnected state.” A “test” consisted of five procedurally identical runs: Tests 1,2: Both Standard Man (SM) and Orthostatic Granny (OG) were selected from the patient menu for simulator runs of two minutes. Tests 2–6: SM was subjected to blood loss of 300, 900, 1500, 2400, or 3000 ml at 15 or 16 seconds (consistent within a test) and run for two minutes. Tests 7–11: OG was subjected to blood loss of 150, 750, 1500, 2250, 3000 ml and run for two minutes. The logs of key vital signs for each run were analyzed using EXCEL. RESULTS: Overall, several types of variations were found: a lack of reproducibility of starting baseline values including CVP, partO2 with some markedly atypical curves. high-amplitude “cycling” (e.g. for CVP) – before blood loss the curves of different runs in a test were completely superimposed; after blood loss the curves diverged despite identical amount of blood loss. emergence of chaotic divergent values after an apparent steady state (e.g. heart rate see Figure 1).Figure 1: CVP and heart rate from the OG/750 ml blood loss test.CONCLUSIONS: The lack of reproducibility of baseline values and hemodynamic responses of the simulator can create further challenges in validating technical and behavioral scores of participants in training and exam situations.