‘Read-and-do’ response to a digital cognitive aid in simulated cardiac arrest: the Medical Assistance eXpert 2 randomised controlled trial

Abstract

Editor—The use of cognitive aids to guidepractitioners on medical procedure steps is recognised by the medical community.1Société française d’anesthésie réanimation. Aides cognitives en anesthésie réanimation. Date of access 2016. Available from: https://sfar.org/espace-professionel-anesthesiste-reanimateur/outils-professionnels/boite-a-outils/aides-cognitives-en-anesthesie-reanimation/.Google Scholar, 2Staender S, Fairley-Smith A, Bratteboe G, Whitaker D, Mellin-Olsen J, Borshoff D. An Emergency Quick Reference Guide. Brussels, Belgium: European Society of Anaesthesiology.Google Scholar We recently investigated the efficacy of a handheld sequential digital cognitive aid (DCA), the Medical Assistance eXpert (MAX) smartphone application, in simulated crises. We found that MAX improved technical skills in all scenarios except cardiopulmonary arrest.3Lelaidier R. Balança B. Boet S. et al.Use of a hand-held digital cognitive aid in simulated crises: the MAX randomized controlled trial.Br J Anaesth. 2017; 119: 1015-1021Abstract Full Text Full Text PDF PubMed Scopus (30) Google Scholar In civil aviation, the way pilots use cognitive aids depends on the level of urgency. Read-and-do response (follow the procedures step by step) is used for critical situations whereas the do–verify response (first act then check) is used for routine checklists.4Gordon S. Mendenhall P. O’Connor B. Beyond the checklist: what else health care can learn from aviation teamwork and safety. Cornell University Press, Ithaca, NY2013Google Scholar, 5Degani A. Wiener E.L. Cockpit checklists: concepts, design, and use.Hum Factor. 1993; 35: 345-359Crossref Scopus (201) Google Scholar We therefore revised the cardiopulmonary resuscitation (CPR) procedure in the MAX DCA so that only read-and-do responses were possible and investigated whether this improved technical skills and non-technical skills in CPR as compared with usual practice. Anaesthesia residents in their first to fifth postgraduate year and consultant anaesthesiologists were enrolled. The protocol, available at clinicaltrials.gov (NCT03253770), was approved by the Lyon University Hospital Ethics Committee (France). The new version of the DCA MAX gathered protocols extracted from national and international recommendations in the form of a smartphone application.1Société française d’anesthésie réanimation. Aides cognitives en anesthésie réanimation. Date of access 2016. Available from: https://sfar.org/espace-professionel-anesthesiste-reanimateur/outils-professionnels/boite-a-outils/aides-cognitives-en-anesthesie-reanimation/.Google Scholar, 6Monsieurs K.G. Nolan J.P. Bossaert L.L. et al.European resuscitation council guidelines for resuscitation 2015.Resuscitation. 2015; 95: 1-80Abstract Full Text Full Text PDF PubMed Scopus (641) Google Scholar Participants were randomised into three groups: use of DCA (MAX+ group), use of a paper cognitive aid (PCA group) published by the French Society of Anaesthesia and Intensive Care (SFAR), and no cognitive aid (Control group). Group and scenario were randomly assigned. Participants in the MAX+ group and the PCA group were asked to strictly follow the cognitive aid throughout the scenario and were invited to guide the care provided by team members during the high-fidelity simulation session on mannequins. Participants in the control group did not have access to a cognitive aid to avoid bias. Four CPR scenarios were available for the study. Once all simulation sessions had been performed, two observers scored the videos. Technical skills were determined according to recommendations of the SFAR1Société française d’anesthésie réanimation. Aides cognitives en anesthésie réanimation. Date of access 2016. Available from: https://sfar.org/espace-professionel-anesthesiste-reanimateur/outils-professionnels/boite-a-outils/aides-cognitives-en-anesthesie-reanimation/.Google Scholar and the European Resuscitation Council,6Monsieurs K.G. Nolan J.P. Bossaert L.L. et al.European resuscitation council guidelines for resuscitation 2015.Resuscitation. 2015; 95: 1-80Abstract Full Text Full Text PDF PubMed Scopus (641) Google Scholar and were scored using a predetermined grid adapted from a previous publication.3Lelaidier R. Balança B. Boet S. et al.Use of a hand-held digital cognitive aid in simulated crises: the MAX randomized controlled trial.Br J Anaesth. 2017; 119: 1015-1021Abstract Full Text Full Text PDF PubMed Scopus (30) Google Scholar Non-technical skills were assessed using the Ottawa global rating scale.7Kim J. Neilipovitz D. Cardinal P. Chiu M. A comparison of global rating scale and checklist scores in the validation of an evaluation tool to assess performance in the resuscitation of critically ill patients during simulated emergencies (abbreviated as “CRM Simulator Study IB”).Simul Healthc. 2009; 4: 6-16Crossref PubMed Scopus (93) Google Scholar The primary endpoint was participant technical skills in the MAX+ group compared with the control group. The secondary endpoints were technical skills of participants in the MAX+ group compared with the PCA group and the control group, and comparison of participant non-technical skills between the MAX+ group, PCA group, and control group. Group comparisons were performed using Kruskal–Wallis test. In case of a significant difference among the three groups, multiple comparisons were corrected using the Bonferroni–Dunn test; P<0.05 was considered significant. From April to October 2017, 57 videos were analysed in the per-protocol analysis. At randomisation, age and clinical experience were similar. The median (inter-quartile range, IQR) technical skills was significantly better in the MAX+ group (80 [72–84]) than in the control group (55 [50–60]; P<0.0001; –25, 95% confidence interval [CI] [–29 to –17]) and the PCA group (61 [55–65]; P=0.0002; –19, 95% CI [–23 to –13]). The median technical skills in the PCA group was not significantly different to that in the control group (P=0.43; 95% CI [0 to 11]; Fig. 1a). The median (IQR) non-technical skills was significantly better in the MAX+ group (36 [33–39]) than in the control group (30 [27–34]; P=0.004; –6, 95% CI [–9 to –3]); Fig. 1b). The participants' technical and non-technical skills were significantly improved with the use of the MAX DCA during simulated cardiac arrest scenarios compared with the control group. Technical skills were also significantly better than in the PCA group. This positive effect on technical skills was not found with the first version of the MAX DCA. The hypotheses that were put forward were related to the application ergonomics.3Lelaidier R. Balança B. Boet S. et al.Use of a hand-held digital cognitive aid in simulated crises: the MAX randomized controlled trial.Br J Anaesth. 2017; 119: 1015-1021Abstract Full Text Full Text PDF PubMed Scopus (30) Google Scholar A later review of the recordings suggested that the lack of benefit may also be explained by the way the DCA was used (in that previous study, participants were just encouraged to use the app, without further instruction). In the present study, the design of the MAX DCA integrated user feedback from the previous study, and instructions remained sequential in order to avoid information overload.3Lelaidier R. Balança B. Boet S. et al.Use of a hand-held digital cognitive aid in simulated crises: the MAX randomized controlled trial.Br J Anaesth. 2017; 119: 1015-1021Abstract Full Text Full Text PDF PubMed Scopus (30) Google Scholar, 8Marshall S.D. Lost in translation? Comparing the effectiveness of electronic-based and paper-based cognitive aids.Br J Anaesth. 2017; 119: 869-871Abstract Full Text Full Text PDF PubMed Scopus (7) Google Scholar, 9Low D. Clark N. Soar J. et al.A randomised control trial to determine if use of the iResus© application on a smart phone improves the performance of an advanced life support provider in a simulated medical emergency: the iResus© application in ALS performance.Anaesthesia. 2011; 66: 255-262Crossref PubMed Scopus (93) Google Scholar The read-and-do mode was achieved with the DCA because participants were asked to strictly follow the application and because the design was sequential. Once a step was validated, the next step was presented. Concerning the PCA, it was the instruction given by the study team that forced the participant to use the PCA in a read-and-do mode, and not to perform the tasks and then check them (first act then check). The ‘read-and-do’ use of the DCA was mandatory. It allowed for an orderly administration of drugs and shocks that is critical for advanced life support. It is important to note that cognitive aids can be used in ‘read-and-do’ or ‘first act, then check’ modes. The user must make a choice (herein imposed by the study team). The better technical skills in the MAX+ group compared with that in the PCA group could be explained by the sequential design of the application which is more suitable to CPR. Despite the obligation to use the PCA, it seemed that participants felt overwhelmed by the CPR, failed to read the PCA, and put it aside in order to participate in the tasks, increasing the proportion of unchecked items. Regarding non-technical skills, a significant difference between the MAX+ and control groups was observed. Participants had to hold the DCA, verbalise and validate each step, and could no longer physically perform CPR tasks. They thereby focused on assessment of tasks performed by team members. Furthermore, the requirement to use the DCA probably reduced the leader's cognitive load by indicating the steps to perform. The MAX2 trial has several limitations. First, it was not possible to blind video ratings to the DCA use. Second, the results should be interpreted in the context of a simulated environment. Third, participants were mostly residents, and therefore the results could have been different with more experienced physicians. In addition, the findings strongly suggest that the way of using a cognitive aid is as important as its interface or content. Increasing cognitive aid use in healthcare would require an ergonomically designed aid with a high level of acceptance among healthcare professionals. This implies finding which medical care activities could be translated into agreed procedural steps, corresponding to official guidelines. Thereafter, it would be necessary to clarify how the cognitive aid should be used in each different situation and finally to design team training programs. In conclusion, the MAX DCA improved both technical and non-technical skills during CPR simulation sessions. Further studies are required to investigate the effect of the DCA in real-life cardiac arrest. Chief investigator, who initiated this project and led most simulation sessions and debriefings: J-CC Design of the protocol: PD, J-CC, BB Design of the algorithms and evaluation grids: J-CC, PD Participant recruitment, planning the simulation sessions, and took part in the majority of sessions: PD Played confederate roles in simulated crises: TR, AF, ML, JS, LD Assessment of technical and non-technical performance on video recordings: J-CC, PD Statistical analysis of the data: BB Writing of the manuscript: PD Review of MAX algorithms and evaluation grids and different iterations of the present manuscript: TR, J-JL, AF, ML, BB, KT, SB Advice for the conception of the study and comments and corrections to the different iterations of the present manuscript: SB, FL, KT, RL CJC holds the chief executive position in the MEDAE start-up which develops and promotes the digital cognitive aid Medical Assistant eXpert (MAX). MEDAE has been created on November 2, 2018 (i.e. more than 1 year after the end of the herein data collection). The other authors declare that they have no competing interests. The authors thank Jacques Sitruk for playing his own role as an anaesthetist nurse in most scenarios. The authors also extend special thanks to Chris Blakeley and Philip Robinson for their precious help.