Virtual Standardized Patients Research Articles

Exploration of the impact of baseline clinician learner characteristics on motivational interviewing skill improvement following training with a virtual standardized patient.

Exploration of the impact of baseline clinician learner characteristics on motivational interviewing skill improvement following training with a virtual standardized patient.

Suicide prevention safety planning in the US Department of defense: Qualitative assessment of training.

Suicide rates in the military are a significant public health concern. The suicide prevention safety planning intervention is a brief and effective intervention in which a provider and patient work collaboratively to recognize warning signs of a suicidal crisis and create a defined list of coping strategies and supports for use during future crises (Stanley & Brown, 2012). Implementation of safety planning has been supported by the Department of Defense (DoD), yet readily available training and continuing education for healthcare providers in this intervention is limited and passive in nature. Existing safety planning training experiences and needs of DoD behavioral health providers are unknown. The present study is an exploratory qualitative assessment of current safety planning in DoD to inform the design and development of an interactive virtual standardized training patient. Ten military behavioral health providers completed semi-structured interviews. Thematic content analysis was conducted, and three themes are described in this paper: variety of training received, barriers to receiving suicide safety planning training, and desires for future training. Lack of protected time for training and a lack of access to training resources were identified as key barriers. Rich data obtained can help inform the key design features and relevance of new safety planning intervention training approaches.

Analysis of virtual standardized patients for assessing clinical fundamental skills of medical students: a prospective study

BackgroundHistory-taking is an essential clinical competency for qualified doctors. The limitations of the standardized patient (SP) in taking history can be addressed by the virtual standardized patient (VSP). This paper investigates the accuracy of virtual standardized patient simulators and evaluates the applicability of the improved system’s accuracy for diagnostic teaching support and performance assessment.MethodsData from the application of VSP to medical residents and students were gathered for this prospective study. In a human–machine collaboration mode, students completed exams involving taking SP histories while VSP provided real-time scoring. Every participant had VSP and SP scores. Lastly, using the voice and text records as a guide, the technicians will adjust the system’s intention recognition accuracy and speech recognition accuracy.ResultsThe research revealed significant differences in scoring across several iterations of VSP and SP (p < 0.001). Across various clinical cases, there were differences in application accuracy for different versions of VSP (p < 0.001). Among training groups, the diarrhea case showed significant differences in speech recognition accuracy (Z = -2.719, p = 0.007) and intent recognition accuracy (Z = -2.406, p = 0.016). Scoring and intent recognition accuracy improved significantly after system upgrades.ConclusionVSP has a comprehensive and detailed scoring system and demonstrates good scoring accuracy, which can be a valuable tool for history-taking training.

Evaluating a Virtual Flipped Classroom Approach to Nurse Practitioner Telehealth Competency Development.

Evidence supports the use of multi-modal approaches to develop nurse practitioner (NP) students' telehealth competency. A virtual flipped classroom approach that included eLearning, interactive webinars, and virtual standardized patient (SP) simulations was implemented and evaluated to teach NP students to use telehealth and other connected health technologies to improve care for underserved populations. Analysis of data from multiple-choice quizzes, surveys, and SP evaluations indicated students achieved high levels of knowledge, met the learning objectives, demonstrated above-average competency during telehealth simulations that improved over time, and were highly satisfied with the learning experiences. The educational benefits of a flipped classroom can be realized within a fully online learning experience. Future research should examine the effects of repeated simulation opportunities on telehealth competency development. [J Nurs Educ. 2024;63(8):546-551.].

Simulation-Based Telemedicine Training Program for Residents in Physical Medicine and Rehabilitation: A Virtual Standardized Patient Activity

Background: The COVID-19 pandemic has ushered in a new era of healthcare delivery, emphasizing the crucial role of telemedicine. This study aimed to address the training needs of physical medicine and rehabilitation (PM&amp;R) residents in telemedicine through an Interactive Virtual Telemedicine training program utilizing standardized patient methodology. Methods: A tailored virtual standardized patient simulation module was developed for PM&amp;R residents. Prior to the training, residents completed pre-training questionnaires. The training consisted of (1) a 2-hour classroom didactic session covering telemedicine strategies for conducting physical examinations and triaging, as well as anticipated challenges. (2) a 2-hour zoom telemedicine simulation training exposed residents to three distinct patient scenarios (sickle cell crisis, acute low back pain, and neurorehabilitation). (3) a three-stage debriefing tool was employed for post-training reflection. Results: Eight PM&amp;R residents, predominantly males, with varying prior exposure to telemedicine, participated in the study. Post-training evaluations indicated that residents overwhelmingly found the simulation-based telemedicine training valuable and worth the didactic time invested. Data analysis demonstrated a significant increase in residents’ knowledge and confidence in telemedicine skills. Discussion: The study’s findings suggest that the telemedicine training program effectively enhanced residents’ understanding and confidence in telemedicine skills. Future research endeavors could involve a larger and more diverse group of residents to further assess training effectiveness and gather a wider array of perspectives on telemedicine. Expanding the use of standardized patients in telemedicine simulation training holds promise for advancing medical education in this evolving healthcare landscape.

The effectiveness of virtual reality training on knowledge, skills and attitudes of health care professionals and students in assessing and treating mental health disorders: a systematic review

Background Virtual reality (VR) training can enhance health professionals’ learning. However, there are ambiguous findings on the effectiveness of VR as an educational tool in mental health. We therefore reviewed the existing literature on the effectiveness of VR training on health professionals’ knowledge, skills, and attitudes in assessing and treating patients with mental health disorders.MethodsWe searched MEDLINE, PsycINFO (via Ovid), the Cochrane Library, ERIC, CINAHL (on EBSCOhost), Web of Science Core Collection, and the Scopus database for studies published from January 1985 to July 2023. We included all studies evaluating the effect of VR training interventions on attitudes, knowledge, and skills pertinent to the assessment and treatment of mental health disorders and published in English or Scandinavian languages. The quality of the evidence in randomized controlled trials was assessed with the Cochrane Risk of Bias Tool 2.0. For non-randomized studies, we assessed the quality of the studies with the ROBINS-I tool.ResultsOf 4170 unique records identified, eight studies were eligible. The four randomized controlled trials were assessed as having some concern or a high risk of overall bias. The four non-randomized studies were assessed as having a moderate to serious overall risk of bias. Of the eight included studies, four used a virtual standardized patient design to simulate training situations, two studies used interactive patient scenario training designs, while two studies used a virtual patient game design. The results suggest that VR training interventions can promote knowledge and skills acquisition.ConclusionsThe findings indicate that VR interventions can effectively train health care personnel to acquire knowledge and skills in the assessment and treatment of mental health disorders. However, study heterogeneity, prevalence of small sample sizes, and many studies with a high or serious risk of bias suggest an uncertain evidence base. Future research on the effectiveness of VR training should include assessment of immersive VR training designs and a focus on more robust studies with larger sample sizes.Trial registrationThis review was pre-registered in the Open Science Framework register with the ID-number Z8EDK.

Artificial intelligence in virtual standardized patients: Combining natural language understanding and rule based dialogue management to improve conversational fidelity

Introduction Advances in natural language understanding have facilitated the development of Virtual Standardized Patients (VSPs) that may soon rival human patients in conversational ability. We describe herein the development of an artificial intelligence (AI) system for VSPs enabling students to practice their history taking skills. Methods Our system consists of (1) Automated Speech Recognition (ASR), (2) hybrid AI for question identification, (3) classifier to choose between the two systems, and (4) automated speech generation. We analyzed the accuracy of the ASR, the two AI systems, the classifier, and student feedback with 620 first year medical students from 2018 to 2021. Results System accuracy improved from ∼75% in 2018 to ∼90% in 2021 as refinements in algorithms and additional training data were utilized. Student feedback was positive, and most students felt that practicing with the VSPs was a worthwhile experience. Conclusion We have developed a novel hybrid dialogue system that enables artificially intelligent VSPs to correctly answer student questions at levels comparable with human SPs. This system allows trainees to practice and refine their history-taking skills before interacting with human patients.

History-taking level and its influencing factors among nursing undergraduates based on the virtual standardized patient testing results: Cross sectional study

BackgroundHistory-taking is an essential skill for nurses. In nursing education, it is necessary to objectively assess history-taking skills in a way that accurately reflects differences among students. Current history-taking evaluation methods lack objectivity, consistency and standardization, which makes it difficult to identify factors that influence history-taking skill. A virtual standardized patient (VSP) can provide history-taking practice with repeatability and consistency. It can make objective and standardized assessment possible by eliminating the subjectivity of different teachers and patients. PurposeTo evaluate the history-taking skills of nursing undergraduates using a VSP, and to explore its independent influencing factors. MethodsA cross-sectional design was employed with a sample of convenience from 3 universities. All 174 nursing undergraduates had their history-taking level evaluated using one VSP via computer or mobile terminal. For each query raised by the students, the VSP could give a preset response. The responses were assigned corresponding scores according to their importance, with a total score of 100. The students' general demographics were documented, and their self-efficacy, critical thinking and communication skills were measured online with the Nursing History-taking Self-Efficacy Questionnaire, Yoon's Critical Thinking Disposition Instrument and the Supportive Communicative Scale. Univariate analysis, Pearson correlation analysis and multiple linear regression were performed. ResultsThe mean total history-taking score was 58.53 ± 14.32. History-taking scores were well discriminated among the students tested, with 10% of the students scoring above 80, 37% between 60 and 80, 22% between 50 and 60 and 31% scored below 50. The history-taking level of the nursing students was significantly associated with ethnicity, previous academic performance, and Supportive Communicative Scale scores. ConclusionsUsing the VSP as a history-taking assessment method is an effective way to achieve a relatively objective, standardized and consistent assessment of history-taking education. Ethnicity, previous academic performance and supportive communication skills independently influenced the students' history-taking level. Mature history-taking skills require not only solid theoretical knowledge but good communication skills.

Building psychiatric advanced practice student nurse competency to conduct comprehensive diagnostic interviews using two types of online simulation methods.

Through innovative use of established technologies, online nursing programs can provide psychiatric mental health nurse practitioner (PMHNP) students with robust virtual experiences for learning professional competencies, including those involving psychomotor and affective domains. To describe a virtual simulation teaching methodology using online text-based simulations of patient visits prior to a virtual standardized patient (SP) encounter in an asynchronous online course for PMHNP students. Student learning experiences were framed by the Framework for the 21st Century Learning. Two types of online simulations were employed: (1) four text-based exercises, created with a testing tool in a learning management system, allowed students to practice the critical thinking processes behind diagnostic interviewing, and (2) a telehealth simulation with an SP, where students engaged the patient, conducted a diagnostic interview, and discussed the treatment plan with the patient. Following the telehealth simulation, students completed self-assessments, received individual feedback from the SP and clinical faculty, and discussed experiences in small groups. Students demonstrated professional competencies and developed self-awareness and self-efficacy through reflection and discussion. This teaching approach can potentially enhance learning and build greater career skills including conducting diagnostic interviews using therapeutic communication skills for patients with stigmatizing mental health conditions.

Virtual Standardized Patients for Mental Health Education.

The training of psychiatrists and other mental health professionals requires education on a range of interpersonal, communication, and psychotherapy techniques. Classroom and workshop training must be augmented by experiential learning with feedback for skill implementation with fidelity. Virtual standardized patients (VSPs) are computerized conversational agents that can support experiential learning through standardized, consequence-free training environments at reduced costs. Research on mental health VSPs is rife with feasibility and acceptability pilot studies across various training populations and settings. Users have generally reported positive reactions to training with VSPs, though frustrations with some VSP speech recognition or VSP response relevance has been reported. Several studies have demonstrated a promising transfer of clinical skills from VSP training to human standardized patients and randomized trials supporting improved skill relative to reading or academic study are encouraging. As technology improves and natural language processing and accurate computer response generation for broad ranging conversational topics emerges, the field would benefit from research on the characteristics of effective VSPs for a range of purposes and trainee populations. Well-designed randomized evaluations of VSPs relative to best practices in education are needed, particularly regarding the impact of VSPs on clinical practice among actual patients.

Virtual Standardized Patients vs Academic Training for Learning Motivational Interviewing Skills in the US Department of Veterans Affairs and the US Military

Despite the need for effective and scalable training in motivational interviewing (MI) that includes posttraining coaching and feedback, limited evidence exists regarding the effectiveness of using virtual (computerized) standardized patients (VSPs) in such training. To evaluate the efficacy of training with a VSP on the acquisition and maintenance of MI skills compared with traditional academic study. This study was a 2-group, parallel-training randomized trial of 120 volunteer health care professionals recruited from a Department of Veterans Affairs and Department of Defense medical facility. Motivational interviewing skill was coded by external experts blinded to training group and skill assessment time points. Data were collected from October 17, 2016, to August 12, 2019. After a computer course on MI, participants trained during two 45-minute sessions separated by 3 months. The 2 randomized training conditions included a branching storyline VSP, which provided MI skill rehearsal with immediate and summative feedback, and a control condition, which included academic study of content from the computerized MI course. Measurement of MI skill was based on recorded conversations with human standardized patients, assessed using the Motivational Interviewing Treatment Integrity 4.2.1 coding system, measured at baseline, after training, and after additional training in the randomized condition 3 months later. A total of 120 volunteers (83 [69%] women), with a mean (SD) of 13.6 (10.3) years of health care experience, participated in the study; 61 were randomized to receive the intervention, and 59 were randomized to the control group. Those assigned to VSP training had significantly greater posttraining improvement in technical global scores (0.23; 95% CI, 0.03-0.44; P = .02), relational global scores (0.57; 95% CI, 0.33-0.81; P = .001), and the reflection-to-question ratio (0.23; 95% CI, 0.15-0.31; P = .001). Differences were maintained after the 3-month additional training session, with more improvements achieved after the 3-month training for the VSP trainees on the reflection-to- question ratio (0.15; 95% CI, 0.07-0.24; P = .001). This randomized trial demonstrated a successful transfer of training from a VSP to human standardized patients. The VSP MI skill outcomes were better than those achieved with academic study and were maintained over time. Virtual standardized patients have the potential to facilitate dissemination of MI and may be useful for training in other evidence-based skills and treatments. ClinicalTrials.gov Identifier: NCT04558060.

A Review of Telepresence, Virtual Reality, and Augmented Reality Applied to Clinical Care

This scoping review article explores the application of telepresence (TPr), virtual reality (VR), and augmented reality (AR) to clinical care. A literature search of key words was conducted from January 1990 through May 2019 of the following databases: PubMed/Medline, American Psychological Association PsycNET, Pubmed/Medline, Cochrane, Embase, PsycINFO, Web of Science, Scopus, OTSeeker, ABI/INFORM, computer-mediated communication (CMC), technology-mediated communications, Arts & Humanities Citation Index, Project Muse, ProQuest Research Library Plus, Sociological abstracts, Computers and Applied Sciences Complete and IT Source. It focused on concept areas: (1) TPr related to technologies; (2) virtual, augmented, reality, environment; (3) technology or computer-mediated communication; (4) clinical therapeutic relationship (boundaries, care, communication, connect, engagement, empathy, intimacy, trust); (5) telebehavioral health; (6) psychotherapy via technology; and (7) medicine/health care. Inclusion criteria were concept area 1 in combination with 2–7 and 2 or 3 in combination with any of 4–7. From a total of 5214 potential references, the authors found 512 eligible for full-text review and found 85 papers directly relevant to the concepts. From papers’ references and a review of books and popular literature about TPr, virtual reality (VR), and augmented reality (AR), 13 other sources of information were found. The historical evolution of TPr, VR, and AR shows that definitions, foci of studies (e.g., social neuroscience to business), and applications vary; assessments of TPr also vary widely. Studies discuss VR, AR, and TPr in medicine (e.g., rehabilitation, robotics), experimental psychology (laboratory, field, mixed), and behavioral health. Virtual environment (VE) designs aid the study of interpersonal communication and behavior, using standardized social interaction partners, virtual standardized patients, and/or virtual humans—all contingent on the participants’ experience of presence and the ability to engage. Additional research is needed to standardize experimental and clinical interventions, while maintaining ecological validity. Technology can significantly improve quality of care, access to new treatments and training, if ethical and reimbursement issues are better explored.

The Emotional and Behavioral Impact of Delivering Bad News to Virtual versus Real Standardized Patients: A Pilot Study

Phenomenon: Virtual standardized patients (vSPs) are becoming increasingly common in medical education, though one limitation of vSPs is the artificiality of computer-based simulators. Past research on the use of vSPs has not clearly established whether learners have different emotional responses to real SPs (rSPs) compared with vSPs; however, understanding learners’ emotional responses to vSPs is important in providing realistic learning experiences and establishing the validity of this teaching and assessment tool. This study compared the emotional experiences of individuals who interacted with rSPs and vSPs. Approach: Sixty medical students at a medical school in the southeastern United States participated in the study. Participants were randomly assigned to deliver bad news to an rSP or vSP. The vSP for this study used a hybrid intelligence model that allowed a person to “inhabit” the vSP. Salivary cortisol and a self-report measure of mood—the Profile of Mood States, Second Edition (POMS 2)—were gathered before and after delivering the bad news. The SP and 2 independent evaluators rated the behavioral performance of each participant in real and virtual conditions. Participants also rated the performance of the SP. Findings: Participants in both conditions reported increased negative emotionality on the POMS 2 following the SP interaction. There were no significant between-group differences on the POMS 2 or salivary cortisol concentration following the SP interaction. Ratings by the SP and independent evaluators indicated that participants performed similarly on most interpersonal dimensions, except tone of voice. Participants perceived the vSP as less realistic than the rSP. Insights: These results suggest that medical students may have similar emotional and behavioral responses when delivering bad news to a vSP when compared to an rSP. These findings provide support for the continued use of vSPs in training learners to deliver bad news and other communication-based skills and to assess their performance on these tasks.

Using virtual standardized patients to accurately assess information gathering skills in medical students

Introduction: Practicing a medical history using standardized patients is an essential component of medical school curricula. Recent advances in technology now allow for newer approaches for practicing and assessing communication skills. We describe herein a virtual standardized patient (VSP) system that allows students to practice their history taking skills and receive immediate feedback.Methods: Our VSPs consist of artificially intelligent, emotionally responsive 3D characters which communicate with students using natural language. The system categorizes the input questions according to specific domains and summarizes the encounter. Automated assessment by the computer was compared to manual assessment by trained raters to assess accuracy of the grading system.Results: Twenty dialogs chosen randomly from 102 total encounters were analyzed by three human and one computer rater. Overall scores calculated by the computer were not different than those provided by the human raters, and overall accuracy of the computer system was 87%, compared with 90% for human raters. Inter-rater reliability was high across 19 of 21 categories.Conclusions: We have developed a virtual standardized patient system that can understand, respond, categorize, and assess student performance in gathering information during a typical medical history, thus enabling students to practice their history-taking skills and receive immediate feedback.

Virtual Standardized Patient Simulation: Case Development and Pilot Application to High-Value Care.

High-value care (HVC) suggests that good history taking and physical examination should lead to risk stratification that drives the use or withholding of diagnostic testing. This study describes the development of a series of virtual standardized patient (VSP) cases and provides preliminary evidence that supports their ability to provide experiential learning in HVC. This pilot study used VSPs, or natural language processing-based patient avatars, within the USC Standard Patient platform. Faculty consensus was used to develop the cases, including the optimal diagnostic testing strategies, treatment options, and scored content areas. First-year resident physician learners experienced two 90-minute didactic sessions before completing the cases in a computer laboratory, using typed text to interview the avatar for history taking, then completing physical examination, differential diagnosis, diagnostic testing, and treatment modules for each case. Learners chose a primary and 2 alternative "possible" diagnoses from a list of 6 to 7 choices, diagnostic testing options from an extensive list, and treatments from a brief list ranging from 6 to 9 choices. For the history-taking module, both faculty and the platform scored the learners, and faculty assessed the appropriateness of avatar responses. Four randomly selected learner-avatar interview transcripts for each case were double rated by faculty for interrater reliability calculations. Intraclass correlations were calculated for interrater reliability, and Spearman ρ was used to determine the correlation between the platform and faculty ranking of learners' history-taking scores. Eight VSP cases were experienced by 14 learners. Investigators reviewed 112 transcripts (4646 learner query-avatar responses). Interrater reliability means were 0.87 for learner query scoring and 0.83 for avatar response. Mean learner success for history taking was scored by the faculty at 57% and by the platform at 51% (ρ correlation of learner rankings = 0.80, P = 0.02). The mean avatar appropriate response rate was 85.6% for all cases. Learners chose the correct diagnosis within their 3 choices 82% of the time, ordered a median (interquartile range) of 2 (2) unnecessary tests and completed 56% of optimal treatments. Our avatar appropriate response rate was similar to past work using similar platforms. The simulations give detailed insights into the thoroughness of learner history taking and testing choices and with further refinement should support learning in HVC.

Enhancing Nursing Education Through Affordable and Realistic Holographic Mixed Reality: The Virtual Standardized Patient for Clinical Simulation.

Nurses serve a valuable role in the healthcare industry. Nurses are trained with the skills and knowledge to thrive in a fast-paced, evolving environment. In order to meet the complex and diverse needs of patients, nurses must be able to assess and prioritize care to produce safe and high-quality outcomes. Simulation is an established method of educating nursing students and preparing nurses to respond appropriately to situations they are likely to encounter in practice. Traditional nursing simulation devices are prohibitively expensive for many nursing education institutions. The development of augmented, mixed, and virtual reality simulation delivery offers a new platform for simulation, known as immersive simulation. Immersive simulation can virtually place nursing students in situations that are difficult to arrange in actual clinical practicums or that occur rarely but for which nurses need to be prepared. Additionally, the hardware required to deliver immersive simulation is much cheaper than that of traditional nursing simulation devices. This chapter describes the virtual standardized patient application delivered via mixed reality immersive simulation. This chapter also discusses the research initiative currently underway to assess student perceptions to this modality of health training simulation.

Virtual Standardized Patient

Background: Higher education nursing programs include a physical/health assessment course that prepares graduates to provide excellent care based on data collected through assessment. Finding a teaching strategy that engages students as well as educates them in health assessment techniques is a challenge that all nurse educators share.&#x0D; Objectives: The objectives of this study were to determine the level of engagement and learning that occurred with undergraduate and graduate students who participated in a health assessment course that included the use of an online virtual standardized patient (VSP).&#x0D; Design: Quantitative, descriptive survey research was utilized to determine student nurses’ perception of the use of VSP in a physical assessment course. &#x0D; Setting/Participants: Students in a traditional on-campus BSN program and students in an online RN to BSN and MSN program were included in this study.&#x0D; Methods: After receiving IRB approval, course leaders provided students a standardized questionnaire of two (2) multiple choice and six (6) Likert-style questions. The questions evaluated the participants’ ease of use, comparison of interaction between online virtual and live human patients, ability to perform an exam on a virtual patient, and ability to communicate with a virtual patient. Questionnaires were administered to volunteers in the three identified health assessment courses (BSN, RN-BSN, and MSN). Also evaluated was the participant’s perception of their ability to transfer knowledge from the VSP to the live patient. All students participated in the VSP with completion of the questionnaire optional.&#x0D; Results: Findings in this study generally supported the educational value of using a virtual standardized patient in teaching both undergraduate and graduate students’ health assessment. Variations between the groups were found.&#x0D; Conclusion: Use of a virtual standardized patient is a positive teaching strategy for teaching health assessment in both undergraduate and graduate nursing programs.

Virtual Antenatal Encounter and Standardized Simulation Assessment (VANESSA): Pilot Study.

BackgroundPrenatal counseling at the limits of newborn viability involves sensitive interactions between neonatal providers and families. Empathetic discussions are currently learned through practice in times of high stress. Decision aids may help improve provider communication but have not been universally adopted. Virtual standardized patients are increasingly recognized as a modality for education, but prenatal counseling simulations have not been described. To be valuable as a tool, a virtual patient would need to accurately portray emotions and elicit a realistic response from the provider.ObjectiveTo determine if neonatal providers can accurately identify a standardized virtual prenatal patient’s emotional states and examine the frequency of empathic responses to statements made by the patient.MethodsA panel of Neonatologists, Simulation Specialists, and Ethicists developed a dialogue and identified empathic responses. Virtual Antenatal Encounter and Standardized Simulation Assessment (VANESSA), a screen-based simulation of a woman at 23 weeks gestation, was capable of displaying anger, fear, sadness, and happiness through animations. Twenty-four neonatal providers, including a subgroup with an ethics interest, were asked to identify VANESSA’s emotions 28 times, respond to statements, and answer open-ended questions. The emotions were displayed in different formats: without dialogue, with text dialogue, and with audio dialogue. Participants completed a post-encounter survey describing demographics and experience. Data were reported using descriptive statistics. Qualitative data from open ended questions (eg, “What would you do?”) were examined using thematic analysis.ResultsHalf of our participants had over 10 years of clinical experience. Most participants reported using medical research (18/23, 78%) and mortality calculators (17/23, 74%). Only the ethics-interested subgroup (10/23, 43%) listed counseling literature (7/10, 70%). Of 672 attempts, participants accurately identified VANESSA’s emotions 77.8% (523/672) of the time, and most (14/23, 61%) reported that they were confident in identifying these emotions. The ethics interest group was more likely to choose empathic responses (P=.002). Participants rated VANESSA as easy to use (22/23, 96%) and reported that she had realistic dialogue (15/23, 65%).ConclusionsThis pilot study shows that a prenatal counseling simulation is feasible and can yield useful data on prenatal counseling communication. Our participants showed a high rate of emotion recognition and empathy in their responses.

Developing a Conversational Virtual Standardized Patient to Enable Students to Practice History-Taking Skills.

Although traditional virtual patient simulations are designed to teach and assess clinical reasoning skills, few employ conversational dialogue with the patients. The virtual standardized patients (VSPs) described herein represent standardized patients that students interview using natural language. Students take histories and develop differential diagnoses of the VSPs as much as they would with standardized or actual patients. The student-VSP interactions are recorded, creating a comprehensive record of questions and the order in which they were asked, which can be analyzed to assess information-gathering skills. Students document the encounter in an electronic medical record created for the VSPs. The VSP was developed by integrating a dialogue management system (ChatScript) with emotionally responsive 3D characters created in a high-fidelity game engine (Unity). The system was tested with medical students at the Ohio State University College of Medicine. Students are able to take a history of a VSP, develop a differential diagnosis, and document the encounter in the electronic medical record. Accuracy of the VSP responses ranged from 79% to 86%, depending on the complexity of the case, type of history obtained, and skill of the student. Students were able to accurately develop an appropriate differential diagnosis on the basis of the information provided by the patient during the encounter. The VSP enables students to practice their history-taking skills before encounters with standardized or actual patients. Future developments will focus on creating an assessment module that will automatically analyze VSP sessions and provide immediate student feedback.

Natural Language Understanding Performance & Use Considerations in Virtual Medical Encounters.

A virtual standardized patient (VSP) prototype was tested for natural language understanding (NLU) performance. The conversational VSP was evaluated in a controlled 61 subject study over four repetitions of a patient case. The prototype achieved more than 92% appropriate response rate from naïve users on their first attempt and results were stable by their fourth case repetition. This level of performance exceeds prior efforts and is at a level comparable of accuracy as seen in human conversational patient training, with caveats. This level of performance was possible due to the use of a unified medical taxonomy underpinning that allows virtual patient language training to be applied to all cases in our system as opposed to benefiting a single patient case.