Text-based Scenario Research Articles

Is a Picture Worth 280 Characters?: Contextually Realistic Graphics vs. Plain Text in Survey Experiments

Abstract As survey experiments have become increasingly common in political science, some scholars have questioned whether inferences about the real world can be drawn from experiments involving hypothetical, text-based scenarios. In response to this criticism, some researchers recommended using realistic, context-heavy vignettes while others argue that abstract vignettes do not generate substantially different results. We contribute to this debate by evaluating whether incorporating contextually realistic graphics into survey experiment vignettes affects experimental outcomes. We field three original experiments that vary whether respondents are shown a realistic graphic or a plain text description during an international crisis. In our experiments, varying whether respondents are shown realistic graphics or plain text descriptions generally yields little difference in outcomes. Our findings have implications for survey methodology and experiments in political science – researchers may not need to invest the time to develop contextually realistic graphics when designing experiments.

Incorporating Patient Values in Large Language Model Recommendations for Surrogate and Proxy Decisions.

Surrogates, proxies, and clinicians making shared treatment decisions for patients who have lost decision-making capacity often fail to honor patients' wishes, due to stress, time pressures, misunderstanding patient values, and projecting personal biases. Advance directives intend to align care with patient values but are limited by low completion rates and application to only a subset of medical decisions. Here, we investigate the potential of large language models (LLMs) to incorporate patient values in supporting critical care clinical decision-making for incapacitated patients in a proof-of-concept study. We simulated text-based scenarios for 50 decisionally incapacitated patients for whom a medical condition required imminent clinical decisions regarding specific interventions. For each patient, we also simulated five unique value profiles captured using alternative formats: numeric ranking questionnaires, text-based questionnaires, and free-text narratives. We used pre-trained generative LLMs for two tasks: 1) text extraction of the treatments under consideration and 2) prompt-based question-answering to generate a recommendation in response to the scenario information, extracted treatment, and patient value profiles. Model outputs were compared with adjudications by three domain experts who independently evaluated each scenario and decision. Automated extractions of the treatment in question were accurate for 88% (n = 44/50) of scenarios. LLM treatment recommendations received an average Likert score by the adjudicators of 3.92 of 5.00 (five being best) across all patients for being medically plausible and reasonable treatment recommendations, and 3.58 of 5.00 for reflecting the documented values of the patient. Scores were highest when patient values were captured as short, unstructured, and free-text narratives based on simulated patient profiles. This proof-of-concept study demonstrates the potential for LLMs to function as support tools for surrogates, proxies, and clinicians aiming to honor the wishes and values of decisionally incapacitated patients.

Customer comfort during service robot interactions

Customer comfort during service interactions is essential for creating enjoyable customer experiences. However, although service robots are already being used in a number of service industries, it is currently not clear how customer comfort can be ensured during these novel types of service interactions. Based on a 2 × 2 online between-subjects design including 161 respondents using pictorial and text-based scenario descriptions, we empirically demonstrate that human-like (vs machine-like) service robots make customers feel more comfortable because they facilitate rapport building. Social presence does not underlie this relationship. Importantly, we find that these positive effects diminish in the presence of service failures.

The effect of cognitive bias modification for interpretation (CBM-I) on social anxiety and mood: Compared text-based scenarios training with picture-word cue training.

The effect of cognitive bias modification for interpretation (CBM-I) on social anxiety and mood: Compared text-based scenarios training with picture-word cue training.

Forecasting for a Fractured Land: A Case Study of the Communication and Use of Aftershock Forecasts from the 2016 Mw 7.8 Kaikōura Earthquake in Aotearoa New Zealand

Abstract Operational earthquake forecasts (OEFs) are represented as time-dependent probabilities of future earthquake hazard and risk. These probabilities can be presented in a variety of formats, including tables, maps, and text-based scenarios. In countries such as Aotearoa New Zealand, the U.S., and Japan, OEFs have been released by scientific organizations to agencies and the public, with the intent of providing information about future earthquake hazard and risk, so that people can use this information to inform their decisions and activities. Despite questions being raised about the utility of OEF for decision-making, past earthquake events have shown that agencies and the public have indeed made use of such forecasts. Responses have included making decisions about safe access into buildings, cordoning, demolition safety, timing of infrastructure repair and rebuild, insurance, postearthquake building standards, postevent land-use planning, and public communication about aftershocks. To add to this body of knowledge, we undertook a survey to investigate how agencies and GNS Science staff used OEFs that were communicated following the Mw 7.8 2016 Kaikōura earthquake in Aotearoa New Zealand. We found that agencies utilized OEFs in many of the ways listed previously, and we document individual employee’s actions taken in their home-life context. Challenges remain, however, regarding the interpretation of probabilistic information and applying this to practical decision-making. We suggest that science agencies cannot expect nontechnical users to understand and utilize forecasts without additional support. This might include developing a diversity of audience-relevant OEF information for communication purposes, alongside advice on how such information could be utilized.

A Scenario-Based Survey of Expert Echocardiography Recommendations for Patients With Staphylococcus aureus Bacteremia at Varying Risk for Endocarditis

Echocardiography to detect infective endocarditis is regarded as a key quality indicator in the care of patients with Staphylococcus aureus bacteremia, but its application varies markedly between reported series. Understanding the reasons for this variation in practice is important to improve the use of this investigation. To identify expert clinicians' preferred echocardiography strategy for a variety of S aureus bacteremia scenarios in a hypothetical setting free from extrinsic constraints. Anonymous web-based survey study comprising 50 text-based scenarios describing patients with S aureus bacteremia and various combinations of risk factors for endocarditis. Other variables included patient age and the presence of an extracardiac focus of infection warranting prolonged treatment. The survey was emailed to participants between September 2018 and March 2019. Each respondent was asked to recommend 1 of 6 echocardiography strategies for up to 8 randomly selected scenarios. Respondents were primarily infectious diseases physicians, and more than half reported an annual caseload of more than 20 cases of S aureus bacteremia. The proportion of respondents selecting each of the 6 echocardiography strategies was calculated alongside Wilson score confidence intervals. Modified Fleiss κ statistics were used to described interrespondent variability. Generalized estimating equations were used to assess the associations between respondent- and scenario-level variables and the recommendation of an echocardiography strategy with a low negative likelihood ratio for infective endocarditis (ie, a highly exclusionary strategy). A total of 656 respondents from 24 countries provided 4837 echocardiography recommendations across the 50 scenarios. Echocardiography recommendations were associated with scenarios' burden of endocarditis risk (multivariate odds ratio per point of the VIRSTA score, 1.4; 95% CI, 1.4-1.5; P < .001). Poor interrespondent agreement was seen across all scenarios (modified Fleiss κ, 0.06; 95% CI, 0.05-0.07) but was most notable for scenarios with a lower risk of endocarditis (modified Fleiss κ, 0.04; 95% CI, 0.03-0.05). The presence of an extracardiac focus of infection was also associated with the choice of echocardiography strategy (odds ratio for highly exclusionary strategy, 0.51; 95% CI, 0.45-0.58). Respondent location in continental Europe was associated with recommendations in favor of a highly exclusionary strategy (odds ratio, 1.8; 95% CI, 1.3-2.5) compared with location in Australia or New Zealand. In this study, expert clinicians demonstrated active stratification by risk of endocarditis when making echocardiography recommendations for hypothetical patients with S aureus bacteremia. Substantial disagreement existed as to whether patients at lower risk of endocarditis should undergo transesophageal echocardiography-based echocardiography strategies.

An open randomized controlled study comparing an online text-based scenario and a serious game by Belgian and Swiss pharmacy students

IntroductionTo compare online learning tools, looped, branch serious game (SG) and linear text-based scenario (TBS), among a sample of Belgian and Swiss pharmacy students. MethodsOpen randomized controlled study. The lesson was based on the case of a benign cough in a healthy child. A randomized sample of 117 students: only the Swiss students had attended a previous lecture on coughs. Participation rate, pre- and post-experience Likert scales and students’ clinical knowledge were measured. ResultsOur primary hypothesis was demonstrated: students favored the SG even if navigation was rated as more complex, and students who performed the SG better understood the aim of pharmacist triage in case of cough. The influence of the SG appeared to be linked to the presence of a previous lecture in the curriculum. Discussion and conclusionSG and TBS are effective to teach pharmacist triage. Higher SG complexity should be used to teach the aim of pharmacist triage in the case of a specific disease and could be an alternative to simulated patients. A simpler TBS does not require a previous lecture and a debriefing to be fully effective.

Imagine the bright side of life: A randomized controlled trial of two types of interpretation bias modification procedure targeting adolescent anxiety and depression.

IntroductionAnxiety and depression are highly prevalent during adolescence and characterized by negative interpretation biases. Cognitive bias modification of interpretations (CBM-I) may reduce such biases and improve emotional functioning. However, as findings have been mixed and the traditional scenario training is experienced as relatively boring, a picture-based type of training might be more engaging and effective.MethodsThe current study investigated short- and long-term effects (up to 6 months) and users’ experience of two types of CBM-I procedure in adolescents with heightened symptoms of anxiety or depression (N = 119, aged 12–18 year). Participants were randomized to eight online sessions of text-based scenario training, picture-word imagery training, or neutral control training.ResultsNo significant group differences were observed on primary or secondary emotional outcomes. A decrease in anxiety and depressive symptoms, and improvements in emotional resilience were observed, irrespective of condition. Scenario training marginally reduced negative interpretation bias on a closely matched assessment task, while no such effects were found on a different task, nor for the picture-word or control group. Subjective evaluations of all training paradigms were relatively negative and the imagery component appeared particularly difficult for adolescents with higher symptom levels.ConclusionsThe current results question the preventive efficacy and feasibility of both CBM-I procedures as implemented here in adolescents.

Evaluating automated goals for home care support

Purpose – The purpose of this paper is to evaluate an approach to automating goals for supporting home care, with a view to understanding user experience when defining such goals and hence identifying improvements that could be made to the approach. Design/methodology/approach – The study was designed to answer the key research question of whether users can understand, formulate and relate to automated goals for home care. In order to do this, a fictional text-based scenario was used about a couple with care needs. This helped to explore the feasibility, acceptability and usability of goals to manage care at home. Face-to-face qualitative interviews were undertaken with ten participants with a background in social care: four social care professionals; one health care professional; one formal carer; one informal carer; and three end users. Findings – Overall, participants were positive about being able to control the MATCH (Mobilising Advanced Technologies for Care at Home) system through the use of goals. The results from the participant interviews will be used to help guide potential improvements to the home care system. The main issue that emerged from the study is that it would be valuable to think in terms of outcomes as a higher level than goals. A second consideration is that it would be desirable to adopt terminology that can be understood by all stakeholders. Research limitations/implications – The study has demonstrated that automated goals for home care have a useful role to play and can be successfully used by end users and carers. Although the range of participants in the study was limited, it has allowed confidence to be built in the approach and has identified useful pointers for future development. Practical implications – With the evaluation and validation of the goal-based approach, it has encouraged the developers to make automated goals more widely available in future deployment of the home care system. Social implications – The use of automated goals to support home care has been shown to be acceptable to end users and carers. This will allow future home care systems to offer more personal and better customised services to those receiving telecare. Originality/value – The study provides a unique evaluation of the use of automated goals to support home care. Previous use of goals in the literature has been for highly technical applications, so their application to home care is novel and speculative. The study has demonstrated that the approach is viable, useful, and usable by end users and carers.

Probabilistic Graphical Modeling of Terrorism Threat Recognition Using Bayesian Networks and Monte Carlo Simulation

The objective of this study was to identify and model the comprehension and decision making of law enforcement personnel with respect to terrorism-centric behaviors. In this study, 44 participants provided 1,496 judgments ranked on an 11-point Likert-type suspicion scale about individual text-based scenario components emulating real-world events encountered during routine policing. The data were analyzed to assess the influence of jurisdiction, training, experience, and terrorism familiarity on the recognition of terrorism-centric behaviors. Measurements of those factors were used to formulate a Bayesian network, and Monte Carlo simulation was performed to estimate their effect on terrorism-centric cognitive judgment and decision making, defined by the response to a wide variety of simulated terrorism-centric behaviors using text-based scenarios.

Surprise, memory, and retrospective judgment making: Testing cognitive reconstruction theories of the hindsight bias effect.

Hindsight bias has been shown to be a pervasive and potentially harmful decision-making bias. A review of 4 competing cognitive reconstruction theories of hindsight bias revealed conflicting predictions about the role and effect of expectation or surprise in retrospective judgment formation. Two experiments tested these predictions examining the effects of manipulating the information presented in a text-based scenario and its congruency with the given outcome on surprise, hindsight bias, and recall. The results of Experiment 1 revealed evidence of hindsight bias after exposure to incongruent and ambivalent outcomes but not after exposure to congruent outcomes. Experiment 2 replicated the hindsight bias results and found that the ratio of outcome consistent information recalled was higher than expected in the incongruent and ambivalent conditions but equaled the ratio presented to participants in the congruent condition. The results were interpreted as supporting the general predictions of sense-making models of the hindsight bias. A refined version of this model is discussed.

Effectiveness of Photographic Attachments to Text-Based Scenario on Elderly Users in Usability Tests.

Effectiveness of Photographic Attachments to Text-Based Scenario on Elderly Users in Usability Tests.

Reasoning When It Counts: Students’ Rationales for Action on a Professionalism Exam

Previous research explored students' reasoning in the face of professional dilemmas, using interviews in response to videotaped scenarios. This study determined the effects of a change in context (to a written exam) and format (video versus text scenarios) on students' response patterns. Fifty-three students were randomized to videotaped or text-based scenarios in the context of a mock written exam. Responses were coded by two raters. Interrater reliability was high (kappa = 0.872). There were no differences in response patterns between the video and text groups. When compared with the interview setting, students' exam responses showed a shift towards more "acceptable" rationales for action; however, they still considered implications for themselves. This shift in responses indicates that students took the exam seriously. Their continued reference to implications for themselves might therefore reflect a sense that their status as students makes these considerations legitimate; alternatively, students' interpretation of altruism may be different than what the profession avows.