Hospital Report Cards Research Articles

Do Australian health consumers understand and intend to use Patient Reported Experience Measures (PREMs) when selecting a hospital?: A qualitative exploration of online hospital report cards

Do Australian health consumers understand and intend to use Patient Reported Experience Measures (PREMs) when selecting a hospital?: A qualitative exploration of online hospital report cards

The elicitation of patient and physician preferences for calculating consumer-based composite measures on hospital report cards: results of two discrete choice experiments

PurposeThe calculation of aggregated composite measures is a widely used strategy to reduce the amount of data on hospital report cards. Therefore, this study aims to elicit and compare preferences of both patients as well as referring physicians regarding publicly available hospital quality informationMethodsBased on systematic literature reviews as well as qualitative analysis, two discrete choice experiments (DCEs) were applied to elicit patients’ and referring physicians’ preferences. The DCEs were conducted using a fractional factorial design. Statistical data analysis was performed using multinomial logit modelsResultsApart from five identical attributes, one specific attribute was identified for each study group, respectively. Overall, 322 patients (mean age 68.99) and 187 referring physicians (mean age 53.60) were included. Our models displayed significant coefficients for all attributes (p < 0.001 each). Among patients, “Postoperative complication rate” (20.6%; level range of 1.164) was rated highest, followed by “Mobility at hospital discharge” (19.9%; level range of 1.127), and ‘‘The number of cases treated” (18.5%; level range of 1.045). In contrast, referring physicians valued most the ‘‘One-year revision surgery rate’’ (30.4%; level range of 1.989), followed by “The number of cases treated” (21.0%; level range of 1.372), and “Postoperative complication rate” (17.2%; level range of 1.123)ConclusionWe determined considerable differences between both study groups when calculating the relative value of publicly available hospital quality information. This may have an impact when calculating aggregated composite measures based on consumer-based weighting.

Referring physicians' intention to use hospital report cards for hospital referral purposes in the presence or absence of patient-reported outcomes: a randomized trial

PurposeThis study aims to determine the intention to use hospital report cards (HRCs) for hospital referral purposes in the presence or absence of patient-reported outcomes (PROs) as well as to explore the relevance of publicly available hospital performance information from the perspective of referring physicians.MethodsWe identified the most relevant information for hospital referral purposes based on a literature review and qualitative research. Primary survey data were collected (May–June 2021) on a sample of 591 referring orthopedists in Germany and analyzed using structural equation modeling. Participating orthopedists were recruited using a sequential mixed-mode strategy and randomly allocated to work with HRCs in the presence (intervention) or absence (control) of PROs.ResultsOverall, 420 orthopedists (mean age 53.48, SD 8.04) were included in the analysis. The presence of PROs on HRCs was not associated with an increased intention to use HRCs (p = 0.316). Performance expectancy was shown to be the most important determinant for using HRCs (path coefficient: 0.387, p < .001). However, referring physicians have doubts as to whether HRCs can help them. We identified “complication rate” and “the number of cases treated” as most important for the hospital referral decision making; PROs were rated slightly less important.ConclusionsThis study underpins the purpose of HRCs, namely to support referring physicians in searching for a hospital. Nevertheless, only a minority would support the use of HRCs for the next hospital search in its current form. We showed that presenting relevant information on HRCs did not increase their use intention.

Breast cancer patients' visual attention to information in hospital report cards: An eye-tracking study on differences between younger and older female patients.

To (1) explore how women visually attend to a hospital report card (HRC), (2) explore whether visual attention of younger and older women (patients and non-patients) differs. Eye-tracking study with a short survey. Participants (N = 37) were provided with a hypothetical realistic HRC. Total dwell times and fixation counts were measured while participants viewed the information. Overall, no differences existed between younger and older women. Visual attention to the hospital of choice (vs not of choice) and to indicators perceived as most important (vs least important) did not differ. However, women with higher health literacy looked longer at the HRC than women with lower health literacy. Also, per fixation, older patients (vs younger patients) looked longer at the hospital of choice and at indicators perceived most important. Pre-existing conceptions of what information is relevant might result in more in-depth information processing among older patients than younger patients. In general, differences in level of health literacy, rather than (chronological) age, seem to be relevant to take into account when designing and/or updating HRCs.

The use intention of hospital report cards among patients in the presence or absence of patient-reported outcomes

BackgroundSo far, the adoption of hospital report cards (HRCs) falls short of expectations. One promising strategy is to modify the content of HRCs by presenting patient-reported outcomes (PROs). ObjectiveTo identify the key determinants influencing patients to use HRCs for hospital decision making and determine the effect of presenting PROs on HRCs on their use intention. MethodsPrimary survey data were collected (5/6–2021) on a sample of 2000 randomly selected insurees from a German statutory health insurance who have undergone elective hip arthroplasty surgery. ResultsOverall, 447 participants (mean age 66.56) completed the survey and were included in the analysis. Respondents rated “PROs” as most important for the hospital choice followed by “Mobility at hospital discharge” and “Confirmed diagnosis rate”. Patients generally perceive HRCs to be a good idea that makes searching for a hospital more interesting. We identified attitude and social influence as the most important determinants for using HRCs (p<.001 each). The presence of PROs on HRCs was not associated with an increased intention to use HRCs neither in our descriptive analysis (p=.593), nor in our research model (p=.763). ConclusionsPatients value PROs to be an important information for choosing a hospital. Nevertheless, health policy makers should note that presenting PROs on HRCs as a single approach is not likely to increase the use of HRCs.

Testing the effects on information use by older versus younger women of modality and narration style in a hospital report card.

BackgroundHospital report cards (HRCs) are usually presented in a textual and factual format, likely hampering information processing.ObjectiveThis study aimed to investigate the effects of audiovisual and narrative information in HRCs on user responses, and to test differences between older and younger women.DesignA 2 (modality [textual vs. audiovisual]) × 3 (narration style [factual vs. process narrative vs. experience narrative]) online experiment was conducted. Information about breast cancer care was used as a case example. Age (younger [<65] vs. older [≥65]) was included as a potential effect modifier.Setting and ParticipantsA total of 631 disease‐naïve women (M age = 56.06) completed an online survey. The outcomes were perceived cognitive load, satisfaction, comprehension, information recall and decisional conflict. Data were analysed using AN(C)OVAs.ResultsAudiovisual (vs. textual) information resulted in higher information satisfaction across age groups, but was associated with lower comprehension in older women. An experience narrative (vs. factual information) increased satisfaction with attractiveness and emotional support of the information only in older women. A three‐way interaction effect was found, suggesting that older women were most satisfied with the comprehensibility of audiovisual factual or textual process narrative information. Younger women were most satisfied with the comprehensibility of audiovisual process narrative or textual factual information.Discussion and ConclusionAudiovisual and narrative information in an HRC showed beneficial effects on satisfaction measures. In particular, audiovisual information could be incorporated into HRCs to increase satisfaction with information.Public ContributionLay persons helped in optimizing the visuals used in the stimulus materials by checking for clarity.

Hospital report cards: Quality competition and patient selection

Hospital ‘report cards’ policies involve governments publishing information about hospital quality. Such policies often aim to improve hospital quality by stimulating competition between hospitals. Previous empirical literature lacks a comprehensive theoretical framework for analysing the effects of report cards. We model a report card policy in a market where two hospitals compete for patients on quality under regulated prices. The report card policy improves the accuracy of the quality signal observed by patients. Hospitals may improve their published quality scores by costly quality improvement or by selecting healthier patients to treat. We show that increasing information through report cards always increases quality and only sometimes induces selection. Report cards are more likely to increase patient welfare when quality scores are well risk-adjusted, where the cost of selecting patients is high, and the cost of increasing quality is low.

Quality information disclosure and health insurance demand: evidence from VA hospital report cards.

This study examines the effect of public reporting of quality information on the demand for public insurance. In particular, we examine the effect of the introduction of Veterans Affairs (VA) hospital quality report cards in 2008. Using data from the Current Population Survey in 2005-2015, we find that new information about the quality of a VA hospital had a significant effect on VA coverage among veterans living in the same Metropolitan Statistical Area (MSA). Despite the significant effect on VA coverage, the quality report did not have a spillover effect on veterans' labor supply. Moreover, updated quality information released in later years, which was presented in a less straightforward form, led to no additional changes in VA coverage. These findings suggest that quality reports for public insurance programs can be used as a policy lever to facilitate take up decision among potential beneficiaries.

Characteristics and decision making of hospital report card consumers: Lessons from an onsite-based cross-sectional study

BackgroundHospitals report cards (HRCs) have had little impact on the hospital choice of patients. Thus, health policy makers should learn more about HRC consumers to better understand how to present and target hospital-related quality information. ObjectiveWe sought to learn more about consumers of HRCs and determine the impact of the complexity and tailoring of HRCs on the hospital choice. MethodsWe used primary data drawn from an onsite-based survey, conducted in 2017 at Germany’s premier portal, Weisse Liste (N = 635). We performed hierarchical multivariate logistic regression models to identify main predictors associated with hospital choices. ResultsHRC consumers differ from the national online population and the national population in general. Eighty percent of those patients or family members, who have used a HRC before, confirmed an impact on the hospital choice. The quality of hospital choices decreased with an increasing level of complexity (p < .001); the latter was identified as a significant predictor for making good choices. However, tailoring HRCs did not have an impact on the quality of the hospital choice (p > .05). ConclusionsHRCs have a significant impact on the hospital choice among report card consumers. Health policy makers might focus on decreasing the level of complexity; this, more than tailoring report cards, may help consumers make good hospital choices.

Abbildung urologischer Inhalte auf online Bewertungsportalen zur Krankenhauswahl in Deutschland – eine systematische Übersicht

BACKGROUND : The acquisition of information on hospitals' quality-of-care is an important aspect for patients' decision-making, especially in the setting of elective interventions. Moreover, hospitals depend on a transparent representation of their performance in the current competitive setting. This study aims to evaluate the representation and rating of online hospital report cards with special focus on urological disease and departments. MATERIAL UND METHODS : We undertook a systematic internet search to identify eligible online hospital report cards. The quality of hospital report cards was rated on a scoring system that was developed specifically for this purpose. RESULTS: Ten of 42 (24 %) identified hospital report cards were included. Urological content was available in 9 of 10 (90 %); detailed information on surgical interventions and outcomes was given in 3 (33 %) hospital navigators. Recommended criteria for evaluation of transparency and reliability of medical information were most present in the Weisse Liste, and the hospital-navigators of AOK and BARMER. The Weisse Liste is currently the only HON-code certified hospital report card in Germany. CONCLUSION : There is a broad, but divers range of online hospital report cards available for patients in Germany. The evaluation of urological content of hospital report cards is currently feasible in terms of analysis of general parameters, but not on outcome specific quality indicators. Clarity and reduction on essential information is warranted to ensure user-oriented platforms.

Verstehen Laien Informationen über die Krankenhausqualität? Eine empirische Überprüfung am Beispiel der risikoadjustierten Mortalität

IntroductionThe effect of public reporting to improve quality in healthcare is reduced by the limited intelligibility of information about the quality of healthcare providers. This may result in worse health-related choices especially for older people and those with lower levels of education. There is, as yet, little information as to whether laymen understand the concepts behind quality comparisons and if this comprehension is correlated with hospital choices. MethodsAn instrument with 20 items was developed to analyze the intelligibility of five technical terms which were used in German hospital report cards to explain risk-adjusted death rates. Two online presentations of risk-adjusted death rates for five hospitals in the style of hospital report cards were developed. An online survey of 353 volunteers tested the comprehension of the risk-adjusted mortality rates and included an experimental hospital choice. ResultsThe intelligibility of five technical terms was tested: risk-adjusted, actual and expected death rate, reference range and national average. The percentages of correct answers for the five technical terms were in the range of 75.0-60.2%. Between 23.8% and 5.1% of the respondents were not able to answer the question about the technical term itself. The least comprehensible technical terms were “risk-adjusted death rate” and “reference range”. The intelligibility of the 20 items that were used to test the comprehension of the risk-adjusted mortality was between 89.5% and 14.2%. The two items that proved to be least comprehensible were related to the technical terms “risk-adjusted death rate” and “reference range”. For all five technical terms it was found that a better comprehension correlated significantly with better hospital choices. DiscussionWe found a better than average intelligibility for the technical terms “actual and expected death rate” and for “national average”. The least understandable were “risk-adjusted death rate” and “reference range”. Since the self-explanatory technical terms “actual and expected death rate” and “national average” are easy to understand and the comprehension is correlated with hospitals choices, we recommend using them for the presentation of measures which contain risk-adjusted mortality. The technical terms “risk-adjusted death rate” and “reference range” should stay in the background, since comprehension problems can be expected and explanations would have to be provided.

Patients' Awareness, Usage and Impact of Hospital Report Cards in the US.

Little knowledge is available about the importance of hospital report cards in the US from the patients' perspective. It also remains unknown whether specific report cards with a stronger emphasis on clinical measures have a greater impact on hospital choice than general report cards that focus on online-derived ratings. The aim of this study was to determine the awareness and usage of hospital report cards as well as their impact on hospital choice in the US. We conducted a cross-sectional study by surveying a stratified online sample (N=1332) to ensure representativeness to the US online population (February 2015). Overall, 75% of all respondents (mean age 45.4years; 54% female) were aware of hospital report cards. Among these, 56% had used a report card to search for a hospital, and 80% of report card users stated having been influenced by a report card. Both the awareness and usage of general report cards were shown to be higher than for specific report cards. No significant differences could be detected regarding the impact between general or specific report cards on hospital choice. Our results indicate that hospital report cards play a considerable role among patients when searching for a hospital in the US; however, patients do not seem to have a preference regarding the type of report cards they use when selecting a hospital.

Public reporting of hospital quality shows inconsistent ranking results

BackgroundEvidence from the US has demonstrated that hospital report cards might generate confusion for consumers who are searching for a hospital. So far, little is known regarding hospital ranking agreement on German report cards as well as underlying factors creating disagreement. ObjectiveThis study examined the consistency of hospital recommendations on German hospital report cards and discussed underlying reasons for differences. MethodsWe compared hospital recommendations for three procedures on four German hospital report cards. The agreement between two report cards was determined by Cohen’s-Kappa. Fleiss’ kappa was applied to evaluate the overlap across all four report cards. ResultsOverall, 43.40% of all hospitals were labeled equally as low, middle, or top performers on two report cards (hip replacement: 43.2%; knee replacement: 42.8%; percutaneous coronary intervention: 44.3%). In contrast, 8.5% of all hospitals were rated a top performer on one report card and a low performer on another report card. The inter-report card agreement was slight at best between two report cards (κmax=0.148) and poor between all four report cards (κmax=0.111). ConclusionsTo increase the benefit of public reporting, increasing the transparency about the concept of – medical – “quality” that is represented on each report card seems to be important. This would help patients and other consumers use the report cards that most represent one’s individual preferences.

Hospital Quality Reporting in the United States: Does Report Card Design and Incorporation of Patient Narrative Comments Affect Hospital Choice?

To explore the impact of hospital report card design and incorporation of patient narrative comments on consumers' choices of hospitals. Primary data collected from an online survey with 1,350 respondents in February, 2015. A randomized 2 (narrative comments: yes, no)×3 (design: representation of clinical performance in textual, star, numerical formats) between-subject online-based cross-sectional experiment. In 51 percent of all cases, respondents selected the hospital with the best clinical results. Report cards with a numerical design induced choices more focused on clinical ratings (56.0 percent chose the highest rated hospital) than those with textual information (48.1 percent) or star ratings (47.3 percent) (p<.001). Report cards without narrative comments (49.7 percent) and with narratives (51.4 percent) were not associated with significant difference in selecting top-rated clinical hospitals (p=.376). But there were significant interactions affecting choice of hospitals among exposure to narratives, formatting of clinical performance, and respondents' education. Consumers have a difficult time synthesizing quality data in various formats. Hospital report cards continue to pose challenging choices, especially for those with limited education. Narrative comments in their earliest emerging forms do not seem to be altering hospital choice as much as the literature has suggested for other providers, but they may have consequential impact on the choices of certain subsets of consumers.

Calculations of Financial Incentives for Providers in a Pay-for-Performance Program: Manual Review Versus Data From Structured Fields in Electronic Health Records.

Hospital report cards and financial incentives linked to performance require clinical data that are reliable, appropriate, timely, and cost-effective to process. Pay-for-performance plans are transitioning to automated electronic health record (EHR) data as an efficient method to generate data needed for these programs. To determine how well data from automated processing of structured fields in the electronic health record (AP-EHR) reflect data from manual chart review and the impact of these data on performance rewards. Cross-sectional analysis of performance measures used in a cluster randomized trial assessing the impact of financial incentives on guideline-recommended care for hypertension. A total of 2840 patients with hypertension assigned to participating physicians at 12 Veterans Affairs hospital-based outpatient clinics. Fifty-two physicians and 33 primary care personnel received incentive payments. Overall, positive and negative agreement indices and Cohen's kappa were calculated for assessments of guideline-recommended antihypertensive medication use, blood pressure (BP) control, and appropriate response to uncontrolled BP. Pearson's correlation coefficient was used to assess how similar participants' calculated earnings were between the data sources. By manual chart review data, 72.3% of patients were considered to have received guideline-recommended antihypertensive medications compared with 65.0% by AP-EHR review (κ=0.51). Manual review indicated 69.5% of patients had controlled BP compared with 66.8% by AP-EHR review (κ=0.87). Compared with 52.2% of patients per the manual review, 39.8% received an appropriate response by AP-EHR review (κ=0.28). Participants' incentive payments calculated using the 2 methods were highly correlated (r≥0.98). Using the AP-EHR data to calculate earnings, participants' payment changes ranged from a decrease of $91.00 (-30.3%) to an increase of $18.20 (+7.4%) for medication use (interquartile range, -14.4% to 0%) and a decrease of $100.10 (-31.4%) to an increase of $36.40 (+15.4%) for BP control or appropriate response to uncontrolled BP (interquartile range, -11.9% to -6.1%). Pay-for-performance plans that use only EHR data should carefully consider the measures and the structure of the EHR before data collection and financial incentive disbursement. For this study, we feel that a 10% difference in the total amount of incentive earnings disbursed based on AP-EHR data compared with manual review is acceptable given the time and resources required to abstract data from medical records.

Internet-based Report Cards for Hospital Choice Making in Germany: A Clinical Area-focused Perspective

Objectives: Aims of this study are to provide (i) a presentation of quality of care information about German hospitals on internet report cards, and (ii) derive a guidance tool for using certain report cards. Methods: We undertook (i) a systematic internet search to identify hospital report cards, (ii) whereby several inclusion and exclusion criteria were applied, (iii) We developed a scheme to describe the presented quality information, (iv) We also analysed the report cards and (v) evaluated the results and derived further usage guidance. Results: 18 report cards were included. 8 report cards present nationwide quality information while ten report cards focus on regional areas. The AOK Krankenhaus-navigator provides the most information about 5 clinical areas of the external quality assurance (§ 137 SGB V). Regarding the remaining 26 clinical areas, the TK Klinikführer, the Barmer Krankenhausnavi, and the Weisse Liste provide the largest amount of quality information. Qualitätskliniken.de reports on 9, the Klinikführer Rheinland on 8, the Bremer Krankenhausspiegel on 4, and the Hamburger Krankenhausspiegel on 2 additional clinical areas. Conclusions: This study shows that a differentiated view for each clinical area is necessary when searching for information about the quality of hospitals. Further studies should assess the quality of the report cards as well as the information that they display.

Abstract 213: Treatment Delays in Patients with ST-Elevation Myocardial Infarction Across All Hospitals Performing Primary Percutaneous Coronary Intervention in Quebec, Canada: Results of a Third Field Evaluation

Background: Our government-funded independent cardiology evaluation unit conducted a third systematic province-wide field evaluation of ST-elevation myocardial infarction (STEMI) in collaboration with an interdisciplinary committee of clinical experts. Primary percutaneous coronary intervention (PPCI) is the predominant reperfusion treatment for STEMI in Quebec and in 2 previous evaluations (2006-7; 2008-9), there were wide variations in treatment delays across the 13 PPCI hospitals. Initiatives to improve delays have included individualized hospital report cards, presentations and discussions of inter-regional analyses with clinicians and decision-makers and the province-wide implantation of an ambulance (prehospital) ECG program. Our objective was to describe the differences in STEMI processes of care over time that these initiatives may have produced. Methods: Medical charts of all patients with a final diagnosis of acute myocardial infarction who presented with characteristic symptoms to an emergency department (ED) during a 6-month period in 2013-2014 were reviewed. Trained medical record librarians collected data on clinical characteristics and time points of care. An automated algorithm was used to identify STEMI. All patients with STEMI who presented directly to the ED of one of the 13 PPCI centers (by ambulance or other means) and were treated with PPCI were included in this preliminary analysis. ECGs were reviewed at our core laboratory. Results: During the 6-month study period, 735 patients with STEMI presented directly to the ED of a PPCI hospital (including ambulance bypass of local non-PPCI hospitals) and were treated with PPCI. Center patient volume varied from 29 to 117. Over 80% (596/735) of the patients arrived by ambulance (up from 71% in 2009-9) and of these, 93% (552/596) had a prehospital ECG compared with 20% in 2008-9. The median ED door-in-door-out delay was 26 min (10-90th percentile: 8-72) compared with 49 min (17-126) in 2008-9 (p=0.001). The median door-to-device delay was 59 min (30-125), compared with 77 min (38-163) during 2008-9 (p=0.017), and varied from a median of 38 to 87 min across the 13 PPCI hospitals. For patients arriving by ambulance with documented first medical contact time, 48% (260/541) had a first medical contact-to-device time ≤ 90 min compared with 35% in 2008-9 (p=0.06). Conclusions: This third province-wide evaluation shows important improvements in Quebec’s system of STEMI care. Since 2008-9, ED door-in-door-out delay improved by 47% and door-to-device delay for direct admission PPCI improved by 23%. These improvements have occurred after local, regional and provincial efforts to monitor and reduce treatment delays and the implantation of a province-wide prehospital ECG program. However, the wide variation in delays across PPCI centers underlines the need for continued evaluation and improvement initiatives.

Text Comprehensibility of Hospital Report Cards

Objectives: Recently, the number of hospital report cards that compare quality of hospitals and present information from German quality reports has greatly increased. Objectives of this study were to a) identify suitable methods for measuring the readability and comprehensibility of hospital report cards, b) to obtain reliable information on the comprehensibility of texts for laymen, c) to give recommendations for improvements and d) to recommend public health actions. Methods: The readability and comprehensibility of the texts were tested with a) a computer-aided evaluation of formal text characteristics (readability indices Flesch (German formula) and 1. Wiener Sachtextformel formula), b) an expert-based heuristic analysis of readability and comprehensibility of texts (counting technical terms and analysis of text simplicity as well as brevity and conciseness using the Hamburg intelligibility model) and c) a survey of subjects about the comprehensibility of individual technical terms, the assessment of the comprehensibility of the presentations and the subjects' decisions in favour of one of the 5 presented clinics due to the better quality of data. In addition, the correlation between the results of the text analysis with the results from the survey of subjects was tested. Results: The assessment of texts with the computer-aided evaluations showed poor comprehensibility values. The assessment of text simplicity using the Hamburg intelligibility model showed poor comprehensibility values (-0.3). On average, 6.8% of the words used were technical terms. A review of 10 technical terms revealed that in all cases only a minority of respondents (from 4.4% to 39.1%) exactly knew what was meant by each of them. Most subjects (62.4%) also believed that unclear terms worsened their understanding of the information offered. The correlation analysis showed that presentations with a lower frequency of technical terms and better values for the text simplicity were better understood. Conclusion: The determination of the frequency of technical terms and the assessment of text simplicity using the Hamburg intelligibility model were suitable methods to determine the readability and comprehensibility of presentations of quality indicators. The analysis showed predominantly poor comprehensibility values and indicated the need to improve the texts of report cards.

Information presentation features and comprehensibility of hospital report cards: design analysis and online survey among users.

BackgroundImproving the transparency of information about the quality of health care providers is one way to improve health care quality. It is assumed that Internet information steers patients toward better-performing health care providers and will motivate providers to improve quality. However, the effect of public reporting on hospital quality is still small. One of the reasons is that users find it difficult to understand the formats in which information is presented.ObjectiveWe analyzed the presentation of risk-adjusted mortality rate (RAMR) for coronary angiography in the 10 most commonly used German public report cards to analyze the impact of information presentation features on their comprehensibility. We wanted to determine which information presentation features were utilized, were preferred by users, led to better comprehension, and had similar effects to those reported in evidence-based recommendations described in the literature.MethodsThe study consisted of 5 steps: (1) identification of best-practice evidence about the presentation of information on hospital report cards; (2) selection of a single risk-adjusted quality indicator; (3) selection of a sample of designs adopted by German public report cards; (4) identification of the information presentation elements used in public reporting initiatives in Germany; and (5) an online panel completed an online questionnaire that was conducted to determine if respondents were able to identify the hospital with the lowest RAMR and if respondents’ hospital choices were associated with particular information design elements.ResultsEvidence-based recommendations were made relating to the following information presentation features relevant to report cards: evaluative table with symbols, tables without symbols, bar charts, bar charts without symbols, bar charts with symbols, symbols, evaluative word labels, highlighting, order of providers, high values to indicate good performance, explicit statements of whether high or low values indicate good performance, and incomplete data (“N/A” as a value). When investigating the RAMR in a sample of 10 hospitals’ report cards, 7 of these information presentation features were identified. Of these, 5 information presentation features improved comprehensibility in a manner reported previously in literature.ConclusionsTo our knowledge, this is the first study to systematically analyze the most commonly used public reporting card designs used in Germany. Best-practice evidence identified in international literature was in agreement with 5 findings about German report card designs: (1) avoid tables without symbols, (2) include bar charts with symbols, (3) state explicitly whether high or low values indicate good performance or provide a “good quality” range, (4) avoid incomplete data (N/A given as a value), and (5) rank hospitals by performance. However, these findings are preliminary and should be subject of further evaluation. The implementation of 4 of these recommendations should not present insurmountable obstacles. However, ranking hospitals by performance may present substantial difficulties.

Abstract 122: Impact of Using Clinical Data to Profile Hospitals for 30-day Stroke Mortality: Data from the Ontario Stroke Audit

Background: Risk-adjustment models that use administrative data may not fully account for case-mix differences between hospitals. Supplementing these models with clinical data may improve their accuracy. We examined the impact of using data from a clinical registry in addition to administrative data on profiling Ontario hospitals for 30-day stroke mortality. Methods: Ontario Stroke Audit (OSA) data on 7006 acute stroke cases admitted to 54 Ontario hospitals between 2010-11 were obtained. We calculated 30-day risk-standardized mortality (RSMR) from an established risk adjustment model that relied on only hospital discharge data. This model (termed ADMIN) included age, sex, past medical history (including atrial fibrillation, stroke, diabetes, hypertension, heart disease), ambulance arrival, and stroke-type. We then modified this model to create a CLINICAL model by adding statistically significant clinical variables from the OSA data. In addition to the ADMIN model variables, the CLINICAL model included dysphagia, cognitive status, pre-stroke function, and stroke severity. We compared model performance (c-statistics, calibration [Hosmer-Lemeshow chi-square statistic (HLCS], Integrated Discrimination Improvement [IDI]), and hospital rankings (based on RSMR) using Spearman rank correlation. Results: Model performance was noticeably better for the CLINICAL model compared to the ADMIN model; discrimination was higher (c-statistic 0.82 vs. 0.70), calibration better (HLCS p= 0.08 vs &lt;0.01) and predictive accuracy greater (IDI= 0.13, p &lt;0.001). Comparison of RSMRs between the two models showed only moderate correlation (r = 0.653) with substantial variation for some hospitals (Figure). Conclusions: The addition of variables from clinical data to an established administrative data model had a marked effect on hospital rankings. These results suggest that clinical variables could have a meaningful impact on the accuracy of hospital report cards.