Raw Agreement Research Articles

Optimizing identification of resistant hypertension: Computable phenotype development and validation.

Computable phenotypes are constructed to utilize data within the electronic health record (EHR) to identify patients with specific characteristics; a necessary step for researching a complex disease state. We developed computable phenotypes for resistant hypertension (RHTN) and stable controlled hypertension (HTN) based on the National Patient-Centered Clinical Research Network (PCORnet) common data model (CDM). The computable phenotypes were validated through manual chart review. We adapted and refined existing computable phenotype algorithms for RHTN and stable controlled HTN to the PCORnet CDM in an adult HTN population from the OneFlorida Clinical Research Consortium (2015-2017). Two independent reviewers validated the computable phenotypes through manual chart review of 425 patient records. We assessed precision of our computable phenotypes through positive predictive value (PPV) and test validity through interrater reliability (IRR). Among the 156 730 HTN patients in our final dataset, the final computable phenotype algorithms identified 24 926 patients with RHTN and 19 100 with stable controlled HTN. The PPV for RHTN in patients randomly selected for validation of the final algorithm was 99.1% (n = 113, CI: 95.2%-99.9%). The PPV for stable controlled HTN in patients randomly selected for validation of the final algorithm was 96.5% (n = 113, CI: 91.2%-99.0%). IRR analysis revealed a raw percent agreement of 91% (152/167) with Cohen's kappa statistic = 0.87. We constructed and validated a RHTN computable phenotype algorithm and a stable controlled HTN computable phenotype algorithm. Both algorithms are based on the PCORnet CDM, allowing for future application to epidemiological and drug utilization based research.

Accuracy of Physicians in Differentiating Type 1 and Type 2 Myocardial Infarction Based on Clinical Information

BackgroundPhysicians commonly judge whether a myocardial infarction (MI) is type 1 (thrombotic) vs type 2 (supply/demand mismatch) based on clinical information. Little is known about the accuracy of physicians’ clinical judgement in this regard. We aimed to determine the accuracy of physicians’ judgement in the classification of type 1 vs type 2 MI in perioperative and nonoperative settings. MethodsWe performed an online survey using cases from the Optical Coherence Tomographic Imaging of Thrombus (OPTIMUS) Study, which investigated the prevalence of a culprit lesion thrombus based on intracoronary optical coherence tomography (OCT) in patients experiencing MI. Four MI cases, 2 perioperative and 2 nonoperative, were selected randomly, stratified by etiology. Physicians were provided with the patient’s medical history, laboratory parameters, and electrocardiograms. Physicians did not have access to intracoronary OCT results. The primary outcome was the accuracy of physicians' judgement of MI etiology, measured as raw agreement between physicians and intracoronary OCT findings. Fleiss’ kappa and Gwet’s AC1 were calculated to correct for chance. ResultsThe response rate was 57% (308 of 536). Respondents were 62% male; median age was 45 years (standard deviation ± 11); 45% had been in practice for > 15 years. Respondents’ overall accuracy for MI etiology was 60% (95% confidence interval [CI] 57%-63%), including 63% (95% CI 60%-68%) for nonoperative cases, and 56% (95% CI 52%-60%) for perioperative cases. Overall chance-corrected agreement was poor (kappa = 0.05), consistent across specialties and clinical scenarios. ConclusionsPhysician accuracy in determining MI etiology based on clinical information is poor. Physicians should consider results from other testing, such as invasive coronary angiography, when determining MI etiology.

Effects of adopting the Global Lung Function Initiative 2017 reference equations on the interpretation of carbon monoxide transfer factor.

The recently published Global Lung Function Initiative (GLI) carbon monoxide transfer factor (T LCO) reference equations provide an opportunity to adopt a current, all-age, widely applicable reference set. The aim of this study was to document the effect of changing to GLI from commonly utilised reference equations on the interpretation of T LCO results.33 863 T LCO results (48% female, 88% Caucasian, n=930 aged <18 years) from clinical pulmonary function laboratories within three Australian teaching hospitals were analysed. The lower limit of normal (LLN) and proportion of patients with a T LCO below this value were calculated using GLI and other commonly used reference equations.The average T LCO LLN for GLI was similar or lower than the other equations, with the largest difference seen for Crapo equations (median: -1.25, IQR: -1.64, -0.86 mmol·min-1·kPa-1). These differences resulted in altered rates of reduced T LCO for GLI particularly for adults (+1.9% versus Miller to -27.6% versus Crapo), more so than for children (-0.8% versus Kim to -14.2% versus Cotes). For adults, the highest raw agreement for GLI was with Miller equations (94.7%), while for children it was with Kim equations (98.1%). Results were reclassified from abnormal to normal more frequently for younger adults, and for adult females, particularly when moving from Roca to GLI equations (30% of females versus 16% of males).The adoption of GLI T LCO reference equations in adults will result in altered interpretation depending on the equations previously used and to a greater extent in adult females. The effect on interpretation in children is less significant.

Validation of a Simple Tool for Electronic Documentation of Behavioral Responses to Anesthesia Induction.

Anxiety and distress behaviors during anesthesia induction are associated with negative postoperative outcomes for pediatric patients. Documenting behavioral responses to induction is useful to evaluate induction quality at hospitals and to optimize future anesthetics for returning patients, but we lack a simple tool for clinical documentation. The Induction Compliance Checklist is a tool for grading induction behaviors that is well validated for research purposes, but it is not practical for routine documentation in busy clinical practice settings. The Child Induction Behavioral Assessment tool was developed to provide a simple and easy to use electronic tool for clinical documentation of induction behaviors. The aim of this study was to test the Child Induction Behavioral Assessment tool's concurrent validity with the Induction Compliance Checklist and the interrater reliability. This prospective, observational study included 384 pediatric patients undergoing anesthesia inhalation induction. Concurrent validity with the Induction Compliance Checklist and interrater reliability of the Child Induction Behavioral Assessment were evaluated. Two researchers alternated scoring the Induction Compliance Checklist. The 2 researchers independently scored the Child Induction Behavioral Assessment. The anesthesia clinician caring for the patient also independently scored the Child Induction Behavioral Assessment by completing their routine documentation in the patient's medical record. Two age groups were evaluated (ages 1-3 and 4-12 years old). Clinicians' and researchers' Child Induction Behavioral Assessment scores demonstrated a strong correlation with the Induction Compliance Checklist (P < .0001). There was an excellent agreement between the 2 researchers' Child Induction Behavioral Assessment scores for the younger and older age groups, respectively (Kappa [95% CI] = 0.97 (0.94-0.99); K = 0.94 (0.89-0.99)]. The agreement between the researchers and the 117 clinicians who documented Child Induction Behavioral Assessment assessments in the medical record was good overall (intraclass correlation coefficient = 0.70), with fair agreement with the 1- to 3-year-old patients (intraclass correlation coefficient = 0.56) and good agreement for the 4- to 12-year-old patients (intraclass correlation coefficient = 0.74). The Child Induction Behavioral Assessment scale is a simple and practical electronic tool used to document pediatric behavioral responses to anesthesia inductions. This study provides evidence of the tool's validity and reliability for inhalation inductions. Future research is needed at other hospitals to confirm validity.

Inter-review agreement of risk-of-bias judgments varied in Cochrane reviews

ObjectivesThe objective of the study was to measure the level of agreement between Cochrane reviews of overlapping randomized controlled trials (RCTs) regarding risk-of-bias (RoB) judgments. Study Design and SettingOn November 5, 2017, the Cochrane Database of Systematic Reviews was searched for Cochrane reviews on tobacco. Reviews that included overlapping RCTs were included. RoB judgments were extracted from RoB tables using automated data scraping with manual verification and adjustments. Agreement between the reviews was calculated using Conger's generalized kappa coefficient (κ) and raw agreement (a). ResultsWe included 53 Cochrane reviews of 376 RCTs. For the RoB domain “random sequence generation,” the level of agreement between the reviews was substantial with κ = 0.63 (95% confidence interval: 0.56 to 0.71; a = 0.80). There was slight-to-moderate agreement between the reviews regarding the domains “allocation concealment”: κ = 0.51 (0.41 to 0.61), a = 0.75; “blinding”: κ = 0.19 (0.02 to 0.37), a = 0.52; “blinding of outcome assessment”: κ = 0.43 (0.14 to 0.72) a = 0.67; and “incomplete outcome data”: κ = 0.15 (−0.03 to 0.32), a = 0.64. For “blinding of participants and personnel” and “selective reporting”, κ could not be calculated. The raw agreement was 0.40 and 0.42, respectively. ConclusionThe level of agreement between Cochrane reviews regarding RoB judgments ranged from slight to substantial depending on the RoB domain. Further investigations regarding reasons for variation and interventions to improve agreement are needed.

How to Validate Data Quality in a Trauma Registry? The Helsinki Trauma Registry Internal Audit.

Trauma registry data are used for analyzing and improving patient care, comparison of different units, and for research and administrative purposes. Data should therefore be reliable. The aim of this study was to audit the quality of the Helsinki Trauma Registry internally. We describe how to conduct a validation of a regional or national trauma registry and how to report the results in a readily comprehensible form. Trauma registry database of Helsinki Trauma Registry from year 2013 was re-evaluated. We assessed data quality in three different parts of the data input process: the process of including patients in the trauma registry (case completeness); the process of calculating Abbreviated Injury Scale (AIS) codes; and entering the patient variables in the trauma registry (data completeness, accuracy, and correctness). We calculated the case completeness results using raw agreement percentage and Cohen's κ value. Percentage and descriptive methods were used for the remaining calculations. In total, 862 patients were evaluated; 853 were rated the same in the audit process resulting in a raw agreement percentage of 99%. Nine cases were missing from the registry, yielding a case completeness of 97.1% for the Helsinki Trauma Registry. For AIS code data, we analyzed 107 patients with severe thorax injury with 941 AIS codes. Completeness of codes was 99.0% (932/941), accuracy was 90.0% (841/932), and correctness was 97.5% (909/932). The data completeness of patient variables was 93.4% (3899/4174). Data completeness was 100% for 16 of 32 categories. Data accuracy was 94.6% (3690/3899) and data correctness was 97.2% (3789/3899). The case completeness, data completeness, data accuracy, and data correctness of the Helsinki Trauma Registry are excellent. We recommend that these should be the variables included in a trauma registry validation process, and that the quality of trauma registry data should be systematically and regularly reviewed and reported.

Can a physiotherapy student assume the role of an advanced practice physiotherapist in Orthopaedic surgery triage? A prospective observational study

BackgroundAdvanced practice physiotherapists (APP) have helped improve accessibility to orthopaedic outpatient care. Several studies have validated the APP practice model in orthopaedic care, demonstrating high agreement between APPs and orthopaedic surgeons (OS) regarding diagnosis and management. However, as APPs tend to be experienced senior physiotherapists, such a study involving physiotherapy students (PS) has not yet been explored. The objective of this study was to evaluate the agreement for orthopaedic diagnoses and surgical triage between a PS and OSs.MethodsA prospective study involving a final year PS and seven OSs was conducted in a university hospital, after the PS had undergone a three-week intensive training. Eighty-six adult patients referred to OSs for knee osteoarthritis, hip osteoarthritis or shoulder problem were independently evaluated by the PS, and then re-evaluated by an OS. The diagnoses and surgical triage recommendations of both clinicians were analyzed for agreement using raw percent agreement and Cohen’s kappa. Patient satisfaction with the outpatient clinic experience was noted using a modified version of the Visit-Specific Satisfaction Instrument.ResultsOur sample consisted of 86 patients (mean age = 63.4 years). Reasons for consultation included shoulder problems (36%), knee osteoarthritis (52%) and hip osteoarthritis (12%). The raw percent agreement for diagnosis was 95.3%. The agreement for surgical triage was high (κ = 0.86, 95% CI: 0.74–0.98) with a raw agreement of 94.2%. Patient satisfaction was high.ConclusionsThe PS and OSs made similar diagnoses and triage recommendations suggesting that clinical experience alone is not a prerequisite for physiotherapists to help increase accessibility to orthopaedic care.

“The World Will Have a New Face”: Germans and the Post-World War I Global Economic Order

AbstractWorld War I reshaped the international economy. This was, in part, the consequence of British mobilization of resources for its own war effort, which aligned producer interests around the world with those of the United Kingdom. But it was also a consequence of Western policy aimed at excluding German businessmen from global markets. German planners noted during World War I that Great Britain, in particular, was expressing an interest in continuing such exclusion after the war, with potentially enormous economic consequences for Germany. Combatting or preventing such an economic “war after the war” prompted German businessmen and politicians to support a series of policies that would have profoundly changed the institutions and norms of the prewar international economy. These policies ranged from imposing one-sided trade agreements, expanding the mark zone, and establishing German control over Eastern European industries and infrastructure, to creating shipping cartels and imposing compulsory raw material delivery agreements on the Western powers. The result of German efforts to direct trade and investment in ways preferable to the German state would have been a deeply politicized postwar international economy. The article argues that economic questions were thus a central component of German war aims, but that these were not fixed: they evolved over the course of the war in response to changes in the international economy, and they focused not on short-term emergencies but rather on longer-term structural changes.

Diagnostic Yield and Accuracy of Bedside Echocardiography in the Emergency Department in Hemodynamically Stable Patients.

The goal of this study was to determine the diagnostic yield of focused cardiac ultrasound (FOCUS) in hemodynamically stable patients in the emergency department and secondarily to confirm the accuracy of these studies when compared to formal echocardiography. All hemodynamically stable adult patients who had an emergency physician-performed FOCUS examination completed over a 1-year period were identified using our electronic ultrasound database. Hemodynamic stability was defined as presenting systolic blood pressure higher than 90 mm Hg and not requiring any form of positive pressure ventilation. There were 1198 FOCUS examinations performed: 976 in hemodynamically stable patients who were included in our analysis. Twenty-seven percent of patients had new findings, including 154 (16%) new diagnoses of reduced left ventricular function, 105 (11%) new pericardial effusions, and 44 (5%) new diagnoses of RV dilatation. Dyspnea as an indication for the FOCUS examination was the strongest predictor of a positive study. Of patients included, 28% underwent formal echocardiography within 2 days and were analyzed for concordance with regard to left ventricular function and the presence of pericardial effusion. Of 270 studies, 208 were accurate, and 62 were inaccurate, for raw agreement of 77% (κ = 0.53). When stratified by sonographer experience, there was no impact on accuracy. Focused cardiac ultrasound in the emergency department for hemodynamically stable patients revealed new findings in 27% of studies, with a modest correlation with formal echocardiography. In stable patients, FOCUS has the potential for rapid diagnosis of cardiac disease, particularly in patients with dyspnea.

Abstract TP84: Agreement Between Alberta Stroke Program Early Computed Tomography Score and Computed Tomography Perfusion in Patient Selection for Mechanical Thrombectomy After Large Vessel Occlusion Acute Ischemic Stroke

Introduction: CT ASPECTS and CT Perfusion (CTP) are used to select patients for mechanical thrombectomy, an evidence-based treatment for Large Vessel Occlusion (LVO) Acute Ischemic Stroke (AIS). However, discordant results between the two imaging modalities creates uncertainty with respect to the volume of ischemic and infarcted brain tissue, and thus whether to offer revascularization. We sought to investigate the agreement between CT ASPECTS and CTP in selecting patients for mechanical thrombectomy. Hypothesis: CT ASPECTS determined by a neuro-radiologist demonstrates moderate agreement with CTP in selecting patients with anterior circulation, LVO AIS for mechanical thrombectomy. Methods: Over a 7-month period beginning in January 2018, we conducted a retrospective analysis from a large healthcare system’s stroke network database comparing the agreement between favorable CT ASPECTS (defined as score ≥ 6) and favorable CTP. Favorable CTP was defined according to the inclusion criteria from EXTEND-IA, DEFUSE 3, and DAWN, in the 0-6 hour, 6-16 hour, and 6-24 hour time windows, respectively, for patients with ICA or proximal MCA occlusions. Results: Cases were identified in the 0-24 hour window with an ICA or M1 occlusion, baseline CT ASPECTS calculated by a neuro-radiologist, and CTP. The overall raw agreement between CT ASPECTS and CTP for the 145 cases in the 0-6 hour window was 81%, and Cohen’s kappa (κ) was 0.17 (no agreement). In the 6-16 hour window, the overall raw agreement for 46 cases was 78% (κ = 0.38, minimal agreement). In the 6-24 hour window, the overall raw agreement for 58 cases was 53% (κ = 0.14, no agreement). Conclusions: In both early and extended time windows, CT ASPECTS and CTP demonstrate minimal to no agreement beyond chance in patient selection for mechanical thrombectomy. Additional studies are required to determine the most appropriate imaging selection criteria to guide treatment decisions in patients with anterior circulation, LVO AIS.

A computerized scoring system to improve assessment of heparin‐induced thrombocytopenia risk

Essentials Current risk scores for heparin-induced thrombocytopenia (HIT) are not computer-friendly. We compared a new computerized risk score with the 4Ts score in a large healthcare system. The computerized risk score agrees with the 4Ts score 85% of the time. The new score could potentially improve HIT diagnosis via incorporation into decision support. SUMMARY: Background (HIT) is an immune-mediated adverse drug event associated with life-threatening thrombotic complications. The 4Ts score is widely used to estimate the risk for HIT and guide diagnostic testing, but it is not easily amenable to computerized clinical decision support (CDS) implementation. Objectives Our main objective was to develop an HIT computerized risk (HIT-CR) scoring system that provides platelet count surveillance for timing and degree of thrombocytopenia to identify those for whom diagnostic testing should be considered. Our secondary objective was to evaluate clinical management and subsequent outcomes in those identified as being at risk for HIT. Methods We retrospectively analyzed data from a stratified sample of 150 inpatients treated with heparin to compare the performance of the HIT-CR scoring system with that of a clinically calculated 4Ts score. We took a 4Ts score of ≥4 as the gold standard to determine whether HIT diagnostic testing should be performed. Results The best cutoff point of the HIT-CR score was a score of 3, which yielded 85% raw agreement with the 4Ts score and a kappa of 0.69 (95% confidence interval0.57-0.81). Ninety percent of patients with 4Ts score of ≥4 failed to undergo conventionally recommended diagnostic testing; 38% of these experienced persistent, unexplained thrombocytopenia, and 4% suffered life-threatening thrombotic complications suggestive of undiagnosed HIT. Conclusion The HIT-CR scoring system is practical for computerized CDS, agrees well with the 4Ts score, and should be prospectively evaluated for its ability to identify patients who should be tested for HIT.

The forward-backward and dual-panel translation methods are comparable in producing semantic equivalent versions of a heart quality of life questionnaire.

To investigate semantic equivalence between two translated versions of the heart quality of life (HeartQoL) questionnaire produced by the forward-backward and dual-panel methods. The forward-backward and dual-panel versions of HeartQoL were self-administered among 60 participants who met the inclusion criteria of being a native Bahasa Malaysia-speaking Malay, aged 18 and older, having an indexed diagnosis of ischaemic heart disease and being cognitively fit. The administration sequence of the two versions was randomized. Additionally, three sociolinguists, who were blinded to translation processes and survey findings, rated the translated versions against the source version on three aspects of semantic equivalence. Textual content in both translated versions was considerably similar (n=9/14 items, ≈64%). The overall results from weighted kappa, raw agreement, intraclass correlations, and Wilcoxon signed-rank as well as experts' ratings were confirmative of semantic equivalence between the forward-backward and dual-panel versions of the HeartQoL. However, some mixed findings were indicative of potential gaps in both translated versions against the source version. Both the forward-backward and dual-panel methods produced semantically equivalent versions of HeartQoL; but translation alone is insufficient to narrow the subtle gaps caused by differences in culture and linguistic style.

An evaluation of diagnostic agreement rates between advanced practice physiotherapists and paediatric orthopaedic consultants for children with musculoskeletal complaints.

The purpose of the present study was to establish diagnostic agreement rates between orthopaedic consultants and advanced practice physiotherapists (APPs) for paediatric orthopaedic patients, examine the appropriateness of referrals by APPs to consultants and report on the surgical conversion rate (SCR). A retrospective review of all patients referred from an APP clinic to orthopaedic consultants was conducted for 2013, with a 3-year longitudinal follow-up through to the end of 2016. Study participants were two APPs and four orthopaedic consultants at a single site. Descriptive statistics report the proportion of appropriate onward referrals and SCRs. Raw proportion agreement and kappa coefficients were used to evaluate diagnostic agreement rates. The mean raw agreement was 82% (95% confidence interval = 73 to 87), with 12 of the 15 individual codes demonstrating agreement rates ≥90% (range 57-100). Good to excellent agreement was demonstrated in almost all categories based on the kappa coefficient. Eighty-seven per cent of referrals from APPs were deemed to be appropriate by the orthopaedic consultants. The SCR was 23%. This was the first study to evaluate diagnostic agreement rates between APPs and orthopaedic consultants, the appropriateness of onward referral by APPs and the SCR for paediatric musculoskeletal patients referred from an APP clinic. Good to excellent diagnostic agreement was observed for routine elective paediatric orthopaedic patients, in tandem with agreement levels reported in the adult literature. The study demonstrates that APPs are clinically effective in the diagnosis and onward referral of paediatric orthopaedic patients in a triage setting.

Patients with cystic echinococcosis in the three national referral centers of Mongolia: A model for CE management assessment.

BackgroundMongolia is one of the endemic countries for cystic echinococcosis (CE), a zoonotic disease caused by the larval stage of Echinococcus granulosus. The goal of this study is to describe the current clinical management of CE in Mongolia, to capture the distribution of cyst stages of patients treated, and to contrast current practice with WHO-IWGE expert consensus.MethodsHospital records of CE patients treated between 2008 and 2015 at the three state hospitals and fulfilling the inclusion criterion ‘discharge diagnosis CE’ (ICD 10 code B.67.0–67.9) were reviewed. Demographical, geographical, clinical and ultrasonography (US) data were extracted and analyzed. The annual surgical incidence was estimated. The digital copies of US cyst images were independently staged by three international experts following the WHO CE cyst classification to determine the proportions of patients which ideally would have been assigned to the WHO recommended treatment modalities surgery, percutaneous, medical (benzimidazole) treatment and watch & wait.ResultsA total of 290 patient records fulfilled the inclusion criteria of the study. 45.7% of patients were below 15 years of age. 73.7% of CE cysts were located in abdominal organs, predominantly liver. US images of 84 patients were staged and assessed for interrater-agreement. The average raw agreement was 77.2%. Unweighted Kappa coefficient and weighted Kappa was 0.57 and 0.59, respectively. Mean proportions of images judged as stages CE1, CE2, CE3a, CE3b, CE4 and CL were 0.59, 0.01, 0.19, 0.08, 0.03 and 0.11, respectively. 40 cysts met the inclusion criteria of treatment modality analysis. The mean proportions of cases with a single cyst assigned to medical, percutaneous treatment, surgery and watch & wait were 52.5% (95% CI 42–65), 25.8% (95% CI 15–30), 5.1% (95% CI 0–10) and 3.3% (95% CI 0–10), respectively. 13.3% (95% CI 5–25) of cysts were staged as CL and therefore assigned to further diagnostic requirement.ConclusionWHO CE cyst classification and WHO-IWGE expert consensus on clinical CE management is not implemented in Mongolia. This results in exclusively surgical treatment, an unnecessary high risk approach for the majority of patients who could receive medical, percutaneous treatment or observation (watch & wait). Introduction of WHO-IWGE expert consensus and training in ultrasound CE cyst staging would be highly beneficial for patients and the health care services.

Cytology and direct human papillomavirus testing on fine needle aspirates from cervical lymph node metastases of patients with oropharyngeal squamous cell carcinoma or occult primary.

Cervical lymph node fine needle aspirates (FNAs) may represent the only specimens available for an initial characterisation of patients with lymphadenopathy. Morphology and human papillomavirus (HPV) DNA presence were evaluated in FNAs collected from patients with oropharyngeal squamous cell carcinoma (OPSCC) or cancer of unknown primary (CUP). FNA HPV results were compared with those of the respective formalin-fixed paraffin-embedded (FFPE) primary cancer. Liquid-based cytology was performed on FNAs collected in PreservCyt. HPV-DNA was analysed by the INNO-LiPA HPV genotyping Extra II on both cytological and FFPE samples. The CINtec® Histology Kit was used to assess p16 expression in cancer tissues. Forty-seven FNAs were collected from OPSCC and 16 from CUP patients. Cancer cells were found in 35/47 cases (74.5%), while 11 (23.4%) showed only necrosis and one (2.1%) was negative for malignancy. HPV-DNA was detected in 30/47 FNAs (63.8%), mostly harbouring HPV16 (90.0%). An excellent agreement was observed between the FNA and corresponding FFPE HPV status (raw agreement: 97.5%; Cohen κ: 0.94). The HPV test result on the necrotic FNAs completely matched that of the respective primary cancer. FNA HPV testing correctly identified 26/27 HPV-driven OPSCCs (96.3%). HPV was detected in nine of 16 FNAs (56.2%) from CUP patients. HPV status of metastatic cervical lymph node FNAs reflects that of the corresponding primary OPSCCs even when cell integrity in the FNA is not preserved and only necrotic debris are present. In patients with initial CUP, HPV-positivity on the FNA may guide the diagnostic workup and therapeutic management, since it suggests an oropharyngeal origin.

Fallacies of Agreement

Discovering gestures that gain consensus is a key goal of gesture elicitation. To this end, HCI research has developed statistical methods to reason about agreement. We review these methods and identify three major problems. First, we show that raw agreement rates disregard agreement that occurs by chance and do not reliably capture how participants distinguish among referents. Second, we explain why current recommendations on how to interpret agreement scores rely on problematic assumptions. Third, we demonstrate that significance tests for comparing agreement rates, either within or between participants, yield large Type I error rates (&gt;40% for α =.05). As alternatives, we present agreement indices that are routinely used in inter-rater reliability studies. We discuss how to apply them to gesture elicitation studies. We also demonstrate how to use common resampling techniques to support statistical inference with interval estimates. We apply these methods to reanalyze and reinterpret the findings of four gesture elicitation studies.

Diagnostic validity and triage concordance of a physiotherapist compared to physicians\u2019 diagnoses for common knee disorders

BackgroundEmergence of more autonomous roles for physiotherapists warrants more evidence regarding their diagnostic capabilities. Therefore, we aimed to evaluate diagnostic and surgical triage concordance between a physiotherapist and expert physicians and to assess the diagnostic validity of the physiotherapist’s musculoskeletal examination (ME) without imaging.MethodsThis is a prospective diagnostic study where 179 consecutive participants consulting for any knee complaint were independently diagnosed and triaged by two evaluators: a physiotherapist and one expert physician (orthopaedic surgeons or sport medicine physicians). The physiotherapist completed only a ME, while the physicians also had access to imaging to make their diagnosis. Raw agreement proportions and Cohen’s kappa (k) were calculated to assess inter-rater agreement. Sensitivity (Se) and specificity (Sp), as well as positive and negative likelihood ratios (LR+/−) were calculated to assess the validity of the ME compared to the physicians’ composite diagnosis.ResultsPrimary knee diagnoses included anterior cruciate ligament injury (n = 8), meniscal injury (n = 36), patellofemoral pain (n = 45) and osteoarthritis (n = 79). Diagnostic inter-rater agreement between the physiotherapist and physicians was high (k = 0.89; 95% CI:0.83–0.94). Inter-rater agreement for triage recommendations of surgical candidates was good (k = 0.73; 95% CI:0.60–0.86). Se and Sp of the physiotherapist’s ME ranged from 82.0 to 100.0% and 96.0 to 100.0% respectively and LR+/− ranged from 23.2 to 30.5 and from 0.03 to 0.09 respectively.ConclusionsThere was high diagnostic agreement and good triage concordance between the physiotherapist and physicians. The ME without imaging may be sufficient to diagnose or exclude common knee disorders for a large proportion of patients. Replication in a larger study will be required as well as further assessment of innovative multidisciplinary care trajectories to improve care of patients with common musculoskeletal disorders.

Evaluation of the Xpert® HPV assay in the detection of Human Papillomavirus in formalin-fixed paraffin-embedded oropharyngeal carcinomas

ObjectivesThe increasing incidence of HPV-related Oropharyngeal Squamous Cell Carcinoma (OPSCC) and the improved survival of HPV-positive OPSCC highlight the need for effective tools in evaluating HPV status on formalin-fixed paraffin-embedded (FFPE) cancers. To date, there is no agreement regarding the most appropriate method for HPV testing on FFPE materials. We aimed to investigate the performance of the Xpert® HPV assay (Cepheid) on crude lysates from OPSCC FFPE tissues. Materials and methodsCrude lysates were obtained by proteinase K digestion of FFPE tissues that had already been analyzed by the INNO-LiPA HPV assay and p16ink4a immunostaining. Results159 FFPE OPSCCs were evaluated. All the samples provided valid results with the Xpert, whereas three samples (1.8%) were invalid using the INNO-LiPA. Among the remaining 156 cases, 65 (41.7%) were concordantly positive and 87 (55.8%) concordantly negative (raw agreement 0.97, 95% CI: 0.93–0.99; Cohen K 0.95, 95% CI: 0.90–0.99). Type-specific data for the cases that were positive by both methods were completely concordant. Three samples were HPV16-positive with Xpert but negative with INNO-LiPA, while one OPSCC tested negative with Xpert and positive with INNO-LiPA. A very good agreement was observed between the Xpert and the p16 results, which was slightly higher than that for INNO-LiPA (Cohen K 0.87vs. 0.85). ConclusionThe Xpert HPV assay appears to be a very good method for HPV detection and genotyping on FFPE OPSCCs, and requires no prior purification of nucleic acids. This assay showed a very good agreement with INNO-LiPA and p16 findings.

Validation of a Brief Questionnaire Against Direct Observation to Assess Adolescents' School Lunchtime Beverage Consumption

ObjectiveBeverage consumption is an important determinant of youth health outcomes. Beverage interventions often occur in schools, yet no brief validated questionnaires exist to assess whether these efforts improve in-school beverage consumption. This study validated a brief questionnaire to assess beverage consumption during school lunch. MethodsResearchers observed middle school students' (n = 25) beverage consumption during school lunchtime using a standardized tool. After lunch, students completed questionnaires regarding their lunchtime beverage consumption. Kappa statistics compared self-reported with observed beverage consumption across 15 beverage categories. ResultsEight beverages showed at least fair agreement (kappa [κ] > 0.20) for both type and amount consumed, with most showing substantial agreement (κ > 0.60). One beverage had high raw agreement but κ < 0.20. Six beverages had too few ratings to compute κ's. Conclusions and ImplicationsThis brief questionnaire was useful for assessing school lunchtime consumption of many beverages and provides a low-cost tool for evaluating school-based beverage interventions.

Automating the Determination of Prostate Cancer Risk Strata From Electronic Medical Records.

Risk stratification underlies system-wide efforts to promote the delivery of appropriate prostate cancer care. Although the elements of risk stratum are available in the electronic medical record, manual data collection is resource intensive. Therefore, we investigated the feasibility and accuracy of an automated data extraction method using natural language processing (NLP) to determine prostate cancer risk stratum. Manually collected clinical stage, biopsy Gleason score, and preoperative prostate-specific antigen (PSA) values from our prospective prostatectomy database were used to categorize patients as low, intermediate, or high risk by D'Amico risk classification. NLP algorithms were developed to automate the extraction of the same data points from the electronic medical record, and risk strata were recalculated. The ability of NLP to identify elements sufficient to calculate risk (recall) was calculated, and the accuracy of NLP was compared with that of manually collected data using the weighted Cohen's κ statistic. Of the 2,352 patients with available data who underwent prostatectomy from 2010 to 2014, NLP identified sufficient elements to calculate risk for 1,833 (recall, 78%). NLP had a 91% raw agreement with manual risk stratification (κ = 0.92; 95% CI, 0.90 to 0.93). The κ statistics for PSA, Gleason score, and clinical stage extraction by NLP were 0.86, 0.91, and 0.89, respectively; 91.9% of extracted PSA values were within ± 1.0 ng/mL of the manually collected PSA levels. NLP can achieve more than 90% accuracy on D'Amico risk stratification of localized prostate cancer, with adequate recall. This figure is comparable to other NLP tasks and illustrates the known trade off between recall and accuracy. Automating the collection of risk characteristics could be used to power real-time decision support tools and scale up quality measurement in cancer care.