Linear Weighted Kappa Research Articles

Reliability and validity of DSM-IV and DSM-5 methamphetamine use disorder diagnoses using the Chinese Version of the Semi-Structured Assessment for Drug Dependence and Alcoholism (SSADDA)

BackgroundThe Semi-structured Assessment for Drug Dependence and Alcoholism (SSADDA) was developed to assess substance-use disorders and other psychiatric traits. We translated the SSADDA into Chinese and evaluated its inter-rater reliability and concurrent validity in diagnosing DSM-IV methamphetamine (MA) dependence and DSM-5 MA-use disorder (MUD). MethodsThe sample comprised 231 participants who were interviewed using the Chinese SSADDA and the Mini-International Neuropsychiatric Interview (Chinese MINI) for concurrent validation. Of the 231 participants, 191 were interviewed by two different interviewers two weeks apart. We evaluated the inter-rater reliability and concurrent validity of the diagnoses using percent agreement and Cohen's kappa coefficient (κ). Cohen’s linear weighted kappa was used to assess the reliability of DSM-5 MUD severity. ResultsIt showed good inter-rater reliability and no significant differences among the DSM-5 MUD (κ = 0.71), DSM-IV MA abuse or dependence (κ = 0.72), and the DSM-IV diagnoses of MA dependence (κ = 0.66) and abuse (κ = 0.68) tested separately. The weighted kappa was 0.67 across the three DSM-5 MUD severity levels. The reliability of each individual diagnostic criterion for DSM-5 MUD ranged from fair to excellent (κ = 0.41–0.80), except for “repeated attempts to quit/control use” (κ = 0.38). The concurrent validity based on MINI-derived diagnoses ranged from good to excellent (κ = 0.65–0.78). ConclusionsThis study shows that the Chinese version of SSADDA has good reliability and validity among Chinese MA users.

Masked comparison of trypan blue stain and potassium hydroxide with calcofluor white stain in the microscopic examination of corneal scrapings for the diagnosis of microbial keratitis

Purpose:To evaluate the efficacy of trypan blue in direct microscopic examination of corneal scrapings in the diagnosis of non-viral microbial keratitis.Methods:In a prospective, interventional, masked study, 82 consecutive patients were investigated. Direct microscopic examination of the corneal scrapings involved three smears stained with potassium hydroxide with calcofluor white (KOH + CFW), Gram stain (not analyzed), and trypan blue stain and culture for bacteria, fungus, and Acanthamoeba. While KOH + CFW stained slides were examined under a fluorescence microscope, trypan blue-stained slides were examined by two microbiologists (masked to KOH + CFW and culture results) under normal light microscopy. Thirty samples were reexamined for interobserver and intraobserver variability.Results:Out of 82 samples, fungal/fungus-like elements were seen in 48 (58.5%) in KOH + CFW and 38 (46.3%) in trypan blue stain. One microsporidial case, detected in KOH + CFW was negative in trypan blue stain (culture not done). Fungal growth was positive in 23 out of 81 (28.3%) cases cultured, single bacterial species in 18 (22.2%), Pythium insidiosum in three, mixed bacteria and fungus/Pythium in 7 (8.6%), Acanthamoeba in none and 30 (37.0%) samples were sterile. With culture as gold standard, the respective sensitivity (84.9%, 75.7%) and specificity (90.9%, 68.2%) of KOH + CFW stain and trypan blue stain were comparable (p = 0.16, P = 0.06). The interobserver linear weighted kappa score between the two microbiologists was 1.00 while it was 0.86 for intraobserver agreement.Conclusion:Trypan blue stain, an easily available dye to ophthalmologists, is highly efficacious in the diagnosis of fungal keratitis.

Abbreviated Magnetic Resonance Imaging With Breath-Hold Three-Dimensional Magnetic Resonance Cholangiopancreatography: Assessment of Malignant Risk of Pancreatic Intraductal Papillary Mucinous Neoplasm.

For surveillance of pancreatic intraductal papillary mucinous neoplasms (IPMNs), magnetic resonance imaging (MRI)/magnetic resonance cholangiopancreatography (MRCP) is preferred over computed tomography, but the long acquisition time limits its use. To investigate the diagnostic performance of abbreviated MRI with breath-hold (BH) three-dimensional MRCP (abbreviated MRI-BH) for malignant risk assessment of pancreatic IPMN. Retrospective. Two hundred and thirty-five patients with IPMNs (M:F=115:120; mean age ± SD, 66 ± 9 years; typical imaging features with ≥2-year stability [N=172] and histopathologically confirmed [N=63]). 3.0 T/ abbreviated MRI-BH (single-shot fast spin-echo, T1W fat-suppressed gradient-echo sequence, and BH-3D-MRCP). Abbreviated MRI-BH was reviewed by three reviewers, and its diagnostic performance was assessed using the predetermined scoring system. The diagnostic performance for the mural nodule detection was assessed. Additionally, diagnostic performance of abbreviated MRI was compared with that of full-sequence MRI. Area under the receiver operating characteristic curve (AUC) with z-test, and linear-weighted kappa values. Thirty-five patients had malignant IPMN. At a cut-off score ≥3, AUCs of abbreviated MRI-BH for detecting malignant IPMN were 0.959 for reviewer 1, 0.962 for reviewer 2, and 0.956 for reviewer 3. The sensitivity of reviewers 1, 2, and 3 was 97.1% for all, and the specificity was 85.5%, 86.0%, and 85.0%, respectively. Regarding mural nodule detection (N=22), abbreviated MRI-BH demonstrated a sensitivity of 95.5% and a specificity of 88.3% for reviewer 1, a sensitivity of 86.4% and a specificity of 92.0% for reviewer 2, and a sensitivity of 86.4% and a specificity of 89.2% for reviewer 3. There were no significant differences between AUC of abbreviated MRI-BH and that of full-sequence MRI in the three reviewers (P > 0.05). Abbreviated MRI-BH showed good diagnostic performance for detecting malignant IPMNs by using a predetermined scoring system. 3 TECHNICAL EFFICACY STAGE: 2.

Reliability of the Clinical Frailty Scale in very elderly ICU patients: a prospective European study

PurposeFrailty is a valuable predictor for outcome in elderly ICU patients, and has been suggested to be used in various decision-making processes prior to and during an ICU admission. There are many instruments developed to assess frailty, but few of them can be used in emergency situations. In this setting the clinical frailty scale (CFS) is frequently used. The present study is a sub-study within a larger outcome study of elderly ICU patients in Europe (the VIP-2 study) in order to document the reliability of the CFS.Materials and methodsFrom the VIP-2 study, 129 ICUs in 20 countries participated in this sub-study. The patients were acute admissions ≥ 80 years of age and frailty was assessed at admission by two independent observers using the CFS. Information was obtained from the patient, if not feasible, from the family/caregivers or from hospital files. The profession of the rater and source of data were recorded along with the score. Interrater variability was calculated using linear weighted kappa analysis.Results1923 pairs of assessors were included and background data of patients were similar to the whole cohort (n = 3920). We found a very high inter-rater agreement (weighted kappa 0.86), also in subgroup analyses. The agreement when comparing information from family or hospital records was better than using only direct patient information, and pairs of raters from same profession performed better than from different professions.ConclusionsOverall, we documented a high reliability using CFS in this setting. This frailty score could be used more frequently in elderly ICU patients in order to create a more holistic and realistic impression of the patient´s condition prior to ICU admission.

Reliability of the PERFECT scheme assessed by unidigital and bidigital vaginal palpation.

The aim of the present study was to evaluate the inter- and intra-rater reliability of the PERFECT scheme assessed by uni- and bidigital vaginal palpation in young nulligravid women. Forty healthy women [median age 22 (19-34) years] were evaluated by unidigital vaginal palpation by both Examiners A and C, while the other 40 participants [median age 23.5 (19-35) years] were assessed by Examiners B and C by bidigital vaginal palpation. Inter- and intra-rater reliability of all items of the PERFECT scheme were evaluated: P = power (Modified Oxford Scale); E = endurance; R = repetitions; F = fast contractions. Cohen's linear weighted kappa (κw) was used to assess the reliability of the power, while the intraclass correlation coefficient (ICC) was applied for the other items. A priori sample size calculation found that 40 participants would be adequate. Inter-rater reliability of unidigital palpation was considered fair for power (κw = 0.34) and poor for other items (ICC < 0.50); bidigital inter-rater reliability was considered moderate (power: κw = 0.57; endurance: ICC = 0.53; fast contractions: ICC = 0.65, respectively) and low (repetitions: ICC = 0.27). Intra-rater reliability was substantial for power (κw = 0.73 and 0.62), moderate for fast contractions (ICC = 0.63 and 0.51) and poor for repetitions (ICC < 0.50), respectively, for uni- and bidigital palpation. Endurance showed poor (ICC < 0.50) and moderate (ICC = 0.61) reliability for uni- and bidigital palpation. The PERFECT scheme carried out by bidigital vaginal palpation presents higher reliability when two examiners carry out the physical examination. When one examiner is responsible for PFM assessment, both types of palpation are recommended for evaluation of power and fast contraction; endurance should be evaluated using bidigital palpation. Examiners should be careful during the assessment of repetition of sustained contractions because inter- and intra-reliabilities for both types of palpation were classified as poor.

Developing a Parental-Friendly Scoring System for Children with Congenital Hand Differences.

Background: Congenital hand differences (CHD) exhibit enormous diversity and heterogeneity. Surgeons and parents often have different concepts of severity, making things difficult during parental consultation. This study aims to align surgeon/parental views on the severity of the child's CHD using a novel severity classification. Methods: Parents of affected children were asked to score the severity of their child's abnormality pre- and post-consultation using a subjective scale (1-4) without any explanation. Furthermore, parents were asked to rate their concerns about the future function and appearance of their child's hand condition using a similar scale of 1-4. They were then asked to rate the severity of the CHD post-consultation and three months post-operatively following explanation of the 4-point scale, as follows: 1 = treatment possible to normal; 2 = treatment possible to near normal; 3 = treatment possible but always some hand differences; 4 = treatment not possible. The surgeon also independently scored all children using his perception of the scale. Results: Forty-three children with a range of CHD were recruited into the sample. Linear weighted kappa analyses comparing inter-rater agreement showed no agreement between surgeon and parents during the initial scoring without any explanations. However, with explanations added, agreement rose significantly (kappa = 0.437 post-consultation and kappa = 0.706 three months post-op). No correlation was found between severity with both appearance and function (r = 0.277 and r = -0.184, respectively). Conclusions: This study demonstrated that the use of a simple scoring system was able to improve parental understanding of the severity and prognosis of CHD. The system demonstrated a good correlation between surgeon and parents. Such a scoring system can be easily utilised in the outpatient department to manage expectations and reduce anxiety.

Interobserver and intraobserver variability in determining breast density according to the fifth edition of the BI-RADS® Atlas

ObjectiveTo determine the level of agreement within and between observers in the categorization of breast density on mammograms in a group of professionals using the fifth edition of the American College of Radiology’s BI-RADS® Atlas and to analyze the concordance between experts’ categorization and automatic categorization by commercial software on digital mammograms. MethodsSix radiologists categorized breast density on 451 mammograms on two occasions one month apart. We calculated the linear weighted kappa coefficients for inter- and intra-observer agreement for the group of radiologists and between the commercial software and the majority report. We analyzed the results for the four categories of breast density and for dichotomous classification as dense versus not dense. ResultsThe interobserver agreement among radiologists and the majority report was between moderate and nearly perfect for the analysis by category (κ = 0.64 to 0.84) and for the dichotomous classification (κ = 0.63 to 0.84). The intraobserver agreement was between substantial and nearly perfect (κ = 0.68 to 0.85 for 4 categories and k = 0.70 to 0.87 for the dichotomous classification). The agreement between the majority report and the commercial software was moderate both for the four categories (κ = 0.43) and for the dichotomous classification (κ = 0.51). ConclusionAgreement on breast density within and between radiologists using the criteria established in the fifth edition of the BI-RADS® Atlas was between moderate and nearly perfect. The level of agreement between the specialists and the commercial software was moderate.

Coronary artery calcium scoring on non-gated, non-contrast chest computed tomography (CT) using wide-detector, high-pitch and fast gantry rotation: comparison with dedicated calcium scoring CT.

BackgroundOur study assessed the reliability of non-gated, non-contrast chest computed tomography (NCCT) (with high pitch, wide coverage, and fast gantry rotation time, reconstructed at various slice thicknesses), compared with the electrocardiography (ECG)-gated calcium scoring cardiac computed tomography (CaCT), for quantifying coronary artery calcification (CAC).MethodsPatients aged ≥50 years who required clinical NCCT were prospectively enrolled. All CT scans were performed with 256-detector rows; z-axis coverage, 8 cm; pitch, 1.5; and gantry rotation time, 280 ms (table feed, 42.86 cm/s). NCCT was followed by ECG-gated CaCT. The NCCT images were reconstructed at 0.625-, 1.25-, and 2.5-mm slice intervals. The CAC score was calculated on four sets of CT images with a commercially available software using the Agatston method. The CAC scores were divided into four standard Agatston scoring categories (Agatston scores: 0, 1–100, 101–400, and >400). The inter-observer and inter-technique agreements were evaluated for the CAC scores.ResultsTwenty-six patients (M:F, 14:12; mean age, 66.04±6.97 years) were evaluated. Agatston scores showed near-perfect correlation between CaCT and NCCT for each slice thickness. On Bland-Altman analysis, the mean differences of Agatston scores between CaCT and NCCT (slice thicknesses: 0.625, 1.25, and 2.5 mm) were 37.54, 6.67, and −41.04, respectively. Inter-technique concordance was high for the four Agatston scoring categories with linear-weighted kappa values of 0.599, 0.609, and 0.597 for NCCT (slice thicknesses: 0.625, 1.25, and 2.5 mm, respectively). NCCT with 1.25-mm slice thickness showed the strongest correlation with CaCT.ConclusionsCAC quantification with NCCT using a wide detector, high pitch, and high temporal resolution scanning modes correlates very highly with ECG-gated CaCT, and 1.25-mm slice thickness NCCT images are more reliable than other NCCT images.

Complex renal cysts (Bosniak \u2265IIF): interobserver agreement, progression and malignancy rates

ObjectiveThe objective was to assess the interobserver agreement rate, progression rates and malignancy rates in the assessment of complex renal cysts (≥ Bosniak IIF) using a population-based database.MethodsA regional database identified 452 complex renal cysts in 415 patients between 2009 and 2019. Each patient was tracked and followed up using a unique identifier and deterministic linkage methodology. The interobserver agreement rate between radiologists was calculated using a weighted kappa statistic. Progression and malignancy rates of cysts (Bosniak ≥IIF) over the 11-year period were calculated.ResultsThe linear-weighted kappa value was 0.69 for all complex cysts. The rate of progression and regression of Bosniak IIF cysts was 4.6% (7/151) and 3.3% (5/151), respectively. All malignant IIF cysts progressed within 16 months of diagnosis. The malignancy rate of surgically resected Bosniak III and IV cysts was 79.3% (23/29) and 84.5% (39/46), respectively. Of all malignant tumours, 73.8% and 93.7% were of low ISUP grade and low stage, respectively.ConclusionsThis study further confirms that there is a good degree of agreement between radiologists in classifying complex renal masses using the Bosniak classification. The progression rate of Bosniak IIF cysts is low, but the malignancy rates of surgically resected Bosniak IIF, III and IV cysts are high. Benign cysts are frequently resected, and a very high proportion of histopathologically confirmed cancers in complex renal cysts are of low grade and stage.Key Points•There is a good degree of agreement between radiologists in classifying complex renal masses using the Bosniak classification.•The rate of progression of Bosniak IIF cysts is low, and malignant cysts progress early during surveillance. Although the malignancy rates of resected Bosniak IIF, III and IV cysts are high, the rate of benign cyst resection is significant.

Reliability and validity of the Norwegian-language version of the elderly mobility scale in older hospitalised patients.

Reliable and valid assessment tools are needed to evaluate and predict physical function in older hospitalised patients. The purpose of this study was to determine the reliability and validity of the Norwegian-language version of the Elderly Mobility Scale (EMS-N) for use with geriatric patients. Fifty patients admitted to a medical ward in a hospital in Norway were included. The inclusion criteria were acute hospital admission because of medical issues, age ≥65 years and referred to a physiotherapist for a physical function review. The original version of the Elderly Mobility Scale (EMS) was translated from English to Norwegian before initiating this study. EMS-N was tested for internal consistency, test-retest reliability and criterion validity by using the short physical performance battery (SPPB) as the gold standard. Internal consistency was measured by Cronbach's alpha. Test-retest reliability was estimated by linear weighted kappa and the intra-class correlation coefficient (ICC). The mean age of participants was 82 (range 65-95). The main admission diagnoses were acute functional decline (64%, n = 32) or acute infection (26%, n = 13). Internal consistency was estimated at 0.88. Test-retest reliability: six of the seven EMS-N items showed very good consistency, and the last item showed good consistency, ICC was estimated at 0.99. The standard error of measurement (SEM) reflected the absolute reliability, calculated at 0.52. The minimal detectable change (MDC) was calculated with 95 and 90% confidence intervals at 1.44 and 1.21, respectively. Criterion validity was calculated by a correlation analysis of the EMS-N and the SPPB. The rho value was estimated as 0.75, which corresponds to a high correlation and indicated good criterion validity. EMS-N exhibited good internal consistency and good to very good test-retest reliability and criterion validity. EMS can safely be used as an assessment tool for hospitalised geriatric patients.

The development and validation of an obstetric triage acuity index: a mixed-method study

Background There is uncertainty about the applicability of developed country triage scales in the developing population. Objective To develop and validate an obstetric triage scale in Iran. Methods This exploratory sequential mixed-method study was conducted on experienced gynecologists, emergency medicine, midwives, and nurses as evaluators, and pregnant or parturient women as sample. Development of obstetric triage acuity index was done using qualitative study and literature review. Thirty-two evaluators in two rounds evaluated the clinical impact score, content validity ratio, and content validity index. The tool performance was compared with a standard reference in 415 women. The association of urgency levels with hospitalization, resource utilization, and emergency department length of stay were assessed in 599 women. Five midwives independently double-triaged 234 convenience women to estimate inter-rater reliability. Results Out of 191 items in the initial draft, 9 (4.7%) items in the face validity, 33 (18.1%) in the first, and 5 (3.2%) in the second content validity round were removed because not meeting the expected thresholds. Nine items added based on evaluators’ suggestions. Finally, total items had a kappa above 0.74, and S-CVI/Ave, S-CVI/UA were 0.99 and 0.86, respectively. There was a significant correlation between the two tools (r = 0.661, p < 0). Level 1 mothers had a higher chance of using ≥ 2 resources (OR = 16.4, 95% CI = 2.06–129.98) and hospitalization (OR = 18.0, 95% CI = 3.96–81.51) compared to the level 5, after adjusting for gestational age/postpartum, and referral method. Sensitivity, specificity, and accuracy of the scale in predicting hospitalization were 74.8% (95% CI = 69.73–79.49), 80.8% (95% CI = 73.60–86.74), and 76.7% (72.67–80.48), respectively. The over and under-triage rates were 6.1% and 17.1%, respectively. The absolute agreement inter-raters was 86.8%, and the linear weighted kappa was 0.86 (95% CI = 0.81–0.91). Conclusion The designed OB triage scale showed good validity and reliability in the tertiary teaching hospital. Studying this tool at lower levels, non-educational, and public hospitals are recommended for external validity testing.

Estudio de la variabilidad inter- e intraobservador en la determinación de la densidad mamaria según la 5.a edición del Atlas BI-RADS®

ObjectiveTo determine the level of agreement within and between observers in the categorization of breast density on mammograms in a group of professionals using the fifth edition of the American College of Radiology's BI-RADS® Atlas and to analyze the concordance between experts’ categorization and automatic categorization by commercial software on digital mammograms. MethodsSix radiologists categorized breast density on 451 mammograms on two occasions one month apart. We calculated the linear weighted kappa coefficients for inter- and intra-observer agreement for the group of radiologists and between the commercial software and the majority report. We analyzed the results for the four categories of breast density and for dichotomous classification as dense versus not dense. ResultsThe interobserver agreement among radiologists and the majority report was between moderate and nearly perfect for the analysis by category (κ=0.64 to 0.84) and for the dichotomous classification (κ=0.63 to 0.84). The intraobserver agreement was between substantial and nearly perfect (κ=0.68 to 0.85 for 4 categories and k=0.70 to 0.87 for the dichotomous classification). The agreement between the majority report and the commercial software was moderate both for the four categories (κ=0.43) and for the dichotomous classification (κ=0.51). ConclusionAgreement on breast density within and between radiologists using the criteria established in the fifth edition of the BI-RADS® Atlas was between moderate and nearly perfect. The level of agreement between the specialists and the commercial software was moderate.

Reliability of repeat golf club testing sessions with modified club moment of inertia

The moment of inertia of a golf club, quantified about an axis at the butt of the handle, normal to the swing plane, has the potential to influence both clubhead and ball velocity. The purpose of this study was to assess the reliability of clubhead and ball velocity with changes to moment of inertia over repeat testing sessions and, if reliable, to quantify the effect of modifying moment of inertia. Eleven skilled male golfers hit 20 golf shots with three golf clubs, each with a different moment of inertia achieved through adding mass inside the club shaft and repeated this protocol over three sessions. A commercially available launch monitor was used to measure both velocity variables. Test–retest reliability was assessed via (1) limits of agreement, to determine reliability from a change in magnitude perspective and (2) linear-weighted kappa, to determine reliability from a directional perspective. The effect of moment of inertia on clubhead and ball velocity was determined using one-way, repeated measures analysis of variance tests, with partial eta squared being used to quantify the size of the effect. Increasing golf club moment of inertia reliably decreased clubhead and ball velocity, with fair to substantial kappa results revealed between sessions. The magnitude of decrease in these velocities, however, could not be reliably quantified. Statistically, the influence of moment of inertia was considered large (η2 ≥ 0.662 and 0.404) and significant (p < 0.001 and ≤ 0.006) for both clubhead and ball velocity, respectively.

Sensory perception testing by monofilaments in the digits of controls and workers with HAVS

ObjectiveTo determine if heavy manual work affects sensory perception in the digits and whether Semmes–Weinstein monofilaments (SWM) can be used as a screening tool to detect sensory neuropathy in the digits of workers exposed to hand-transmitted vibration (HTV).MethodsA cross-sectional study of office workers, heavy manual workers not exposed to HTV and workers with hand-arm vibration syndrome (HAVS). Sensory perception was measured in the digits by SWM using a forced-choice method to determine variability by sex, age, hand and digit. Frequency distributions were used to determine limit values and linear weighted kappa for intra-digit variability. Poisson regression was used to explore the relationship between sensory perception by SWM and abnormalities of thermal and vibration perception in the hands of workers with HAVS.ResultsThe sensory perception threshold of office workers did not vary by hand or digit. It was significantly lower in women < 30 than women aged ≥ 30 years. The 95th percentile for heavy manual workers was 1.00 (95% CI 0.60–1.00) and significantly higher than for office workers at 0.16 (95% CI 0.16–0.16). Heavy manual workers > 50 years had the highest threshold at 1.40 (95% CI 1.00–2.00). Weighted kappa for reliability was 0.63 (95% CI 0.53–0.70). A mean SWM threshold of ≥ 1.0 gram-force had a 79% sensitivity and 64% specificity for detecting abnormalities of thermal and vibration perception in the ipsilateral index and little fingers of workers with HAVS.ConclusionsSWM are a useful screening tool for detecting sensory loss in the digits of workers exposed to HTV.

Translation, Inter-rater Reliability, Agreement, and Internal Consistency of the Japanese Version of the Cumulated Ambulation Score in Patients after Hip Fracture Surgery.

ABSTRACTObjectives:The aim of this study was to translate the Cumulated Ambulation Score (CAS) from English into Japanese in cooperation with different types of healthcare providers and to investigate its inter-rater reliability and internal consistency.Methods:Two physical therapists at each of three general hospitals in Japan measured the mobility of 50 consecutive post-operative hip fracture patients on two occasions between 2 and 6 days after surgery using the Japanese version of the CAS (CAS-JP). We analyzed the inter-rater reliability and agreement using both the linear weighted kappa and the interclass correlation coefficient; we also analyzed the internal consistency using Cronbach’s alpha coefficient.Results:The mean age of patients was 81 (SD: 11.6) years and 82% were women. Approximately half of the patients had severe cognitive impairment. Kappa was ≥ 0.93 for the three mobility activities and for the total CAS-JP score, the percentage agreement was ≥ 0.98, the ICC was ≥ 0.95, and Cronbach’s alpha coefficient was 0.85.Conclusions:We found that the CAS-JP possessed good inter-rater reliability, agreement, and internal consistency. The CAS-JP is a reliable and easy-to-use evaluation tool suitable for daily clinical practice across different healthcare providers to monitor mobility in older hip fracture patients in Japan. We suggest that CAS-JP be evaluated in future studies for use in younger patients and in other patient groups with mobility problems.

NIMG-10. BRAIN TUMOR MRI STRUCTURED REPORTING ALLOWS CALCULATION OF INTERRATER AGREEMENT OF PATIENTS REVIEWED AT TUMOR BOARD

Abstract INTRODUCTION Longitudinal imaging evaluation of brain tumor patients with MRI is a challenging endeavor, as there is considerable overlap between imaging findings of tumor progression and treatment related changes. There is felt to be a high degree of variation between individual interpreters, particularly in complex cases. Structured reporting systems, such as the recently described brain tumor reporting and data system (BT-RADS), attempt to maximize consistency between reading radiologists, although how successful they are at achieving this is unknown. METHODS Patients with a diagnosis of a primary brain tumor with imaging presented at adult brain tumor board between October 2017 and March 2019 were reviewed. The most recent follow-up MRI was scored on an 8-point scale as described by BT-RADS, ranging from 0 to 4 with increasing suspicion for worsening disease, as described in the original radiologist report. Secondary review of MRI was performed by one of three neuroradiologists routinely participating in brain tumor conference. Interrater agreement between primary and secondary review were calculated using rates of exact agreement and linear weighted kappa. RESULTS Of studies reviewed, 270 out of 275 had imaging and reports suitable for repeat structured scoring. Grade 4 astrocytoma (glioblastoma) was the most common diagnosis, followed by grade 2 oligodendroglioma and grade 2 astrocytoma. The overall agreement rate between initial and secondary review was 83.0% with kappa of 0.89 +/- .05. There was perfect agreement on studies with improvement (1a or 1b) with lower levels of agreement (63.6–82.4%) for studies with worsening imaging findings (3a-4). CONCLUSIONS Using a structured reporting system to categorize MRIs in brain tumor patients allows for precise characterization of variation between report conclusions on primary and secondary review. Overall agreement is good, but variation rates increase with worsening findings which may be harder to correctly interpret.

Intra- and Interobserver Agreement in Hallux Valgus Angle Measurements on Weightbearing and Non-Weightbearing Radiographs

The choice of treatment of hallux valgus deformity is influenced by angles measured on radiographs. Angles of interest are the hallux valgus angle (HVA), 1,2-intermetatarsal angle (IMA), and distal metatarsal articular angle (DMAA), as well as the presence of first metatarsophalangeal joint (MTPJ) subluxation. Guidelines for measuring those angles have been distributed by American Orthopaedic Foot and Ankle Society (AOFAS), although the influence of weightbearing on these angles and its clinical relevance is not clear. We conducted a study to determine the influence of weightbearing and the inter- and intraobserver agreement in the measurement. A total of 104 patients were enrolled in this study. Both weightbearing and non-weightbearing radiographs were obtained. In 2 rounds, 2 orthopedic surgeons and 2 musculoskeletal radiologists measured the angles in blinded digital radiographs according to AOFAS guidelines. Agreement on measurement of HVA, IMA, and DMAA in both weightbearing and non-weightbearing radiographs, as well as the presence of MTPJ subluxation, was calculated using the linear-weighted kappa coefficient and the intraclass correlation coefficient (ICC). Examiner agreement strength was defined according to the guidelines of Landis and Koch. HVA decreases significantly with weightbearing, whereas IMA significantly increases. The change in magnitude was 1° to 2° on average. No significant influence on DMAA could be noted. Interobserver agreement was excellent in both weightbearing and non-weightbearing radiographs for HVA (ICC 0.99 and ICC 0.99, respectively), IMA (ICC 0.98 and ICC 0.86, respectively), and DMAA (ICC 0.95 and ICC 0.97, respectively). The agreement on presence of subluxation was moderate to good (Fleiss kappa 0.50 to 0.63). Weightbearing alters forefoot geometry significantly. Adhering to AOFAS guidelines yields excellent interobserver agreement on HVA, IMA, and DMAA. First MTPJ subluxation presence is not an alternative for DMAA. The magnitude of change in IMA and HVA is small and therefore not clinically important. Both weightbearing and non-weightbearing radiographs can be used for determination of the correct treatment of hallux valgus deformity.

Evaluation of the Clinitek\xae, a point-of-care urinalysis system for the measurement of clinically significant urinary metabolites and detection of haematuria in Schistosoma haematobium infected children in southern C\xf4te d\u2019Ivoire

BackgroundUrinary schistosomiasis, caused by Schistosoma haematobium, remains a significant public health problem worldwide, despite years of efforts to control it. Haematuria is one of the notable indirect indicators of S. haematobium infection and is commonly assessed along with other routine screens using a urinary dipstick test. A portable “field friendly” electronic analyser would offer an automated and thus more objective read-out compared to visual-read dipstick methods.MethodsWithin the framework of a Phase 2 praziquantel dose finding study in preschool- and school-aged children infected with S. haematobium, in southern Côte d’Ivoire, we compared a visual-read of the urine dipstick strips (Multistix PRO, Siemens Healthcare Diagnostics) to an automated reader (CLINITEK Status+ analyser™ Siemens Healthcare Diagnostics). Urine samples were collected from 148 pre-school aged and 152 school-aged children for urinalysis. Values were compared using a linear weighted kappa statistic and Bland–Altman analysis.ResultsA very good correlation between the two methods for nitrites and haematuria was observed (κ coefficient of 0.88 and 0.82, respectively), while a good correlation was observed for leukocytes (κ coefficient of 0.63) A moderate to fair correlation was calculated (κ coefficient ≤ 0.6) for all other parameters. When the results were stratified according to infection intensity, the agreements were stronger from the high infection intensity sample measurements, for most of the parameters.ConclusionOur results demonstrate the device’s utility in detecting haematuria and nitrites but underline the need for further development of this tool in order to improve its performance in the field.

PB2135 VALIDATION OF THE BOSTON UNIVERSITY STAGING SYSTEM IN AL AMYLOIDOSIS

Background:The Mayo 2004 cardiac risk score (Blood 2004; 104(6): 1881) which incorporates NT‐proBNP and cardiac troponins (TnI/TnT) is central to the modern management of AL amyloidosis but presents a problem for institutions where NT‐BNP is not available. To address this issue the Boston University (BU) cardiac risk has been proposed which incorporates BNP instead of NT‐ProBNP (Blood. 2019 Jan 17;133(3):215).Aims:We aimed to provide external validation of the BU staging system on a cohort of patients with AL amyloidosis attending the Princess Alexandra Hospital Amyloidosis Centre.Methods44 patients with newly diagnosed AL amyloidosis had both BNP and NT‐proBNP performed on the same serum sample collected before the initiation of any therapy. Both TnI and BNP assays were performed on Beckman DxI800 immunoassay analyser (Beckman Coulter, Brea, CA, USA) using Quidel Triage BNP immunoassay and AccuTnI two‐site immunoenzymatic assay. NT‐proBNP was analysed on an Cobas E411 analyser (Roche Diagnostics, Basel, Switzerland) via an Elecsys proBNP II Cobas electrochemiluminescence immunoassay. We applied the published BNP cut‐off of 81 ng/L for the BU score, NT‐proBNP cut‐off of 332 ng/L for the Mayo 2004 score along with a TnI cut‐off of 0.1ug/mL for both staging systems. Patients were allocated to stage I if both were below the threshold, stage II if only one was above the threshold, and stage III if both biomarkers were above the threshold. Survival was measured from the time of blood sample collection until the date of death or last follow‐up and calculated by the Kaplan‐Meier method. The log‐rank test was used to assess the impact of cardiac biomarker score on survival. Concordance between Mayo 2004 staging and BU staging scores was determined using linear weighted kappa statistics.Results:Median age of the 44 patients was 67 years and 27% were female. Cardiac involvement was present in 61% and renal involvement in 54%. Initial treatment regimens varied across the cohort, including proteasome inhibitors (n = 15), immunomodulatory agents (n = 8), alkylator based regimens (n = 6), autologous stem cell transplantation (n = 7), no therapy (n = 7) and rituximab‐based therapy (n = 1). Three patients with advanced cardiac disease died before treatment commenced. Median duration of follow‐up of survivors was 7.3 years.Using the BU BNP‐based staging system we identified 12/44 (27%) of patients as stage I, 18/44 (41%) of patients as stage II and 14/44 (31%) of patients as stage III. This correlated closely with stratification via the Mayo score (Table 1), with only 1 patient miscategorised (97.7% agreement, k = 0.98). BU stage was highly predictive of outcome. Median overall survival for BU stage I was not reached, stage II was 40 months and stage III was 5 months (log rank p = 0.0012). Mayo 2004 median overall survival was identical to the BU system for stage I, II and III.Summary/Conclusion:We confirmed that the BU staging system, incorporating a BNP threshold of 81 ng/L obtained at initial consultation, stratified patients in concordance with the Mayo 2004 staging system which uses a NT‐proBNP threshold of 332 ng/L. Furthermore, we also demonstrate that the BU score displays similar prognostic information.image

Pain perception in orthodontic patients: A model considering psychosocial and behavioural aspects.

To estimate the agreement between orthodontic pain perception, evaluated with the visual analog scale (VAS), and psychosocial and behavioural aspects of pain and to estimate the impact of clinical and demographic characteristics on these aspects. Cross-sectional design using non-probabilistic sampling. Orthodontic patients undergoing treatment at orthodontic clinics (n=507 [63.3% women], mean age: 26.32 [SD=11.70] years). Psychosocial and behavioural aspects of pain were evaluated using the Multidimensional Pain Inventory (MPI-Orthodontic). Agreement between the pain impact level assessed according to different methods was estimated using the linear-weighted Kappa (κp ) statistic. Structural models were elaborated to estimate the impact of clinical and demographic characteristics on the psychosocial and behavioural aspects of pain. The fit of the model was evaluated, and the z test (α=5%) was used to estimate the significance of the impact (β). The agreement between VAS and MPI-Orthodontic factors was inadequate (κp =0.028-0.584). The fit of the structural models was adequate. Women, younger individuals, and those who reported difficulty/pain with feeding exhibited greater perception of both the psychosocial and behavioural aspects. Individuals in lower socioeconomic strata who were not satisfied with treatment and did not seek treatment voluntarily exhibited greater perception of the psychosocial aspect of pain. The impact of orthodontic pain on psychosocial and behavioural aspects of patients' lives is a relevant issue. Clinical and demographic characteristics contributed to these aspects; however, pain intensity as a sole measure may be insufficient for an adequate understanding of pain perception.