Inter-rater Agreement Analysis Research Articles

Perspectives on the diagnosis and management of functional cognitive disorder: An international Delphi study.

Current proposed criteria for functional cognitive disorder (FCD) have not been externally validated. We sought to analyse the current perspectives of cognitive specialists in the diagnosis and management of FCD in comparison with neurodegenerative conditions. International experts in cognitive disorders were invited to assess seven illustrative clinical vignettes containing history and bedside characteristics alone. Participants assigned a probable diagnosis and selected the appropriate investigation and treatment. Qualitative, quantitative and inter-rater agreement analyses were undertaken. Eighteen diagnostic terminologies were assigned by 45 cognitive experts from 12 countries with a median of 13 years of experience, across the seven scenarios. Accurate discrimination between FCD and neurodegeneration was observed, independently of background and years of experience: 100% of the neurodegenerative vignettes were correctly classified and 75%-88% of the FCD diagnoses were attributed to non-neurodegenerative causes. There was <50% agreement in the terminology used for FCD, in comparison with 87%-92% agreement for neurodegenerative syndromes. Blood tests and neuropsychological evaluation were the leading diagnostic modalities for FCD. Diagnostic communication, psychotherapy and psychiatry referral were the main suggested management strategies in FCD. Our study demonstrates the feasibility of distinguishing between FCD and neurodegeneration based on relevant patient characteristics and history details. These characteristics need further validation and operationalisation. Heterogeneous labelling and framing pose clinical and research challenges reflecting a lack of agreement in the field. Careful consideration of FCD diagnosis is advised, particularly in the presence of comorbidities. This study informs future research on diagnostic tools and evidence-based interventions.

Can Artificial Intelligence Identify Reading Fluency and Level? Comparison of Human and Machine Performance

This study examined whether an artificial intelligence-based automatic speech recognition system can accurately assess students’ reading fluency and reading level. Participants were 120 fourth-grade students attending public schools in Türkiye. Students read a grade-level text out loud while their voice was recorded. Two experts and the artificial intelligence-based automatic speech recognition system analyzed the recordings for reading errors. Following the analysis, a word error rate was calculated for both the experts and the artificial intelligence-based automatic speech recognition system. Word error rates were converted into reading accuracy rate scores. Inter-rater agreement and linear regression analyses were used to compare the raters’ reading fluency scores, and logistic regression analyses were used to compare the classification of readers according to their reading levels. Results showed that the difference between the scores of the artificial intelligence-based automatic speech recognition system and the expert scores was minimal. This is because there was a very high level of agreement between the artificial intelligence-based automatic speech recognition system and the experts scores. Linear regression analyses showed that the artificial intelligence-based automatic speech recognition system significantly predicted the scores of experts. According to the logistic regression analysis results, the artificial intelligence-based automatic speech recognition system was at least 93% as successful as human raters in classifying readers as poor and good. These results give us hope that reading assessments at classroom, school, regional, national, and even international levels can be conducted more accurately and economically by using artificial intelligence-based systems in the coming years.

An automated flowchart for the Modified Rankin Scale assessment: A multicenter inter-rater agreement analysis.

The Modified Rankin Scale (mRS) is a widely adopted scale for assessing stroke recovery. Despite limitations, the mRS has been adopted as primary outcome in most recent clinical acute stroke trials. Designed to be used by multidisciplinary clinical staff, the congruency of this scale is not consistent, which may lead to mistakes in clinical or research application. We aimed to develop and validate an interactive and automated digital tool for assessing the mRS-the iRankin. A panel of five board-certified and mRS-trained vascular neurologists developed an automated flowchart based on current mRS literature. Two international experts were consulted on content and provided feedback on the prototype platform. The platform contained five vignettes and five real video cases, representing mRS grades 0-5. For validation, we invited neurological staff from six comprehensive stroke centers to complete an online assessment. Participants were randomized into two equal groups usual practice versus iRankin. The participants were randomly allocated in pairs for the congruency analysis. Weighted kappa (kw) and proportions were used to describe agreement. A total of 59 professionals completed the assessment. The kw was dramatically improved among nurses, 0.76 (95% confidence interval (CI) = 0.55-0.97) × 0.30 (0.07-0.67), and among vascular neurologists, 0.87 (0.72-1) × 0.82 (0.66-0.98). In the accuracy analysis, after the standard mRS values for the vignettes and videos were determined by a panel of experts, and considering each correct answer as equivalent to 1 point on a scale of 0-15, it revealed a higher mean of 10.6 (±2.2) in the iRankin group and 8.2 (±2.3) points in the control group (p = 0.02). In an adjusted analysis, the iRankin adoption was independently associated with the score of congruencies between reported and standard scores (beta coefficient = 2.22, 95% CI = 0.64-3.81, p = 0.007). The iRankin adoption led to a substantial or near-perfect agreement in all analyzed professional categories. More trials are needed to generalize our findings. Our user-friendly and free platform is available at https://www.irankinscale.com/.

Ultrafast Brain MRI with Deep Learning Reconstruction for Suspected Acute Ischemic Stroke.

Background Deep learning (DL)-accelerated MRI can substantially reduce examination times. However, studies prospectively evaluating the diagnostic performance of DL-accelerated MRI reconstructions in acute suspected stroke are lacking. Purpose To investigate the interchangeability of DL-accelerated MRI with conventional MRI in patients with suspected acute ischemic stroke at 1.5 T. Materials and Methods In this prospective study, 211 participants with suspected acute stroke underwent clinically indicated MRI at 1.5 T between June 2022 and March 2023. For each participant, conventional MRI (including T1-weighted, T2-weighted, T2*-weighted, T2 fluid-attenuated inversion-recovery, and diffusion-weighted imaging; 14 minutes 18 seconds) and DL-accelerated MRI (same sequences; 3 minutes 4 seconds) were performed. The primary end point was the interchangeability between conventional and DL-accelerated MRI for acute ischemic infarction detection. Secondary end points were interchangeability regarding the affected vascular territory and clinically relevant secondary findings (eg, microbleeds, neoplasm). Three readers evaluated the overall occurrence of acute ischemic stroke, affected vascular territory, clinically relevant secondary findings, overall image quality, and diagnostic confidence. For acute ischemic lesions, size and signal intensities were assessed. The margin for interchangeability was chosen as 5%. For interrater agreement analysis and interrater reliability analysis, multirater Fleiss κ and the intraclass correlation coefficient, respectively, was determined. Results The study sample consisted of 211 participants (mean age, 65 years ± 16 [SD]); 123 male and 88 female). Acute ischemic stroke was confirmed in 79 participants. Interchangeability was demonstrated for all primary and secondary end points. No individual equivalence indexes (IEIs) exceeded the interchangeability margin of 5% (IEI, -0.002 [90% CI: -0.007, 0.004]). Almost perfect interrater agreement was observed (P > .91). DL-accelerated MRI provided higher overall image quality (P < .001) and diagnostic confidence (P < .001). The signal properties of acute ischemic infarctions were similar in both techniques and demonstrated good to excellent interrater reliability (intraclass correlation coefficient, ≥0.8). Conclusion Despite being four times faster, DL-accelerated brain MRI was interchangeable with conventional MRI for acute ischemic lesion detection. © RSNA, 2024 Supplemental material is available for this article. See also the editorial by Haller in this issue.

Mixture polarization in inter-rater agreement analysis: a Bayesian nonparametric index

In several observational contexts where different raters evaluate a set of items, it is common to assume that all raters draw their scores from the same underlying distribution. However, a plenty of scientific works have evidenced the relevance of individual variability in different type of rating tasks. To address this issue the intra-class correlation coefficient (ICC) has been used as a measure of variability among raters within the Hierarchical Linear Models approach. A common distributional assumption in this setting is to specify hierarchical effects as independent and identically distributed from a normal with the mean parameter fixed to zero and unknown variance. The present work aims to overcome this strong assumption in the inter-rater agreement estimation by placing a Dirichlet Process Mixture over the hierarchical effects’ prior distribution. A new nonparametric index λ\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\lambda$$\\end{document} is proposed to quantify raters polarization in presence of group heterogeneity. The model is applied on a set of simulated experiments and real world data. Possible future directions are discussed.

Validity of Ultrasound for the Diagnosis of Arterial Thoracic Outlet Syndrome

ObjectiveThoracic outlet syndrome (TOS) is a rare disorder mostly seen in younger individuals. Although patients’ wellbeing is relevantly impaired, it often takes a long time before the diagnosis is made. Digital subtraction angiography (DSA) is routinely used despite its radiation exposure, which is a major concern in this young patient population. Moreover, DSA offers limited opportunities for functional assessment. In contrast, ultrasonography is widely accessible without causing radiation exposure and allows for flexible functional assessment. The main goal of the study was to investigate whether ultrasound (US) is a viable alternative to DSA in diagnosing arterial TOS (aTOS). MethodsPatients referred to a tertiary centre for evaluation of suspected TOS were recruited into the study. DSA was routinely performed with the patient’s arms both in the raised (abducted) and neutral (adducted) position. Two vascular surgeons and two radiologists assessed the resulting images for presence of aTOS. Additionally, two examiners performed US according to a standardised protocol. The reference for presence of aTOS was the DSA based interdisciplinary vascular conference consensus. Inter-rater agreement and latent class analysis (LCA) were performed between assessors and diagnostic methods. ResultsFifty one consecutive patients (two thirds female) aged 39.3 ± 13.0 years were included within 11 months. US agreement was excellent at 0.94 (0.841 – 0.980), DSA agreement for vascular surgeons was good at 0.779 (0.479 – 1.000), whereas it was moderate at 0.546 (0.046 – 1.000) for radiologists. Results suggest that DSA is untenable as the gold standard for aTOS diagnosis. In LCA, US was shown to be a reliable diagnostic tool for the detection of aTOS. ConclusionUS examination is a valid test for the detection of haemodynamically relevant compression of arteries in the diagnostic work up of aTOS using a standardised protocol. The role of DSA as the gold standard should be reviewed and needs to be reconsidered.

Modeling Bland–Altman Limits of Agreement with Fractional Polynomials—An Example with the Agatston Score for Coronary Calcification

Bland–Altman limits of agreement are very popular in method comparison studies on quantitative outcomes. However, a straightforward application of Bland–Altman analysis requires roughly normally distributed differences, a constant bias, and variance homogeneity across the measurement range. If one or more assumptions are violated, a variance-stabilizing transformation (e.g., natural logarithm, square root) may be sufficient before Bland–Altman analysis can be performed. Sometimes, fractional polynomial regression has been used when the choice of variance-stabilizing transformation was unclear and increasing variability in the differences was observed with increasing mean values. In this case, regressing the absolute differences on a function of the average and applying fractional polynomial regression to this end were previously proposed. This review revisits a previous inter-rater agreement analysis on the Agatston score for coronary calcification. We show the inappropriateness of a straightforward Bland–Altman analysis and briefly describe the nonparametric limits of agreement of the original investigation. We demonstrate the application of fractional polynomials, use the Stata packages fp and fp_select, and discuss the use of degree-2 (the default setting) and degree-3 fractional polynomials. Finally, we discuss conditions for evaluating the appropriateness of nonstandard limits of agreement.

Profound barriers to value management adoption on construction projects in a developing nation

Purpose The adoption of value management (VM) has been on the rise in the construction industry, especially in the past decade. Identifying the main barriers to VM application for ameliorating poor project outcomes in the construction industry is a critical step to overcoming its implementation. Therefore, this study aims to examine the barriers to the adoption of VM in the Nigerian construction industry (NCI) and identify the critical ones for action by construction stakeholders. Design/methodology/approach This study used two rounds of Delphi surveys with 15 VM experts to identify the challenges to VM adoption. The Delphi team’s data from the two surveys were analysed using descriptive statistics. The mean-mode approach was used to identify ten major barriers to VM adoption based on the consensus reached during the second round of the Delphi poll. Following that, an interrater agreement analysis and a significant level analysis were performed. Findings Of the 17 identified barriers hindering VM adoption on construction projects in the NCI, 10 were found to be profound after statistical analyses of the data. The significant barriers include communication gap between the client and other consultants; delay in design preparation because of VM services; the pretext that VM is being practiced in the construction industry; and the wrong perception of VM by construction professionals, among others. The findings of the analysis suggested that the lack of VM adoption in the NCI was because of inadequate knowledge of VM techniques among construction professionals. Therefore, it is advised that VM training be provided for concerned stakeholders to address the lack of VM expertise among Nigerian construction professionals. Practical implications This study investigated the barriers to VM adoption on construction projects in the NCI with the opinions of a few construction professionals who can be regarded as VM experts. As against obtaining large survey responses from construction professionals who may not be knowledgeable in the VM process, the limited VM experts’ responses provide a valid basis for drawing inferences and making recommendations to address the paucity of VM applications in the NCI. Originality/value Through a Delphi survey, the critical barriers to the adoption of VM were identified, along with recommendations to help construction stakeholders overcome the challenges and enjoy the benefits of VM.

Development of a Campus Facility Management Operational Framework Using a Modified Delphi Method

An effective facility management process is crucial to overall firm performance and is becoming a core competency for organizations seeking to gain a competitive advantage. The recent literature on facility management reveals that poor facility management is a major cause of management inefficiency, leading directly to budget overruns, maintenance delays, repetition of work, variations, noncompliance incidents, unnecessary risk, and dissatisfied customers. Therefore, facility management firms should pay close attention to the most critical success factors in order to effectively manage and measure performance. This paper presents a multidimensional campus facility management performance framework (CFMPF) that addresses both operational and systematic factors. Facility management companies, owners, contractors, and consultants can use this framework to identify systemic weaknesses using key indicators to specifically assess the effectiveness and success of facility management. The significance of 45 key facility management success factors across seven categories was determined via a literature review, expert interviews, and the Delphi method. The seven process groups are: Campus Facility Work Force Management, Campus Facility Communication Management, Campus Facility Systems Management, Campus Facility Organizational Management, Campus Facility Financial Management, Campus Sustainability & Environment Management, and Campus Facility Assets Management. To verify the overall agreement via the Delphi rounds, the score percentage and standard deviation to mean ratio were examined. Mean value and interrater agreement (IRA) analyses were conducted to evaluate the correlation between the factors and facility performance. This research adds to the body of knowledge by presenting a systematic operational framework for campus facility management that identifies the underlying factors influencing performance and fills the gaps left by previous studies.

Comparative analysis between the 8th and 7th AJCC as a prognostic tool for patients diagnosed with nonmetastatic head and neck soft-tissue sarcoma.

e23550 Background: Head and neck soft-tissue sarcoma (HN-STS) is a rare entity for which a new prognostic risk stratification has been proposed in the 8th AJCC. The tumor size 2cm and 4cm have been adopted as cutoffs for T1 to T3 classification and the invasion of surrounding structures denotes T4. In opposite, in the 7th edition the primary tumor size of 5 cm determines T1 and T2 classification. This study aimed to compare the 8th and 7th AJCC as a prognostic tool for HN-STS. Additionally, we sought to investigate prognostic factors in this group of patients. Methods: This is a single Tertiary Hospital study approved by the institutional ethical review committee from the AC Camargo Cancer Center. We retrospectively evaluated patients with non-metastatic HN-STS whose histology subtypes are included in the 40th chapter of the 8th AJCC version. We compared the AJCC 7th and 8th T staging with Cohen’s kappa coefficient using inter-rater agreement analysis. The Kaplan-Meier (KM) estimator was used to calculate median survival times; differences in time-to-event outcomes were assessed using the log-rank test. Cox proportional hazard model was used to assess the effect of features in overall survival (OS). Results: Ninety-five HN-STS treated with curative intent from 2005 from 2021 patients diagnosed were evaluated. Surgical resection (94.7%), radiotherapy (48.4%) and chemotherapy (23.1%) were the employed therapeutic approaches. Leiomyosarcoma and liposarcoma (10 of each subtype) were the most common histology subtypes; High grade disease was present in 66.3%. Kappa coefficient of inter-rater agreement was 0.46 with statistical significance (p = 0.03), which indicates poor agreement between AJCC 7th and 8th T staging system. Regarding OS stratified by T size in the AJCC 7th (T1 and T2) and 8th (T1, T2, T3 and T4) was not statistically different between groups (P = 0.83 and P = 0.21, respectively). Patients with T4 (47.6% vs 78.7%, P = 0.04) and high grade (64.0% vs 96.2%, P = 0.003) had worst 5-years OS. In Cox multivariate analysis, patients with 8th edition T4 and high grade had 3.15- and 4.16-fold increased risk of death, respectively. Conclusions: In our series of patients with HN-STS, neither 7th nor 8th AJCC T size category groups discriminated prognosis. Tumor with local invasion (8th AJCC T4) and high grade were all associated with increased risk of mortality.

Determination of the significance of project delivery attributes (PDAs) on sustainable projects in Turkey

Purpose Many parameters influence the success of sustainable projects (SPs) in the architecture–engineering–construction. One of these important parameters is the project delivery attributes (PDA), which are influenced by the project delivery system (PDS) while affecting the selection of it. This study aims to evaluate the significance of PDAs in influencing both the performance and success of SPs in Turkey, where the interest in SPs is high. Design/methodology/approach The impact of PDAs was determined by applying the two-round Delphi method with a semistructured interview involving the main stakeholders of a construction project, like owners, designers, contractors and consultants who played active roles in SPs, as well as academics to theoretically evaluate the issue. The significance of PDAs was assessed using the relative importance index, and the results were validated using the interrater agreement analysis. Findings The study identified key PDAs impacting SPs as owner character, commitment and motivation; simulation and energy modelling; and timing of stakeholders/early involvement. Originality/value The investigation of the significance of the PDAs is a lesser-studied context. Therefore, a research framework that enables an effective set of methods for solving the sectoral problems of PDAs that have impacts on SPs has been proposed. The framework is expected to open new opportunities for the generation and regulation of the PDSs for SPs. The findings will provide valuable insights to project stakeholders, particularly owners, local authorities and policymakers to assess which PDAs have a greater impact on sustainability performance when setting PDSs in SPs in other developing countries.

526. Antimicrobial Selection Based on Precision Metagenomics Compared with Standard Urine Culture/Susceptibility: A Reliability and Inter-rater Agreement Analysis for Application in Patients with Urinary Tract Infections

Abstract Background Management of urinary tract infections (UTIs) is highly variable and ∼50% of affected women may receive inappropriate antimicrobials. Precision Metagenomics-based detection of pathogens and antimicrobial resistance (AMR) markers can inform antimicrobial selection in support of stewardship practices. We investigated inter-physician agreement of culture and metagenomic results in a retrospective analysis. Methods We used de-identified urine samples with results of urine culture &amp; antimicrobial susceptibility testing (AST, n=25, 76% culture-positive) and the Explify® Urinary Pathogen ID/AMR Panel (UPIP, IDbyDNA Inc). UPIP is a Precision Metagenomics method to detect &amp;gt;170 uropathogens and &amp;gt;2,000 AMR markers. Four physicians reviewed culture/AST and UPIP results independently to assess if and how they would treat based on each set of results in a middle-aged, female patient with uncomplicated UTI (no comorbidities or medication allergies). Consensus was defined by simple majority; treatment decisions and antimicrobial choice were adjudicated by an infectious disease trained pharmacist. Inter-rater agreement (Fleiss’ kappa) and reliability for accurate case management (reference: adjudicator) were estimated for each method. Results Analytical agreement for uropathogen identification between the two methods was 72% with 95% positive agreement for common uropathogens. Consensus on whether or not to treat was reached for 96% (24/25) of cases for both culture/AST and UPIP; decisions were concordant between methods for 88% of cases (22/25). Inter-rater agreement was high (culture: k=0.73, UPIP: k=0.68; p&amp;lt; 0.05). Physicians were more likely to decide to treat based on culture results (17/25, 68%) than UPIP (15/25, 60%). UPIP detected additional organism(s) and/or AMR marker(s) compared with culture/AST alone that were informative for stewardship-guided treatment decisions in 28% (7/25) of samples. Figure 1: Physician Consensus Treatment Decisions Based on Results of Urine Sample Analysis by Precision Metagenomics or Urine Culture/AST Four infectious disease physicians independently reviewed paired results from analysis of a set of 25 clinical remnant urine samples by an investigational Precision Metagenomics method (UPIP) and by standard urine culture/AST and determined whether or not treatment would be warranted. Figure 2:Summary of Changes in Antimicrobial Selection based on Interpretation of Precision Metagenomics vs Culture/AST Results An infectious disease pharmacist reviewed and adjudicated hypothetical antimicrobial selections. All antimicrobial selections based on the results of the Precision Metagenomics method (UPIP) were found to be appropriate; in 7/15 cases, hypothetical treatment based on UPIP results would have resulted in a change in management that was evaluated to be consistent with stewardship practices. Conclusion Physician agreement on hypothetical treatment recommendations was comparable between the investigational method (UPIP) and the reference method (urine culture/AST). In no case was there inappropriate consensus to treat based on UPIP results alone. The findings from this pilot analysis support the feasibility of metagenomics-guided management of UTIs. Disclosures Benjamin Briggs, MD, PhD, CloudCath: Advisor/Consultant|CosmosID: Employee Benjamin Briggs, MD, PhD, CloudCath: Advisor/Consultant|CosmosID: Employee Robert Schlaberg, MD, MPH, IDBYDNA INC: Board Member|IDBYDNA INC: Explify-related IP|IDBYDNA INC: Employee|IDBYDNA INC: Stocks/Bonds.

Exploring variations in current clinical practice regarding postoperative pericardial effusion: a questionnaire study

Abstract Background Postoperative pericardial effusion (PPE) is a frequently occurring event after cardiac surgery, potentially leading to life-threatening cardiac tamponade. Unfortunately, no guidelines are currently available on this subject, possibly leading to unwarranted variations in care in clinical practice. Purpose The goal of our study was to determine the diagnostic and therapeutic preferences of cardiologists and cardiothoracic surgeons in the Netherlands regarding PPE, and to evaluate possible variations in this matter between specialties and centers. Methods A web-based questionnaire was sent to all interventional cardiologists and cardiothoracic surgeons in the Netherlands. Diagnostic preferences for the evaluation of PPE were examined. Additionally, clinical preferences were explored in response to 4 patient scenarios with each illustrating a combination of either high or low clinical suspicion of cardiac tamponade and either high or low echocardiographic suspicion. Each scenario was also stratified by 3 different amounts of PPE (&amp;lt;1cm, 1–2cm, &amp;gt;2cm), leading to a total of 12 fictional patient cases (Figure 1). Results From 27 of the 31 contacted centers, a total of 94 clinicians responded to the survey, of whom 46 interventional cardiologists and 48 cardiothoracic surgeons. Cardiologists were more in favor of routine postoperative echocardiography in all patients (44%), whereas cardiothoracic surgeons preferred routine imaging after specific procedures, especially after mitral (85%) and tricuspid (79%) valve surgery. Overall, drainage of PPE (&amp;gt;48 hours postoperatively) was preferred through pericardiocentesis (83%) instead of surgical evacuation (17%). Regarding the patient scenarios, cardiothoracic surgeons were significantly more inclined to evacuate PPE compared to cardiologists for all scenarios combined (51% vs 37%, p&amp;lt;0.001) (Figure 1). In addition, evacuation was overall significantly more preferred by cardiologists employed in surgical centers when compared to those working in non-surgical centers (43% vs 31%, p=0.02). Results from the interrater agreement analysis of groups of clinicians working in the same center varied from very poor up to very good (kappa value of 0.22–0.67), suggesting existing variation of PPE treatment preferences even within one center. Conclusions Our study highlights the significant variation of preferences in contemporary PPE management between cardiothoracic surgeons and interventional cardiologists in the Netherlands, sometimes even within the same center. These findings are the possible consequences of a current lack of general recommendations or guidelines. This could lead to unwarranted variation in care and additional research is necessary to provide general recommendations. Funding Acknowledgement Type of funding sources: None.

Total serum vitamin B12 (cobalamin) LC-MS/MS assay as an arbiter of clinically discordant immunoassay results.

Measurement of the serum levels of vitamin B12 (VB12) is key for evaluating VB12 deficiency-dependent anemia. Immunoassay, the major method for determining VB12, tends to give false-normal results because of the presence of anti-intrinsic factor (IF-Ab) or other factors such as heterophilic antibodies etal. This study aimed to develop a liquid chromatography tandem mass spectrometry (LC-MS/MS) method that is helpful for distinguish false normal VB12 results measured by the immunoassay. Different forms of VB12 were derivatized into CN-B12, which was collected through solid-phase extraction and analyzed via LC-MS/MS. 236 serum samples were measured both by LC-MS/MS and immunoassay, results were compared, and the IF-Ab effect was evaluated. The LC-MS/MS assay afforded a linear slope from 20 to 4,000pmol/L for CN-B12. OH-VB12, methyl-VB12, and CoA-VB12 showed recovery within 89.3-109.5%. The intra-assay CV of VB12 was 2.6-4.1%, whereas the total CV was 9.3-9.8%. Passing-Bablok regression between LC-MS/MS and immunoassay results showed that the slope was 1.085 and the intercept was-15.691. The Bland-Altman plot showed that the mean difference and difference% were-34.6 pmol/L and 0.3%, respectively. Inter-rater agreement analysis showed that the linear weighted kappa value was 0.885, implying good agreement between the two methods. However, two samples were falsely elevated and one sample was falsely normal in the immunoassay compared with LC-MS/MS. The LC-MS/MS method helped in the distinction of false-normal VB12 results shown by the immunoassay. The VB12 LC-MS/MS method can be used as an arbiter of clinically discordant immunoassay results.

Validation of a novel magnetic resonance imaging classification and recommended treatment for lateral elbow tendinopathy

BackgroundLateral epicondylitis is one of the most common upper extremity problems presented to orthopedic surgeons. Despite a rapid and accurate arrival at a diagnosis by clinical examination, there exists no consensus classification for this condition, which hampers clinical approaches for treatment of the disease based on its severity. Thus, the aim of this study was to propose and valiadate a new magnetic resonance imaging (MRI) classification of lateral epicondylitis, staging by tendinosis, the degree of thickness tears of the common extensor tendon (CET) and bone bruise lesion.MethodMRI assessment of the elbow of 75 patients (57 women and 18 men; mean age:51.4 years (range,34–73) from Jan 2014 to Jan 2021 who were diagnosed with lateral epicondylitis were included in the study. MR images were reviewed retrospectively by two independent upper extremities orthopedists and one musculoskeletal radiologist. Inter- and intra-observer reliabilities for the classification were calculated using kappa statistics for the analysis of interrater agreement. Correlation between the stage of the disease and the duration of symptom before MRI was calculated using Kruskal–wallis test.ResultsVarious degrees of CET lesions were demonstrated in this population (Stage I-17, IIA-7, IIB-22 and III-29). Intra-observer agreements of MRI staging were substantial to satisfactory. Inter-observer agreements were moderate to substantial. There was no significant correlation between the disease stage and the patient age or the duration of symptom before MRI.ConclusionOur MRI classification has emerged as one of the most reliable methods to define stages of chronic lateral epicondylitis. At the end, we have suggeted a clearer direction for understanding the disease pathology as well as an appropriate management protocol for each stage of the disease in line with the recent body of literature.

Inappropriate Prescriptions in Older People-Translation and Adaptation to Portuguese of the STOPP/START Screening Tool.

Inappropriate prescribing, which encompasses the prescription of potentially inappropriate medications (PIMs) and potential prescribing omissions (PPOs), is a common problem for older people. The STOPP/START tool enables general practitioners, who are the main prescribers, to identify and reduce the incidence of PIMs and PPOs and appraise an older patient’s prescribed drugs during the diagnosis process to improve the clinical care quality. This study aimed to translate and validate the STOPP/START screening tool to enable its use by Portuguese physicians. A translation-back translation method including the validation of the obtained Portuguese version was used. Intra- and inter-rater reliability and agreement analyses were used in the validation process. A dataset containing the information of 334 patients was analyzed by one GP twice within a 2-week interval, while a dataset containing the information of 205 patients was independently analyzed by three GPs. Intra-rater reliability assessment led to a Kappa coefficient (κ) of 0.70 (0.65–0.74) for the STOPP criteria and 0.60 (0.52–0.68) for the START criteria, considered to be substantial and moderate values, respectively. The results of the inter-rater reliability rating were almost perfect for all combinations of raters (κ > 0.93). The version of the STOPP/START criteria translated into Portuguese represents an improvement in managing the medications prescribed to the elderly. It provides clinicians with a screening tool for detecting potentially inappropriate prescribing in patients older than 65 years old that is reliable and easy to use.

Clinician Agreement on the Classification of Thyroid Nodules Ultrasound Features: A Survey of 2 Endocrine Societies.

Thyroid nodule risk stratification allows clinicians to standardize the evaluation of thyroid cancer risk according to ultrasound features. To evaluate interrater agreement among clinicians assessing thyroid nodules ultrasound features and thyroid cancer risk categories. We surveyed Endocrine Society and Latin American Thyroid Society members to assess their interpretation of composition, echogenicity, shape, margins, and presence of echogenic foci of 10 thyroid nodule cases. The risk category for thyroid cancer was calculated following the American College of Radiology-Thyroid Imaging Reporting & Data System (ACR-TIRADS) framework from individual responses. We used descriptive statistics and Gwet's agreement coefficient (AC1) to assess the primary outcome of interrater agreement for ACR-TIRADS risk category. As secondary outcomes, the interrater agreement for individual features and a subgroup analysis of interrater agreement for the ACR-TIRADS category were performed (ultrasound reporting system, type of practice, and number of monthly appraisals). A total of 144 participants were included, mostly endocrinologists. There was moderate level of agreement for the absence of echogenic foci (AC1 0.53, 95% CI 0.24-0.81) and composition (AC1 0.54, 95% CI 0.36-0.71). The agreement for margins (AC1 0.24, 95% CI 0.15-0.33), echogenicity (AC1 0.34, 95% CI 0.22-0.46), and shape assessment (AC1 0.42, 95% CI 0.13-0.70) was lower. The overall agreement for ACR-TIRADS assessment was AC1 0.29, (95% CI 0.13-0.45). The AC1 of ACR-TIRADS among subgroups was similar. This study found high variation of judgments about ACR-TIRADS risk category and individual features, which poses a potential challenge for the widescale implementation of thyroid nodule risk stratification.

A new Human factors incident taxonomy for members of the public (HFIT-MP): An investigation of escalator incidents

BackgroundEscalators are common across many urban environments, yet, incidents and fatalities can occur during their use. This research takes a Human Factors approach to: 1) produce an Incident Taxonomy for Members of the Public (HFIT-MP); and 2) apply the taxonomy to a unique dataset of serious escalator incidents. MethodsThe HFIT-MP was developed using Human Factors literature, Health and Safety Executive (HSE) guidance, and subject matter experts. 403 narrative escalator incident reports were obtained from the HSE and coded according to the HFIT-MP. Cohen’s Kappa was used for inter-rater agreement and Multiple Correspondence Analysis (MCA) coupled with cluster analysis was used to identify factors that tended to occur together. ResultsThe HFIT-MP consisted of four overarching themes and 25 factors. All factors achieved a Kappa greater than 0.60. Analysis of all incident reports indicated that 93% were falls. MCA identified six groups of factor categories that tended to occur together; for example, falls occurred more frequently in people aged 65 or over, who lost their footing, getting onto the escalator, that was not moving unexpectedly, travelling alone, and moving in a descending direction. ConclusionResults provide health and safety practitioners with target groups (e.g., age), actions (e.g., getting onto the escalator) and locations (e.g., descending escalator) to inform interventions and the need to consider the interaction of multiple factors among incidents. Further application of the HFIT-MP to different sources and modes of data, such as video footage, could further refine the taxonomy for incident investigations.

Agreement Analysis on Angle Characteristics With Automated Gonioscopy.

Automated gonioscopy is a recent method of recording angle pictures. Our study shows that agreement between observers is moderate in most categories used in clinical practice, underlying its clinical usefulness but also room for improvement. Iridocorneal angle (ICA) imaging methods have been recently developed to record gonioscopic images. The purpose of this study was to perform an interrater agreement analysis of ICA photographs. Multicentric cross-sectional observational study. Consecutive patients in 2 ophthalmology departments underwent automated gonioscopy. One picture per quadrant from each eye was selected for randomization. Pictures were graded and analyzed by 4 masked glaucoma specialists regarding angle opening, width, Shaffer and Spaeth gradings and other findings. Fleiss' κ statistics was performed to assess interrater agreement. A total of 50 patients were recruited, with the sample containing a variety of diagnosis including pseudoexfoliation (22%), angle-closure suspect (12%), and pigmentary glaucoma (4%). The sample included phakic (68%) and pseudophakic patients, as well as cases with gonio-implanted surgical devices (10%). There was moderate agreement on angle opening, angle width, presence of angle vessels, and peripheral anterior synechiae (κ=0.435, 0.511, 0.558, 0.488, respectively; P<0.01). Fair agreement was observed regarding Shaffer grading, site of iris root insertion, angle pigmentation features, and the presence of iris processes. Expectedly from a 2-dimensional picture, the configuration of peripheral iris was found to have a poor agreement. Subset analysis on higher quality pictures seemed to improve agreement on pigment but did not further improve on the overall good agreement on angle opening status. Our study shows that automated gonioscopy provides moderate agreement on most clinically relevant features. Binary characteristics such as angle opening or PAS seem to be more robust than more complex angle classification parameters.

Conceptual framework of value management adoption in the Nigerian construction industry

PurposeThe Nigerian construction industry is bedeviled with poor project performance and outcomes which value management (VM) could address if applied. The application of VM on Nigerian construction projects is very minimal due to certain obstacles, namely, lack of VM experts, paucity of knowledge on the techniques, etc., which inhibits the adoption into the construction industry. Therefore, this study aims to develop a conceptual framework of the adoption of VM on construction projects in a typical developing economy.Design/methodology/approachThis study engaged 15 selected VM experts in two rounds of Delphi survey to develop a conceptual framework of VM adoption. The method of data analysis includes mean score, standard deviation, Kendall’s coefficient of concordance, chi-square (χ2) test, interrater agreement analysis and significant level analysis. The developed conceptual framework was sent to a team of local and international VM experts for validation.FindingsThis study reveals that the adoption of VM requires the collective effort of relevant stakeholders in the construction industry. The framework developed presents individual and collective activities to be undertaken by the stakeholders. The activities include training, legislation, government-funded research, etc. Thus, the adoption of innovative management methodology like VM requires the collaboration of academics, construction professional bodies and government parastatals. This will assist in the judicious use of limited construction resources and boost the relevance of the Nigerian construction industry among developing nations and in the global construction market.Originality/valueThis study used the opinions of few construction professionals that can be regarded as VM experts in Nigeria, as against engaging a pool of construction professionals who may not be knowledgeable in VM process. Engaging the few VM experts in the Nigerian construction industry is important to have a valid basis for drawing conclusion, as large questionnaire survey could be possibly filled by inexperienced or unqualified respondents if stringent criteria are not considered at the outset of this study.