Krippendorff's Coefficient Research Articles

Interobserver variability in thyroid ultrasound.

Ultrasound evaluation of thyroid nodules is the preferred technique, but it is dependent on operator interpretation, leading to inter-observer variability. The current study aimed to determine the inter-physician consensus on nodular characteristics, risk categorization in the classification systems, and the need for fine needle aspiration puncture. Four endocrinologists from the same center blindly evaluated 100 ultrasound images of thyroid nodules from 100 different patients. The following ultrasound features were evaluated: composition, echogenicity, margins, calcifications, and microcalcifications. Nodules were also classified according to ATA, EU-TIRADS, K-TIRADS, and ACR-TIRADS classifications. Krippendorff's alpha test was used to assess interobserver agreement. The interobserver agreement for ultrasound features was: Krippendorff's coefficient 0.80 (0.71-0.89) for composition, 0.59 (0.47-0.72) for echogenicity, 0.73 (0.57-0.88) for margins, 0.55 (0.40-0.69) for calcifications, and 0.50 (0.34-0.67) for microcalcifications. The concordance for the classification systems was 0.7 (0.61-0.80) for ATA, 0.63 (0.54-0.73) for EU-TIRADS, 0.64 (0.55-0.73) for K-TIRADS, and 0.68 (0.60-0.77) for K-TIRADS. The concordance in the indication of fine needle aspiration puncture (FNA) was 0.86 (0.71-1), 0.80 (0.71-0.88), 0.77 0.67-0.87), and 0.73 (0.64-0.83) for systems previously described respectively. Interobserver agreement was acceptable for the identification of nodules requiring cytologic study using various classification systems. However, limited concordance was observed in risk stratification and many ultrasonographic characteristics of the nodules.

Why do delays to diagnosis and treatment of lung cancer occur? A mixed methods study of insights from Australian clinicians.

Delays in lung cancer diagnosis and treatment can impact survival. We explored reasons for delays experienced by patients with lung cancer to identify themes and strategies for improvement. We used national timeframe recommendations and standardized definitions to identify General Practitioners and specialists caring for 34 patients who experienced delays in our previous Medicare data linkage study. Clinicians participated in a survey and interview, including qualitative (exploratory, open-ended questions) and quantitative (rating scales) components. Exploratory content analysis, cross-case triangulation, and descriptive statistics were performed. Krippendorff's coefficient was used to assess level of agreement between clinicians and patients, and among clinicians, on perceived delays. Overall, 27 out of 50 (54%) eligible clinicians participated (including 11 respiratory physicians and seven medical oncologists). Dominant themes for perceived causes of delay included referral barriers, limited General Practitioner (GP) awareness of subtle clinical presentations, insufficient radiology interpretation, and lack of cancer coordinators. "Unavoidable" delays may occur due to clinical circumstances. Awareness and uptake of referral and timeframe guidelines were low, with clinicians using professional networks over guidelines. There was no consistent agreement on perceived delays between patients and clinicians, and among clinicians (Krippendorff's coefficient .03 [P=.8]). Strategies for minimizing avoidable delays include efficient GP to specialist referral and more lung cancer coordinators to assist with patient expectations and waitlist management. Clinicians' reliance on experience, rather than guidelines, indicates need to review guideline utility. Raising awareness of benchmarks and unavoidable barriers may recalibrate perceptions of "delays" to diagnosis and treatment of lung cancer.

Muscle MR imaging in tubular aggregate myopathy.

PurposeTo evaluate with Magnetic Resonance (MR) the degree of fatty replacement and edematous involvement in skeletal muscles in patients with Tubular Aggregate Myopathy (TAM). To asses the inter-observer agreement in evaluating muscle involvement and the symmetry index of fatty replacement.Materials and Methods13 patients were evaluated by MR to ascertain the degree of fatty replacement (T1W sequences) according to Mercuri's scale, and edema score (STIR sequences) according to extent and site.ResultsFatty replacement mainly affects the posterior superficial compartment of the leg; the anterior compartment is generally spared. Edema was generally poor and almost only in the superficial compartment of the leg. The inter-observer agreement is very good with a Krippendorff's coefficient >0.9. Data show a total symmetry in the muscular replacement (McNemar-Bowker test with p = 1).ConclusionsMR reveals characteristic muscular involvement, and is a reproducible technique for evaluation of TAM. There may also be a characteristic involvement of the long and short heads of the biceps femoris. It is useful for aimed biopsies, diagnostic hypotheses and evaluation of disease progression.

The biometrical comparison of cardiac imaging methods

Objectives: Biometrical comparison procedures for cardiac imaging methods with continuous outcome are reviewed mainly concentrating on assessment and design adequate comparison of accuracy and precision. Univariate graphical and numerical representation of corresponding deviations is outlined to derive a ‘check list’ of minimum information necessary to compare the measurement methods. Data: The methods reviewed here are illustrated by the comparison of standard 2DE bidimensional cardial volumetry versus assessment using TDE colour imaging in 28 normal probands. Sources: The paired t-test and the corresponding confidence interval approach are used to assess deviations in location of two imaging methods; the test procedures of Maloney and Rastogi Hahn and Nelson and Grubbs are surveyed as proposals for the comparison of precisions in paired data. The Krippendorff coefficient and the Bradley/Blackwood test are illustrated as surrogate measures for method concordance. Conclusions: Since these methods can be performed by simple modification of standard options available in most statistics software packages, this review intends to enable cardiologists to choose appropriate methods for statistical data analysis and representation on their own.