Interobserver Agreement Intraclass Correlation Coefficient Research Articles

Influence factors on registration of three-dimensional contrast-enhanced ultrasound fusion imaging in evaluating the ablative margin - A phantom study.

Registration of three-dimensional contrast-enhanced ultrasound fusion imaging (3DCEUS-FI) is time-consuming to obtain high success rate. To investigate the influence factors on registration success rate of 3DCEUS-FI. Water tank phantoms were made to obtain mimicked pre- and post- radiofrequency ablation three-dimensional contrast-enhanced ultrasound (3DCEUS) and CT images. Orthogonal trials were designed according to factors including size, depth, enhancement level of mimicked tumor, diameter and number of mimicked adjacent vessels. Mimicked pre- and post-RFA 3DCEUS images of 72 trials were fused to assess ablative margin (AM) by two radiologists. With CT images as standard, 3DCEUS-FI accuracy was considered as the consistency of AM evaluation. The inter-observer agreement and the influence factors on registration success rates were analyzed. The intraclass correlation coefficient (ICC) for the consistency of AM evaluation between CT and 3DCEUS-FI in x-axis, y-axis or z-axis was 0.840∼0.948 (P < 0.001). The ICC for inter-observer agreement was 0.840∼0.948 (P < 0.001). The success rates of registration within mimicked vessels with diameter of 2 mm were significantly lower than those with diameter of 3 mm and 4 mm. The mimicked AM measured by 3DCEUS-FI had high accuracy and inter-observer agreement. Diameter of the mimicked adjacent vessels was significantly related to success rate of registration.

Inter-observer agreement of preoperative cardiopulmonary exercise test interpretation in major abdominal surgery

BackgroundAccurate determination of cardiopulmonary exercise test (CPET) derived parameters is essential to allow for uniform preoperative risk assessment. The objective of this prospective observational study was to evaluate the inter-observer agreement of preoperative CPET-derived variables by comparing a self-preferred approach with a systematic guideline-based approach.MethodsTwenty-six professionals from multiple centers across the Netherlands interpreted 12 preoperative CPETs of patients scheduled for hepatopancreatobiliary surgery. Outcome parameters of interest were oxygen uptake at the ventilatory anaerobic threshold (V̇O2VAT) and at peak exercise (V̇O2peak), the slope of the relationship between the minute ventilation and carbon dioxide production (V̇E/V̇CO2-slope), and the oxygen uptake efficiency slope (OUES). Inter-observer agreement of the self-preferred approach and the guideline-based approach was quantified by means of the intra-class correlation coefficient.ResultsAcross the complete cohort, inter-observer agreement intraclass correlation coefficient (ICC) was 0.76 (95% confidence interval (CI) 0.57–0.93) for V̇O2VAT, 0.98 (95% CI 0.95–0.99) for V̇O2peak, and 0.86 (95% CI 0.75–0.95) for the V̇E/V̇CO2-slope when using the self-preferred approach. By using a systematic guideline-based approach, ICCs were 0.88 (95% CI 0.74–0.97) for V̇O2VAT, 0.99 (95% CI 0.99–1.00) for V̇O2peak, 0.97 (95% CI 0.94–0.99) for the V̇E/V̇CO2-slope, and 0.98 (95% CI 0.96–0.99) for the OUES.ConclusionsInter-observer agreement of numerical values of CPET-derived parameters can be improved by using a systematic guideline-based approach. Effort-independent variables such as the V̇E/V̇CO2-slope and the OUES might be useful to further improve uniformity in preoperative risk assessment in addition to, or in case V̇O2VAT and V̇O2peak are not determinable.

Craniometric parameters for the evaluation of platybasia and basilar invagination on magnetic resonance imaging: a reproducibility study.

ObjectiveThe present study aims to perform a reproducibility study of the clivus-canal angle (CCA), Welcker’s basal angle (WBA), and the distance from the odontoid process to Chamberlain’s line (DOCL) on magnetic resonance imaging (MRI).Materials and MethodsTwo medical students and two radiologists respectively evaluated 100 and 50 consecutive MRI scans of adult skulls, selected randomly. Each examiner, working independently and blinded to the previous results, performed readings for each patient on two different occasions. Measurements were performed in T1-weighted sequences acquired in the midsagittal plane. The levels of intraobserver reproducibility and interobserver agreement were evaluated by calculating the intraclass correlation coefficients (ICCs) and the corresponding 95% confidence intervals.ResultsThe mean values obtained by the examiners were 150º for the CCA, 130º for the WBA, and 2.5 mm for the DOCL. The ICC for interobserver agreement was 0.980, 0.935, and 0.967, for the CCA, WBA, and DOCL, respectively, for the students, compared with 0.977, 0.941, and 0.982, respectively, for the radiologists, and 0.980, 0.992, and 0.990, respectively, for all of the examiners together. In the analysis of intraobserver agreement, the ICC ranged from 0.929 to 0.959 for the CCA, from 0.918 to 0.964 for the WBA, and from 0.918 to 0.981 for the DOCL.ConclusionThe measurement of the CCA, WBA, and DOCL appears to show excellent intraobserver reproducibility and interobserver agreement on MRI.

Us Elastography Using Carotid Artery Pulsation May Increase the Diagnostic Accuracy for Thyroid Nodules with Us - Pathology Discordance

To prospectively evaluate the diagnostic performance of ultrasound elastography (USE) using carotid arterial pulsation and to determine interobserver and intraobserver agreements of USE for thyroid nodules. US examination and USE using carotid artery pulsation were performed in 148 patients with 173 nodules in a prospective design. The US features and elasticity contrast index (ECI) were assessed by observer 1 and the ECI was reassessed by observer 2. ROC curve analysis was performed to evaluate the diagnostic performance and to compare the AUC (Az) values of gray-scale US, ECI, and a combined assessment. Intraclass correlation coefficient (ICC) value was used to evaluate the interobserver and intraobserver agreements in the measured ECI values. Among a total of 173 nodules, 95 nodules were malignant and 78 nodules were benign. The mean ECI was significantly higher in malignant nodules (3.1 ± 1.5) than in benign nodules (1.7 ± 0.8) (p < 0.001). The Az values of each dataset were 0.768 (95% CI, 0.698–0.828) for gray-scale US, 0.695 (0.621–0.763) for ECI, and 0.782 (0.713–0.841) for a combined assessment. ICC for interobserver agreement was 0.96 (p < 0.001), and for intraobserver agreement were 0.97 (p < 0.001) and 0.98 (p < 0.001) for observer 1 and 2, respectively. Excellent interobserver and intraobserver agreement was found in thyroid USE. USE using carotid artery pulsation may have a potential role to increase the diagnostic accuracy for nodules with US-pathology discordance with reproducible results.

New MRI Grading System for the Cervical Canal Stenosis

The purpose of this study was to propose a new MRI grading system for cervical canal stenosis and to evaluate the reproducibility of the system. Cervical canal stenosis was classified according to the T2-weighted sagittal images into the following grades: grade 0, absence of canal stenosis; grade 1, subarachnoid space obliteration exceeding 50%; grade 2, spinal cord deformity; and grade 3, spinal cord signal change. The MRI scans of 82 patients (37 men and 45 women; mean age, 65.2 years; range, 60-86 years) were independently analyzed by six radiologists. Interobserver and intraobserver agreements were analyzed using intraclass correlation coefficient (ICC), along with the percentage agreement and kappa statistics. The ICC for interobserver agreement was 0.716-0.802, indicating good-to-excellent agreement. For the distinction among the four grades, the percentage of agreement was 63-64% (κ = 0.60-0.62). The percentage of agreement for the presence of cervical canal stenosis (grade 0 vs grades 1, 2, and 3) was 79-85% (κ = 0.51-0.59). The percentage of agreement for insignificant (grade 0-1) or significant (grade 2-3) stenosis was 81-85% (κ = 0.57-0.66). The percentage of agreement for the presence of spinal cord signal change (grade 0-2 vs grade 3) was 92-95% (κ = 0.70-0.73). The overall intraobserver agreement was excellent, as determined by an ICC of 0.768. The new grading system provides a reliable assessment of cervical canal stenosis.

Data quality and practical challenges of thyroid volume assessment by ultrasound under field conditions - observer errors may affect prevalence estimates of goitre

BackgroundThe ultrasonographic estimation of thyroid size has been advocated as being more precise than palpation to diagnose goitre. However, ultrasound also requires technical proficiency. This study was conducted among Saharawi refugees, where goitre is highly prevalent. The objectives were to assess the overall data quality of ultrasound measurements of thyroid volume (Tvol), including the intra- and inter-observer agreement, under field conditions, and to describe some of the practical challenges encountered.MethodsIn 2007 a cross-sectional study of 419 children (6-14 years old) and 405 women (15-45 years old) was performed on a population of Saharawi refugees with prevalent goitre, who reside in the Algerian desert. Tvol was measured by two trained fieldworkers using portable ultrasound equipment (examiner 1 measured 406 individuals, and examiner 2, 418 individuals). Intra- and inter-observer agreement was estimated in 12 children selected from the study population but not part of the main study. In the main study, an observer error was found in one examiner whose ultrasound images were corrected by linear regression after printing and remeasuring a sample of 272 images.ResultsThe intra-observer agreement in Tvol was higher in examiner 1, with an intraclass correlation coefficient (ICC) of 0.97 (95% CI: 0.91, 0.99) compared to 0.86 (95% CI: 0.60, 0.96) in examiner 2. The ICC for inter-observer agreement in Tvol was 0.38 (95% CI: -0.20, 0.77). Linear regression coefficients indicated a significant scaling bias in the original measurements of the AP and ML diameter and a systematic underestimation of Tvol (a product of AP, ML, CC and a constant). The agreement between re-measured and original Tvol measured by ICC (95% CI) was 0.76 (0.71, 0.81). The agreement between re-measured and corrected Tvol measured by ICC (95% CI) was 0.97 (0.96, 0.97).ConclusionsAn important challenge when using ultrasound to assess thyroid volume under field conditions is to recruit and train qualified personnel to perform the measurements. Methodological studies are important to assess data quality and can facilitate statistical corrections and improved estimates.

Noninvasive Assessment of Liver Fibrosis and Portal Hypertension With Transient Elastography

Hepatic fibrogenesis, the "final" common result of injury to the liver, is believed to be a critical factor leading to hepatic dysfunction and may be important in the pathogenesis of portal hypertension. Thus, accurate assessment of the degree of fibrosis is important clinically. For many years, examination of hepatic histopathology has been considered to be the "gold standard" tool used to assess fibrosis. However, liver biopsy is invasive, and in many instances not favored by patients or physicians. Thus, alternative approaches to measure liver fibrosis would be extremely attractive. To the extent that transient elastography is able to measure "liver stiffness," which is proportional to the degree of liver fibrosis, this technique holds great promise.