Mean Intraclass Correlation Coefficient Value Research Articles

Automatic segmentation and quantification of the optic nerve on MRI using a 3D U-Net.

Pathological conditions associated with the optic nerve (ON) can cause structural changes in the nerve. Quantifying these changes could provide further understanding of disease mechanisms. We aim to develop a framework that automatically segments the ON separately from its surrounding cerebrospinal fluid (CSF) on magnetic resonance imaging (MRI) and quantifies the diameter and cross-sectional area along the entire length of the nerve. Multicenter data were obtained from retinoblastoma referral centers, providing a heterogeneous dataset of 40 high-resolution 3D T2-weighted MRI scans with manual ground truth delineations of both ONs. A 3D U-Net was used for ON segmentation, and performance was assessed in a tenfold cross-validation () and on a separate test-set () by measuring spatial, volumetric, and distance agreement with manual ground truths. Segmentations were used to quantify diameter and cross-sectional area along the length of the ON, using centerline extraction of tubular 3D surface models. Absolute agreement between automated and manual measurements was assessed by the intraclass correlation coefficient (ICC). The segmentation network achieved high performance, with a mean Dice similarity coefficient score of 0.84, median Hausdorff distance of 0.64mm, and ICC of 0.95 on the test-set. The quantification method obtained acceptable correspondence to manual reference measurements with mean ICC values of 0.76 for the diameter and 0.71 for the cross-sectional area. Compared with other methods, our method precisely identifies the ON from surrounding CSF and accurately estimates its diameter along the nerve's centerline. Our automated framework provides an objective method for ON assessment in vivo.

Reliability of electromyography during 2000 m rowing ergometry

PurposeThis study aimed to investigate the reliability of surface electromyography (EMG) assessed at seven muscles during three repeated 2000 m rowing ergometer sessions.MethodsTwelve male well-trained rowers participated in a repeated measures design, performing three 2000 m rowing ergometer sessions interspersed by 3–7 days (S1, S2, S3). Surface electrodes were attached to the gastrocnemius, biceps femoris, gluteus maximus, erector spinae, vastus medialis, rectus abdominis and latissimus dorsi for EMG analysis.ResultsNo differences existed between 2000 m sessions for EMG amplitude for any of the seven muscles (p = 0.146–0.979). Mean coefficient of variation of EMG for 6 of 7 muscles was ‘acceptable’ (12.3–18.6%), although classed as ‘weak’ for gastrocnemius (28.6%). Mean intra-class correlation coefficient values across muscles ranged from ‘moderate’ to ‘very large’ (0.31–0.89). Within-session EMG activation rates of vastus medialis were greater during 0–500 m and 1500–2000 m segments, compared with 500–1000 m and 1000–1500 m (p < 0.05). Values for biceps femoris and gluteus maximus were significantly higher during 1500–2000 m compared to 500–1000 m and 1000–1500 m (p < 0.05). The general pattern was for higher activation rates during 0–500 m and 1500–2000 m compared to 500–1000 m and 1000–1500 m. However, there were no between-sessions differences in EMG for any of the 500 m segments (p > 0.05).ConclusionReliability of EMG values over repeated 2000 m sessions was generally ‘acceptable’. However, EMG was seemingly not sensitive enough to detect potential changes in neural activation between-sessions, with respect to changes in pacing strategy.

The effect of general anesthesia on the test-retest reliability of resting-state fMRI metrics and optimization of scan length.

Resting-state functional magnetic resonance imaging (rs-fMRI) has been known as a powerful tool in neuroscience. However, exploring the test–retest reliability of the metrics derived from the rs-fMRI BOLD signal is essential, particularly in the studies of patients with neurological disorders. Here, two factors, namely, the effect of anesthesia and scan length, have been estimated on the reliability of rs-fMRI measurements. A total of nine patients with drug-resistant epilepsy (DRE) requiring interstitial thermal therapy (LITT) were scanned in two states. The first scan was performed in an awake state before surgery on the same patient. The second scan was performed 2 weeks later under general anesthesia necessary for LITT surgery. At each state, two rs-fMRI sessions were obtained that each one lasted 15 min, and the effect of scan length was evaluated. Voxel-wise rs-fMRI metrics, including the amplitude of low-frequency fluctuation (ALFF), the fractional amplitude of low-frequency fluctuation (fALFF), functional connectivity (FC), and regional homogeneity (ReHo), were measured. Intraclass correlation coefficient (ICC) was calculated to estimate the reliability of the measurements in two states of awake and under anesthesia. Overall, it appeared that the reliability of rs-fMRI metrics improved under anesthesia. From the 15-min data, we found mean ICC values in awake state including 0.81, 0.51, 0.65, and 0.84 for ALFF, fALFF, FC, and ReHo, respectively, as well as 0.80, 0.59, 0.83, and 0.88 for ALFF, fALFF, FC, and ReHo, respectively, under anesthesia. Additionally, our findings revealed that reliability increases as the function of scan length. We showed that the optimized scan length to achieve less variability of rs-fMRI measurements was 3.1–7.5 min shorter in an anesthetized, compared to a wakeful state.

Analysis of Sagittal Position Changes of the Condyle After Mandibular Setback Surgery Across the Four Different Types of Plating Systems.

The authors analyzed the three-dimensional postoperative condylar position change across the plating systems. This retrospective study was conducted with the patients who underwent bilateral sagittal split ramus osteotomy with setback surgery. The condylar change was analyzed from preoperative cone-beam computed tomography to postoperative 1 month (T1) and postoperative 6 months (T2) using superimposition software, automatically merging based on the anterior cranial base. The condylar changes during T1 and T2 were analyzed across the four types of plates (4-hole sliding, heart-shaped, 3-hole sliding, and 4-hole conventional) Mean intraclass correlation coefficient values were consistently high for each measurement (>0.850). During T1, the conventional plate had a decreased condylar anterior distance when compared with the 3-hole sliding plate (P = 0.032). During T2, the conventional plate had an increased condylar posterior distance when compared with the 3-hole sliding plate (P = 0.031). Superimposition software based on the anterior cranial base could be available for measurement of condylar position with highly reproducible results. After bilateral sagittal split ramus osteotomy, the 3-hole sliding plate could effectively compensate for the anterior displacement of the condyle compared to other plates.

Reliability of marginal bone level measurements on digital panoramic and digital intraoral radiographs

ObjectivesThe aim of this study was to evaluate the reliability of bitewing and panoramic radiographs in marginal bone level measurements in terms of inter- and intra-observer agreement.MethodsFifty paired bitewing and panoramic images were used. Eight observers measured marginal bone level at the mesial and distal surfaces of tooth 25 and tooth 35. Thus, in total 100 radiographs of 400 measurements were obtained for each observer. To evaluate intra-observer agreement, three observers re-evaluated the radiographs after a minimum of 1 month. Intra-class correlation coefficient (ICC) was applied to evaluate the inter- and intra-observer agreement. The t test was applied to assess possible difference in measurement between bitewing and panoramic radiographs.ResultsThe mean ICC value of inter-observer agreement was 0.85 for bitewing and 0.66 for panoramic radiographs. The mean intra-observer agreement was 0.92 and 0.76 for bitewing and panoramic radiographs, respectively. There was no statistically significant difference between bitewing and panoramic radiographs in measurements of marginal bone level on maxillary tooth 25, whereas a statistically significant difference was found between the two image modalities on mandible tooth 35.ConclusionBitewing examination should be the choice of image modality for assessment of marginal bone level at premolar region due to good to excellent reliability and low radiation dose. However if a panoramic radiograph already exists, a rough estimation of marginal bone level at premolar region is clinically acceptable bearing in mind that the bone height of the mandible premolar region might be overestimated as compared to bitewing radiograph.

Development and validation of the autotransplanted maxillary canine radiological index.

The purpose of this study was to propose and validate an index evaluating 2D and 3D radiographic variables of autotransplanted maxillary canines. Setting and sample population are from the Department of Oral and Maxillofacial Surgery at University Hospitals Leuven. Eight oral‐maxillofacial surgeons rated 12 autotransplanted maxillary canines and adjacent bone using 11 rating variables. A new autotransplanted maxillary canine radiological index (AMCRI) was proposed. It consisted of 11 variables. These variables were based on 2D (intraoral) and 3D Cone Beam Computed Tomography (CBCT) radiographs. Intraclass correlation coefficient (ICC) and Fleiss's kappa statistics were performed to analyze intrarater and interrater agreement. Considering cumulative assessment of the AMCRI, the mean ICC value for the interrater agreement of the eight examiners was 0.94, representing an excellent agreement. Intrarater agreement was 0.91. The AMCRI is an objective tool in rating radiological outcome of autotransplanted canines and adjacent bone, when compared with the contralateral canine.

Observation of different tumor motion magnitude within liver and estimate of internal motion margins in postoperative patients with hepatocellular carcinoma.

AimsTo assess motion magnitude in different parts of the liver through surgical clips in postoperative patients with hepatocellular carcinoma and to examine the correlation between the clip and diaphragm motion.MethodsFour-dimensional computed tomography images from 30 liver cancer patients under thermoplastic mask immobilization were selected for this study. Three to seven surgical clips were placed in the resection cavity of each patient. The liver volume on computed tomography image was divided into the right upper (RU), right middle (RM), right lower (RL), hilar, and left lobes. Agreement between the clip and diaphragm motion was assessed by calculating intraclass correlation coefficient, and Bland–Altman analysis (Diff). Furthermore, population-based and patient-specific margins for internal motion were evaluated.ResultsThe clips located in the RU lobe showed the largest motion, (7.5±1.6) mm, which was significantly more than in the RM lobe (5.7±2.8 mm, p=0.019), RL lobe (4.8±3.3 mm, p=0.017), and hilar lobe (4.7±2.7 mm, p<0.001) in the cranial–caudal direction. The mean intraclass correlation coefficient values between the clip and diaphragm motion were 0.915, 0.735, 0.678, 0.670, and the mean Diff values between them were 0.1±0.8 mm, 2.3±1.4 mm, 3.1±2.0 mm, 2.4±1.5 mm, when clips were located in the RU lobe, RM lobe, RL lobe, and hilar lobe, respectively. The clip and diaphragm motions had high concordance when clips were located in the RU lobe. Internal margin can be reduced from 5 mm in the cranial–caudal direction based on patient population average and to 3 mm based on patient-specific margins.ConclusionsThe motion magnitude of clips varied significantly depending on their location within the liver. The diaphragm was a more appropriate surrogate for tumor located in the RU lobe than for other lobes.

Test-retest reliability of 3D spiral fast-spin-echo pseudo-continuous arterial spin labeling for cerebral perfusion imaging of subcortical gray matter in healthy adults

To evaluate the test-retest reliability of 3D spiral fast-spin-echo (FSE) pseudo-continuous arterial spin labeling (3D pc-ASL) for cerebral perfusion imaging of the subcortical gray matter in healthy adults in resting state. 3D spiral FSE ASL and 3D T1-weighted fast spoiled gradient recalled echo (3D T1-FSPGR) sequences were used for cerebral perfusion imaging in 8 healthy adult subjects, and a repeat imaging examination was performed after one week. The subcortical gray matter structures including the putamen, globus pallidus, caudate nucleus, thalamus, amygdala and hippocampus were segmented on the brain structural 3D images to generate the cerebral blood flow (CBF) map. The CBF value was extracted based on the segmented images and the CBF maps. The reliability and reproducibility of the measurements were evaluated using the intraclass correlation coefficient (ICC) and Bland-Altman plot. The mean ICC value of the subcortical gray matter structures was 0.88∓0.04 (range, 0.80-0.93). The Bland-Altman plot analysis demonstrated that the differences between the two measurements at all the points corresponding to the subcortical gray matter structures were within 95% CI of the limits of agreement. 3D spiral FSE pseudo-continuous ASL is a reliable method for assessing the perfusion of the cerebral subcortical gray matter structures.

Ensuring consistent reporting of clinical pharmacy services to enhance reproducibility in practice: an improved version of DEPICT.

DEPICT (Descriptive Elements of Pharmacist Intervention Characterization Tool) was created in response to the frequently reported issue of poor intervention description across studies assessing the impact of clinical pharmacy activities. The aim of this study was to create an improved version of DEPICT (i.e. DEPICT 2) to better characterize clinical pharmacy services in order to ensure consistent reporting, therefore enhancing reproducibility of interventions in practice. A qualitative approach through a thematic content analysis was performed to identify components of pharmacist interventions described in 269 randomized controlled trials. A preliminary version of DEPICT 2 was applied independently by two authors to a random sample of 85 of the 269 RCTs and reliability determined by the prevalence-adjusted bias-adjusted kappa (PABAK) or the intraclass correlation coefficient (ICC). The final version of DEPICT 2 was compared against DEPICT 1. The final version of DEPICT 2 comprised 146 items and 11 domains. The inter-rater agreement analysis showed that DEPICT presented good to optimal reproducibility, with a mean PABAK value of 0.87 (95% CI 0.85-0.89) and a mean ICC value of 0.88 (95% CI 0.62-1.14). The mean difference between items checked in the two versions (DEPICT 2 - DEPICT 1) was 10.58 (95% CI 9.55-11.61), meaning that approximately 11 more components were identified in the new version of DEPICT. DEPICT 2 is a reliable tool to characterize components of clinical pharmacy services, which should be used to ensure consistent reporting of interventions to allow their reproducibility in practice.

Is Diaphragm Motion a Good Surrogate for Liver Tumor Motion?

To evaluate the relationship between liver tumor motion and diaphragm motion. Fourteen patients with hepatocellular carcinoma (10 of 14) or liver metastases (4 of 14) undergoing radiation therapy were included in this study. All patients underwent single-slice cine-magnetic resonance imaging simulations across the center of the tumor in 3 orthogonal planes. Tumor and diaphragm motion trajectories in the superior-inferior (SI), anterior-posterior (AP), and medial-lateral (ML) directions were obtained using an in-house-developed normalized cross-correlation-based tracking technique. Agreement between the tumor and diaphragm motion was assessed by calculating phase difference percentage, intraclass correlation coefficient, and Bland-Altman analysis (Diff). The distance between the tumor and tracked diaphragm area was analyzed to understand its impact on the correlation between the 2 motions. Of all patients, the mean (±standard deviation) phase difference percentage values were 7.1% ± 1.1%, 4.5% ± 0.5%, and 17.5% ± 4.5% in the SI, AP, and ML directions, respectively. The mean intraclass correlation coefficient values were 0.98 ± 0.02, 0.97 ± 0.02, and 0.08 ± 0.06 in the SI, AP, and ML directions, respectively. The mean Diff values were 2.8 ± 1.4 mm, 2.4 ± 1.1 mm, and 2.2 ± 0.5 mm in the SI, AP, and ML directions, respectively. Tumor and diaphragm motions had high concordance when the distance between the tumor and tracked diaphragm area was small. This study showed that liver tumor motion had good correlation with diaphragm motion in the SI and AP directions, indicating diaphragm motion in the SI and AP directions could potentially be used as a reliable surrogate for liver tumor motion.

Reliability analysis of two methods for measuring active enamel demineralization: An in vitro study.

Objective:The aim was to evaluate the intra- and inter-examiner reliability of two methods, namely, digital caliper [DC] and computerized image analysis software (Image Tool [IT] version 4.1) for measuring the area of an active enamel demineralization lesion. The effect on that measurement of a window that delimits the lesion was also assessed.Materials and Methods:Thirty-eight circular artificial enamel demineralization lesions (r = 2 mm) were produced on bovine teeth in vitro, which were then measured three times by three examiners, with and without the presence of a window delimiting the area to be measured. The data were analyzed by the intraclass correlation coefficient (ICC) with 95% confidence interval (CI) and by the Altman and Bland analysis.Results:The intra-examiner correlation using the IT method showed excellent reproducibility (mean ICC values 0.922-0.970 with the delimiting window and 0.915-0.990 without the window). However, the intra-examiner correlation using the DC method showed comparatively less reproducibility in the measurements (mean ICC values 0.458-0.648 with the delimiting window and 0.378-0.665 without the window). The inter-examiner correlations showed very good reproducibility of the measurements regardless of the presence or absence of the window for both the DC method (0.811-0.846) and IT method (0.953-0.994). However, the latter method showed less variability within the measurements.Conclusion:However, statistically no significant difference was found between both methodologies. Nevertheless, computerized image analysis with the Image Tool software demonstrated higher intra- and inter-examiner reliability than the digital caliper method, in estimating the area of the enamel demineralization lesion, regardless of the delimiting window.

Influence of the family cluster effect on psychosocial variables in families undergoing BRCA1/2 genetic testing for cancer susceptibility

This study evaluated the influence of the family cluster effect on behavioral and psychological variables among individuals undergoing BRCA1/2 genetic testing for cancer susceptibility. French-Canadian women (n = 552) and men (n = 104) from 140 different families undergoing BRCA1/2 testing between years 1998 and 2004 completed a self-administered questionnaire on a wide range of behavioral and psychological variables. The impact of the family cluster effect on 29 variables was assessed using the intraclass correlation coefficient (ICC) as computed from multilevel random-effect models. ICC values were statistically significant for 22 of the 29 variables. Overall, the mean ICC value was 0.10. The ICC values for knowledge about hereditary breast and ovarian cancer and frequency of alcohol consumption were 0.25. The ICC value for the Impact of Event Scale, which is a commonly used measure of cancer-specific distress, was 0.14. Our results suggest that the family cluster effect influences the majority of behavioral and psychosocial variables. When studying psychosocial aspects of genetic testing for cancer susceptibility, the family cluster effect should be routinely accounted for when determining sample size and statistical methods.

Development and Reliability of an Observation Method to Assess Food Intake of Young Children in Child Care

To our knowledge, a direct observation protocol for assessing dietary intake among young children in child care has not been published. This article reviews the development and testing of a diet observation system for child care facilities that occurred during a larger intervention trial. Development of this system was divided into five phases, done in conjunction with a larger intervention study; (a) protocol development, (b) training of field staff, (c) certification of field staff in a laboratory setting, (d) implementation in a child-care setting, and (e) certification of field staff in a child-care setting. During the certification phases, methods were used to assess the accuracy and reliability of all observers at estimating types and amounts of food and beverages commonly served in child care. Tests of agreement show strong agreement among five observers, as well as strong accuracy between the observers and 20 measured portions of foods and beverages with a mean intraclass correlation coefficient value of 0.99. This structured observation system shows promise as a valid and reliable approach for assessing dietary intake of children in child care and makes a valuable contribution to the growing body of literature on the dietary assessment of young children.

Reliability and validity of plain radiographs to assess angulation of small finger metacarpal neck fractures: Human cadaveric study

To quantify reliability and validity of plain radiographs for assessing the degree of small finger metacarpal neck fracture angulation, we created typical two-fragments fractures in 30 adult cadaveric specimens. Reliability and validity of different radiographic measurement methods were determined by the intraclass correlation coefficient (ICC) and the Bland and Altman graphical approach. Intraobserver and interobserver reliability was high with any radiographic measurement method. Mean ICCs values (95% confidence intervals) varied from 0.76 (0.56-0.88) to 1.00 (0.99-1.00). The graphical approach confirmed good agreement. Validity was substantial when the fracture angle was measured between the line along the longitudinal axis of the metacarpal shaft and the line from the center of the metacarpal head to the fracture site on lateral radiographs. Mean ICCs values varied from 0.70 (0.36-0.86) to 0.79 (0.5-0.90). The graphical analysis also indicated good agreement. In contrast, considerable lack of validity was observed when the angle was measured on oblique radiographs. Although the mean ICCs values varied from 0.68 (0.12-0.88) to 0.74 (0.05-0.90), suggesting substantial correlation, the graphical analysis provided evidence for poor validity. There was systematic bias with oblique radiographs consistently producing higher readings (up to 35 degrees ). In summary, reliability and validity are good only when the degree of small finger metacarpal neck fracture angulation is measured after drawing lines on lateral radiographs. Oblique radiograph measurements consistently produce higher readings.