Intraclass Correlation Coefficient Estimates Research Articles

Multi Detector-Row Computed Tomography (MDCT) had excellent reproducibility of coronary calcium measurements

Objective To assess reproducibility of coronary calcium measurements with Multi Detector-Row Computed Tomography (MDCT) imaging and to evaluate whether different measurement protocols, slice thickness, and cardiovascular risk factors affect inter and intra-observer reproducibility. Study Design and Setting In a cross-sectional design, the study population comprised 199 healthy postmenopausal women. Coronary calcium was assessed using a 16-MDCT (Philips Mx 8000 IDT 16). Images were made using 1.5 and 3.0 mm slice thicknesses. To assess inter and intra-observer reproducibility, the images were read by two observers. One observer read the images of 52 subjects twice. The Agatston score, a volume, and a mass measurement were used to quantify coronary calcium. Reproducibility was determined by estimation of mean, absolute, and relative differences between scores of the observers and by estimation of intraclass correlation coefficients (ICC). Results 60.3% participants ( n = 120) had a positive calcium score. Median Agatston score for the first observer was 2.20 with a range of 0–2,019.The reproducibility of coronary calcium measurements between and within observers was excellent with ICC of >0.95, and small mean, absolute, and relative differences. Reproducibility findings were similar for 1.5 mm slices as for 3.0 mm slices, and equal for Agatston, volume, and mass measurements. Conclusion Reproducibilility of coronary calcium measurement using MDCT is excellent, irrespective of slice thickness and type of calcium parameter.

Reliability studies can be designed more efficiently by using variance components estimates from different sources

Objectives Reliability studies are frequently organized within the context of a large (multicenter) study, with only a small sample of subjects measured by the observers of the large study. To estimate interobserver reliability, data from the large study are not frequently used. In this article, the advantages of combining data from the reliability study and the large study to improve the estimation of intra-class correlation coefficients (ICCs) are highlighted. Study Design and Setting This was done within the scope of estimating fat percentages in the Project On Preterm and Small-for-gestational-age infants-19 (POPS-19) study and with simulations. To calculate ICCs, three approaches were used: (1) the classical approach using data from a reliability study only, (2) the combined variances approach using inter-subject variances from the POPS-19 study, and (3) the maximum likelihood approach using all data. Results The ICCs (95% confidence interval [CI]) for fat percentage calculated by the three approaches were 0.84 (0.57, 0.99), 0.94 (0.90, 0.97), and 0.94 (0.88, 0.97), respectively. Conclusion The efficient use of data by combining data from a small reliability study with the data from the large study itself for the calculation of ICCs will lead to more precise ICCs.

Reliability of Scores from Physical Activity Monitors in Adults with Multiple Sclerosis

We examined the reliability of scores from physical activity monitors in a sample of 193 individuals with multiple sclerosis (MS) who wore a pedometer and an accelerometer for a 7-day period. There were no significant differences among days for the pedometer (p = .12) or the accelerometer (p = .15) indicating that week and weekend days can be analyzed in a single intra-class correlation (ICC) analytic model. The 7 days of monitoring yielded ICC estimates of .93 for both the pedometer and accelerometer, and a minimum of 3 days yielded a reliability of .80 for both the pedometer and accelerometer. Results indicated that physical activity monitor scores are reliable measures of physical activity for individuals with MS.

Evaluating measurement error in estimates of worker exposure assessed in parallel by personal and biological monitoring.

While studies indicate that the attenuating effects of imperfectly measured exposure can be substantial, they have not had the requisite data to compare methods of assessing exposure for the same individuals monitored over common time periods. We examined measurement error in multiple exposure measures collected in parallel on 32 groups of workers. Random-effects models were applied under both compound symmetric and exponential correlation structures. Estimates of the within- and between-worker variances were used to contrast the attenuation bias in an exposure-response relationship that would be expected using an individual-based exposure assessment for different exposure measures on the basis of the intra-class correlation coefficient (ICC). ICC estimates ranged widely, indicative of a great deal of measurement error in some exposure measures while others contained very little. There was generally less attenuation in the biomarker data as compared to measurements obtained by personal sampling and, among biomarkers, for those with longer half-lives. The interval ICC estimates were often-times wide, suggesting a fair amount of imprecision in the point estimates. Ignoring serial correlation tended to over estimate the ICC values. Although personal sampling results were typically characterized by more intra-individual variability than inter-individual variability when compared to biological measurements, both types of data provided examples of exposure measures fraught with error. Our results also indicated substantial imprecision in the estimates of exposure measurement error, suggesting that greater emphasis needs to be given to studies that collect sufficient data to better characterize the attenuating effects of an error-prone exposure measure.

Variabilidad genética del crecimiento en progenies selectas de <i>Pinus radiata</i>.

Heredabilidad en sentido estricto (h2), correlación genética intra-clase (Γ2IC) y componentes de varianza para la altura, diámetro a la altura del pecho (DAP) y volumen del fuste, a los siete años de edad después de la plantación, fueron analizados en dos ensayos de progenie de Pinus radiata establecidos en el sur de Chile, en orden a examinar la variabilidad genética del crecimiento en una generación avanzada de la especie. Estos ensayos incluyeron árboles provenientes de propagación vegetativa y generativa. Las heredabilidades fueron estimadas para los árboles originados de semillas, y mostraron los siguientes valores: altura, 0.08-0.13; diámetro a la altura del pecho, 0-0.13; volumen del fuste, 0-0.09. Los valores de la correlación genética intra-clase, estimados para el grupo de árboles originados de propagación vegetativa, fueron moderados a bajos (Γ2IC: altura, 0.1-0.11; diámetro a la altura del pecho, 0.01-0.05; volumen del fuste, 0.03-0.06). No obstante la presencia de heredabilidades y correlaciones moderadas para algunas características y, la variación relativamente limitada entre grupos de árboles reflejan la fuerte selección realizada a las progenies. En general, los árboles provenientes de estacas presentaron una tendencia a la menor variación intra-genotípica, y evidenciaron magnitudes relativas de las varianzas debida al grupo (σ2G) mayores a las varianzas familiares (σ2F), en cuatro de seis caracteres evaluados, indicando una tendencia a la mayor homogeneidad en el crecimiento, característica común de la propagación vegetativa.

Estimates of Intraclass Correlation for Variables Related to Behavioral HIV/STD Prevention in a Predominantly African American and Hispanic Sample of Young Women

Studies designed to evaluate HIV and STD prevention interventions often involve random assignment of groups such as neighborhoods or communities to study conditions (e.g., to intervention or control). Investigators who design group-randomized trials (GRTs) must take the expected intraclass correlation coefficient (ICC) into account in sample size estimation to have adequate power; however, few published ICC estimates exist for outcome variables related to HIV and STD prevention. The Prevention Options for Women Equal Rights (POWER) study was a GRT designed to evaluate a campaign to increase awareness and use of condoms among young African American and Hispanic women. The authors used precampaign and postcampaign data from the POWER study to estimate ICCs (unadjusted and adjusted for covariates) for a variety of sexual behavior and other variables. To illustrate the impact of ICCs on power, the authors present sample-size calculations and demonstrate how ICCs of differing magnitude will affect estimates of required sample size.

Anterior chamber angle measurement with optical coherence tomography: Intraobserver and interobserver variability

To assess intraobserver and interobserver variability of anterior segment optical coherence tomography (AS-OCT) as an objective diagnostic tool to quantify the anterior chamber angle and opening width. University Eye Clinic, Lübeck, Germany. The anterior chamber angle and opening width were assessed in 18 eyes of 9 healthy volunteers by 2 observers. Intraobserver reproducibility was evaluated by calculating an intraclass correlation coefficient (ICC) in a mixed model. Each observer had a separate model using information from 5 scans. Interobserver variability was determined by Bland-Altman analysis. The ICC was calculated in a mixed model using a residual maximum likelihood method. The results of 3 and 5 repeated scans were evaluated to indicate the change to 1 measurement application. The mean anterior chamber angle measurement was 35.9 degrees +/- 5.7 (SD) for observer A and 36.2 +/- 5.7 degrees for observer B. The ICC was 0.94 and 0.91, respectively. The mean opening width was 315 +/- 62 microm for observer A and 317 +/- 60 microm for observer B. The ICC was 0.97 and 0.93, respectively. Interobserver comparisons showed a mean difference between anterior chamber angle measurements of -0.27 +/- 1.6 degrees, a limit of agreement (LOA) interval from -3.52 to 2.98 degrees, and an ICC estimate of 0.96. The mean difference in opening width measurements was 2.40 +/- 12.40 microm, the LOA from -27.20 to 22.40 microm, and the estimated ICC 0.96. Using 1 instead of the mean of 5 measurements, the LOA range increased by 3.46 degrees for the anterior chamber angle and 30.0 microm for the opening width. Anterior chamber angle and opening width measurements by OCT showed low intraobserver and interobserver variability, indicating OCT is a valuable technique for quantitative assessment that provides reproducible measurements and objective documentation by different examiners.

Estimates of intraclass correlation coefficient and design effect for surveys and cluster randomized trials on injection use in Pakistan and developing countries

To assess injection practices and to test interventions aimed at reducing unsafe injections in developing countries, cluster surveys and cluster randomized trials are needed. The design of cluster-based studies requires estimates of intraclass correlation coefficients that have to be obtained from previous studies. This study presents such estimates. Data were derived from a cross-sectional study of injection use and health seeking in Pakistan that used 34 clusters to select 1150 study subjects aged > or =3 months. We analysed variance to separate its components. Most of intraclass correlation coefficients were in the range of 0.01-0.05. For proportion of injections received during last 3 months, mean number of injections received and health seeking during the past 3 months the intraclass correlation coefficients were 0.02, 0.04 and 0.02, respectively. These estimates can be useful in designing cluster surveys and cluster randomized trials for injection safety in Pakistan and other developing countries.

Reliability and Validity Evidence of Physical Activity Measures in Individuals with Multiple Sclerosis

There is limited information regarding the reliability and validity of scores from measures of physical activity among individuals with multiple sclerosis (MS). This is important as gait and ambulatory abnormalities associated with loss of function and feeling in limbs and loss of balance and coordination may influence the accuracy of accelerometers and pedometers. Likewise, cognitive dysfunction in areas of memory encoding and retrieval may influence the accuracy of self-report surveys that involve a recall of physical activity. PURPOSE: This study examined the (1) reliability of scores from a pedometer and an accelerometer across a 7-day period and (2) the validity of scores from 2 self-report surveys and the pedometer and accelerometer during a 7-day period among individuals with MS. METHODS: Participants (N = 196; Mean age = 46.1 ± 9.8 years; 23 males and 173 females, Mean duration of MS = 9.0 ± 7.1 years) with an established definite diagnosis of MS completed 2 self-report measures of physical activity (GLTEQ and IPAQ) and then wore a pedometer (Yamax SW-200) and a single-axis accelerometer (ActiGraph) during the waking hours of a 7-day period. RESULTS: Repeated measures ANOVA indicated that there was not a statistically significant difference among days for the pedometer (p = .12) and the accelerometer (p = .15). Intra-class correlation (ICC) analyses indicated that the 7-days of monitoring yielded ICC estimates of .93 for both the pedometer and accelerometer, and a minimum of 3 days is necessary to achieve a reliability of .80 for both the pedometer and accelerometer. Pearson correlation analyses indicated that there was a strong correlation between scores from the pedometer and accelerometer (r = .82); a mo derate-to-strong correlation between scores from the GLTEQ and IPAQ (r = .59); and moderate-to-strong correlations between scores from the objective and self-report measures (range of r = .46−.53). CONCLUSION: This study provides evidence for scores from self-report and objective measures as valid and reliable measures of physical activity among those with MS.

Reliability Generalization of the Psychopathy Checklist Applied in Youthful Samples

This study examines the average reliability of Hare Psychopathy Checklists (PCLs) adapted for use in samples of youthful offenders (aged 12 to 21 years). Two forms of reli ability are examined: 18 α estimates of internal consistency and 18 intraclass correlation (two or more raters) estimates of interrater reliability. The results, an average internal consistency reliability estimate of .85 and an average interrater reliability estimate of .91, are both acceptable. The only study characteristic that accounts for a sizable amount of variance in both forms of reliability is the standard deviation of the PCL total score reported for a study. Total score variation and reliability estimate magnitudes are positively related.

Model mis‐specification and overestimation of the intraclass correlation coefficient in cluster randomized trials

Intraclass correlation coefficient (ICC) estimates must be provided when reporting the results of a cluster randomized trial. This study demonstrates that estimating this parameter with one-way ANOVA and an underlying mixed-effects statistical model leads to biased estimates. The bias depends on the effect size of the studied treatment.

Increasing the Degrees of Freedom in Existing Group Randomized Trials

This study describes a method for incorporating external estimates of intraclass correlation to improve the precision for the analysis of an existing group-randomized trial. The authors use a random-effects meta-analytic approach to pool the information across studies, which takes into account any interstudy heterogeneity that may exist. This approach can be used in several different situations to estimate the degrees of freedom available for an adjusted test of the intervention effect in a study where the challenges of group-randomized trials were not fully considered when the study was planned. The authors discuss the limitations of this approach and the circumstances in which it is likely to be helpful.

Increasing the Degrees of Freedom in Future Group Randomized Trials

This article builds on the previous article by Blitstein et al. (2005), which showed how external estimates of intraclass correlation can be used to improve the precision for the analysis of an existing group randomized trial. The authors extend that work to sample size estimation and power analysis for future group-randomized trials. Often this approach will allow a smaller study than would otherwise be possible without sacrificing statistical power. Such studies are needed, for example, as pilot studies to help plan for a full-scale efficacy trial, as replication studies, or in situations in which resource constraints prohibit a larger trial. The authors discuss the circumstances under which this strategy will be most helpful and the risks associated with conducting smaller studies.

Interval estimation and hypothesis testing of intraclass correlation coefficients: the generalized variable approach

In this paper, we propose a novel approach using the concept of generalized variable (GV) for the confidence interval estimation of the difference of two intraclass correlation coefficients under unequal family sizes. This approach can also easily provide P-values for hypothesis testing. Simulation results show that the GV approach can provide confidence intervals with good coverage properties and perform hypothesis testing with satisfactory type-I error control. Furthermore, the confidence intervals and P-values by GV approach can be easily obtained by simulation. Therefore the GV approach is a suitable candidate for making inference concerning two intraclass correlation coefficients.

Intra-class correlation estimates for assessment of vitamin A intake in children.

In many community-based surveys, multi-level sampling is inherent in the design. In the design of these studies, especially to calculate the appropriate sample size, investigators need good estimates of intra-class correlation coefficient (ICC), along with the cluster size, to adjust for variation inflation due to clustering at each level. The present study used data on the assessment of clinical vitamin A deficiency and intake of vitamin A-rich food in children in a district in India. For the survey, 16 households were sampled from 200 villages nested within eight randomly-selected blocks of the district. ICCs and components of variances were estimated from a three-level hierarchical random effects analysis of variance model. Estimates of ICCs and variance components were obtained at village and block levels. Between-cluster variation was evident at each level of clustering. In these estimates, ICCs were inversely related to cluster size, but the design effect could be substantial for large clusters. At the block level, most ICC estimates were below 0.07. At the village level, many ICC estimates ranged from 0.014 to 0.45. These estimates may provide useful information for the design of epidemiological studies in which the sampled (or allocated) units range in size from households to large administrative zones.

On confidence intervals of a common intraclass correlation coefficient

This study presents a novel approach for the confidence interval estimation of a common intraclass correlation coefficient derived from several samples under unequal family sizes. This approach is developed using the concept of generalized pivots. Comparisons are made with a large sample procedure on the coverage probabilities.

About the data designs for estimation of genetic parameters in animal breeding studies

The simple random one-way linear model and ANOVA estimators of random effects, variance components and intraclass correlation are studied. A related sire model in animal genetic studies is considered. The study is focused on the effect of data design on the accuracy of estimates. The patterns of sampling variances of estimates depending on the genetic determination of the trait and data design are examined and optimal designs for different genetic parameters are compared in the balanced case.

Assessing Intervention Effects in a School-Based Nutrition Intervention Trial: Which Analytic Model Is Most Powerful?

This article compares four mixed-model analyses valid for group-randomized trials (GRTs) involving a nested cohort design with a single pretest and posttest. This study makes estimates of intraclass correlations (ICCs) available to investigators planning GRTs addressing dietary outcomes. It also provides formulae demonstrating the potential benefits to the standard error of the intervention effect (sigma(delta)) from adjustments for both fixed and time-varying covariates and correlations over time. These estimates will allow other researchers using these variables to plan their studies by estimating a priori detectable differences and sample size requirements for any of the four analytic options. These methods are demonstrated using data from the Teens Eating for Energy and Nutrition at School study. Mixed-model analyses of covariance proved to be the most powerful analysis in that data set. The formulae may be applied to any dependent variable in any GRT given corresponding information for those variables on the parameters that define the formulae.

Patterns of intra-cluster correlation from primary care research to inform study design and analysis

ObjectiveTo provide information concerning the magnitude of the intraclass correlation coefficient (ICC) for cluster-based studies set in primary care. Study design and settingReanalysis of data from 31 cluster-based studies in primary care to estimate intraclass correlation coefficients from random effects models using maximum likelihood estimation. ResultsICCs were estimated for 1,039 variables. The median ICC was 0.010 (interquartile range [IQR] 0 to 0.032, range 0 to 0.840). After adjusting for individual- and cluster-level characteristics, the median ICC was 0.005 (IQR 0 to 0.021). A given measure showed widely varying ICC estimates in different datasets. In six datasets, the ICCs for SF-36 physical functioning scale ranged from 0.001 to 0.055 and for SF-36 general health from 0 to 0.072. In four datasets, the ICC for systolic blood pressure ranged from 0 to 0.052 and for diastolic blood pressure from 0 to 0.108. ConclusionThe precise magnitude of between-cluster variation for a given measure can rarely be estimated in advance. Studies should be designed with reference to the overall distribution of ICCs and with attention to features that increase efficiency.

A new approach for interval estimation and hypothesis testing of a certain intraclass correlation coefficient: the generalized variable method.

We consider the problems of interval estimation and hypothesis testing for the intraclass correlation coefficient in an interrater reliability study when both raters and subjects are assumed to be randomly selected from populations of raters and subjects, respectively. We propose a novel approach for the confidence interval estimation and hypothesis testing using the concepts of generalized confidence interval (GCI) and generalized P-values. A simulation study is conducted to investigate the coverage probabilities of the GCI approach relative to the modified large sample (MLS) approach. Both methods tend to provide somewhat conservative coverage. Relative to the MLS approach, the GCI approach is closer to the correct (nominal) coverage for a two-sided interval, but farther to the correct coverage for a one-sided lower interval. Unlike the MLS approach, the GCI approach can also easily provide P-values. The fact that the GCI approach is suitable for confidence interval estimation and obtaining P-values makes the GCI approach a suitable candidate for making inference about interrater reliability.