Intraclass Correlation Coefficient Estimates Research Articles

Anterior chamber angle measurement with optical coherence tomography: Intraobserver and interobserver variability

To assess intraobserver and interobserver variability of anterior segment optical coherence tomography (AS-OCT) as an objective diagnostic tool to quantify the anterior chamber angle and opening width. University Eye Clinic, Lübeck, Germany. The anterior chamber angle and opening width were assessed in 18 eyes of 9 healthy volunteers by 2 observers. Intraobserver reproducibility was evaluated by calculating an intraclass correlation coefficient (ICC) in a mixed model. Each observer had a separate model using information from 5 scans. Interobserver variability was determined by Bland-Altman analysis. The ICC was calculated in a mixed model using a residual maximum likelihood method. The results of 3 and 5 repeated scans were evaluated to indicate the change to 1 measurement application. The mean anterior chamber angle measurement was 35.9 degrees +/- 5.7 (SD) for observer A and 36.2 +/- 5.7 degrees for observer B. The ICC was 0.94 and 0.91, respectively. The mean opening width was 315 +/- 62 microm for observer A and 317 +/- 60 microm for observer B. The ICC was 0.97 and 0.93, respectively. Interobserver comparisons showed a mean difference between anterior chamber angle measurements of -0.27 +/- 1.6 degrees, a limit of agreement (LOA) interval from -3.52 to 2.98 degrees, and an ICC estimate of 0.96. The mean difference in opening width measurements was 2.40 +/- 12.40 microm, the LOA from -27.20 to 22.40 microm, and the estimated ICC 0.96. Using 1 instead of the mean of 5 measurements, the LOA range increased by 3.46 degrees for the anterior chamber angle and 30.0 microm for the opening width. Anterior chamber angle and opening width measurements by OCT showed low intraobserver and interobserver variability, indicating OCT is a valuable technique for quantitative assessment that provides reproducible measurements and objective documentation by different examiners.

Estimates of intraclass correlation coefficient and design effect for surveys and cluster randomized trials on injection use in Pakistan and developing countries

To assess injection practices and to test interventions aimed at reducing unsafe injections in developing countries, cluster surveys and cluster randomized trials are needed. The design of cluster-based studies requires estimates of intraclass correlation coefficients that have to be obtained from previous studies. This study presents such estimates. Data were derived from a cross-sectional study of injection use and health seeking in Pakistan that used 34 clusters to select 1150 study subjects aged > or =3 months. We analysed variance to separate its components. Most of intraclass correlation coefficients were in the range of 0.01-0.05. For proportion of injections received during last 3 months, mean number of injections received and health seeking during the past 3 months the intraclass correlation coefficients were 0.02, 0.04 and 0.02, respectively. These estimates can be useful in designing cluster surveys and cluster randomized trials for injection safety in Pakistan and other developing countries.

Reliability and Validity Evidence of Physical Activity Measures in Individuals with Multiple Sclerosis

There is limited information regarding the reliability and validity of scores from measures of physical activity among individuals with multiple sclerosis (MS). This is important as gait and ambulatory abnormalities associated with loss of function and feeling in limbs and loss of balance and coordination may influence the accuracy of accelerometers and pedometers. Likewise, cognitive dysfunction in areas of memory encoding and retrieval may influence the accuracy of self-report surveys that involve a recall of physical activity. PURPOSE: This study examined the (1) reliability of scores from a pedometer and an accelerometer across a 7-day period and (2) the validity of scores from 2 self-report surveys and the pedometer and accelerometer during a 7-day period among individuals with MS. METHODS: Participants (N = 196; Mean age = 46.1 ± 9.8 years; 23 males and 173 females, Mean duration of MS = 9.0 ± 7.1 years) with an established definite diagnosis of MS completed 2 self-report measures of physical activity (GLTEQ and IPAQ) and then wore a pedometer (Yamax SW-200) and a single-axis accelerometer (ActiGraph) during the waking hours of a 7-day period. RESULTS: Repeated measures ANOVA indicated that there was not a statistically significant difference among days for the pedometer (p = .12) and the accelerometer (p = .15). Intra-class correlation (ICC) analyses indicated that the 7-days of monitoring yielded ICC estimates of .93 for both the pedometer and accelerometer, and a minimum of 3 days is necessary to achieve a reliability of .80 for both the pedometer and accelerometer. Pearson correlation analyses indicated that there was a strong correlation between scores from the pedometer and accelerometer (r = .82); a mo derate-to-strong correlation between scores from the GLTEQ and IPAQ (r = .59); and moderate-to-strong correlations between scores from the objective and self-report measures (range of r = .46−.53). CONCLUSION: This study provides evidence for scores from self-report and objective measures as valid and reliable measures of physical activity among those with MS.

Reliability Generalization of the Psychopathy Checklist Applied in Youthful Samples

This study examines the average reliability of Hare Psychopathy Checklists (PCLs) adapted for use in samples of youthful offenders (aged 12 to 21 years). Two forms of reli ability are examined: 18 α estimates of internal consistency and 18 intraclass correlation (two or more raters) estimates of interrater reliability. The results, an average internal consistency reliability estimate of .85 and an average interrater reliability estimate of .91, are both acceptable. The only study characteristic that accounts for a sizable amount of variance in both forms of reliability is the standard deviation of the PCL total score reported for a study. Total score variation and reliability estimate magnitudes are positively related.

Model mis‐specification and overestimation of the intraclass correlation coefficient in cluster randomized trials

Intraclass correlation coefficient (ICC) estimates must be provided when reporting the results of a cluster randomized trial. This study demonstrates that estimating this parameter with one-way ANOVA and an underlying mixed-effects statistical model leads to biased estimates. The bias depends on the effect size of the studied treatment.

Increasing the Degrees of Freedom in Existing Group Randomized Trials

This study describes a method for incorporating external estimates of intraclass correlation to improve the precision for the analysis of an existing group-randomized trial. The authors use a random-effects meta-analytic approach to pool the information across studies, which takes into account any interstudy heterogeneity that may exist. This approach can be used in several different situations to estimate the degrees of freedom available for an adjusted test of the intervention effect in a study where the challenges of group-randomized trials were not fully considered when the study was planned. The authors discuss the limitations of this approach and the circumstances in which it is likely to be helpful.

Increasing the Degrees of Freedom in Future Group Randomized Trials

This article builds on the previous article by Blitstein et al. (2005), which showed how external estimates of intraclass correlation can be used to improve the precision for the analysis of an existing group randomized trial. The authors extend that work to sample size estimation and power analysis for future group-randomized trials. Often this approach will allow a smaller study than would otherwise be possible without sacrificing statistical power. Such studies are needed, for example, as pilot studies to help plan for a full-scale efficacy trial, as replication studies, or in situations in which resource constraints prohibit a larger trial. The authors discuss the circumstances under which this strategy will be most helpful and the risks associated with conducting smaller studies.

Interval estimation and hypothesis testing of intraclass correlation coefficients: the generalized variable approach

In this paper, we propose a novel approach using the concept of generalized variable (GV) for the confidence interval estimation of the difference of two intraclass correlation coefficients under unequal family sizes. This approach can also easily provide P-values for hypothesis testing. Simulation results show that the GV approach can provide confidence intervals with good coverage properties and perform hypothesis testing with satisfactory type-I error control. Furthermore, the confidence intervals and P-values by GV approach can be easily obtained by simulation. Therefore the GV approach is a suitable candidate for making inference concerning two intraclass correlation coefficients.

Intra-class correlation estimates for assessment of vitamin A intake in children.

In many community-based surveys, multi-level sampling is inherent in the design. In the design of these studies, especially to calculate the appropriate sample size, investigators need good estimates of intra-class correlation coefficient (ICC), along with the cluster size, to adjust for variation inflation due to clustering at each level. The present study used data on the assessment of clinical vitamin A deficiency and intake of vitamin A-rich food in children in a district in India. For the survey, 16 households were sampled from 200 villages nested within eight randomly-selected blocks of the district. ICCs and components of variances were estimated from a three-level hierarchical random effects analysis of variance model. Estimates of ICCs and variance components were obtained at village and block levels. Between-cluster variation was evident at each level of clustering. In these estimates, ICCs were inversely related to cluster size, but the design effect could be substantial for large clusters. At the block level, most ICC estimates were below 0.07. At the village level, many ICC estimates ranged from 0.014 to 0.45. These estimates may provide useful information for the design of epidemiological studies in which the sampled (or allocated) units range in size from households to large administrative zones.

On confidence intervals of a common intraclass correlation coefficient

This study presents a novel approach for the confidence interval estimation of a common intraclass correlation coefficient derived from several samples under unequal family sizes. This approach is developed using the concept of generalized pivots. Comparisons are made with a large sample procedure on the coverage probabilities.

About the data designs for estimation of genetic parameters in animal breeding studies

The simple random one-way linear model and ANOVA estimators of random effects, variance components and intraclass correlation are studied. A related sire model in animal genetic studies is considered. The study is focused on the effect of data design on the accuracy of estimates. The patterns of sampling variances of estimates depending on the genetic determination of the trait and data design are examined and optimal designs for different genetic parameters are compared in the balanced case.

Assessing Intervention Effects in a School-Based Nutrition Intervention Trial: Which Analytic Model Is Most Powerful?

This article compares four mixed-model analyses valid for group-randomized trials (GRTs) involving a nested cohort design with a single pretest and posttest. This study makes estimates of intraclass correlations (ICCs) available to investigators planning GRTs addressing dietary outcomes. It also provides formulae demonstrating the potential benefits to the standard error of the intervention effect (sigma(delta)) from adjustments for both fixed and time-varying covariates and correlations over time. These estimates will allow other researchers using these variables to plan their studies by estimating a priori detectable differences and sample size requirements for any of the four analytic options. These methods are demonstrated using data from the Teens Eating for Energy and Nutrition at School study. Mixed-model analyses of covariance proved to be the most powerful analysis in that data set. The formulae may be applied to any dependent variable in any GRT given corresponding information for those variables on the parameters that define the formulae.

Patterns of intra-cluster correlation from primary care research to inform study design and analysis

ObjectiveTo provide information concerning the magnitude of the intraclass correlation coefficient (ICC) for cluster-based studies set in primary care. Study design and settingReanalysis of data from 31 cluster-based studies in primary care to estimate intraclass correlation coefficients from random effects models using maximum likelihood estimation. ResultsICCs were estimated for 1,039 variables. The median ICC was 0.010 (interquartile range [IQR] 0 to 0.032, range 0 to 0.840). After adjusting for individual- and cluster-level characteristics, the median ICC was 0.005 (IQR 0 to 0.021). A given measure showed widely varying ICC estimates in different datasets. In six datasets, the ICCs for SF-36 physical functioning scale ranged from 0.001 to 0.055 and for SF-36 general health from 0 to 0.072. In four datasets, the ICC for systolic blood pressure ranged from 0 to 0.052 and for diastolic blood pressure from 0 to 0.108. ConclusionThe precise magnitude of between-cluster variation for a given measure can rarely be estimated in advance. Studies should be designed with reference to the overall distribution of ICCs and with attention to features that increase efficiency.

A new approach for interval estimation and hypothesis testing of a certain intraclass correlation coefficient: the generalized variable method.

We consider the problems of interval estimation and hypothesis testing for the intraclass correlation coefficient in an interrater reliability study when both raters and subjects are assumed to be randomly selected from populations of raters and subjects, respectively. We propose a novel approach for the confidence interval estimation and hypothesis testing using the concepts of generalized confidence interval (GCI) and generalized P-values. A simulation study is conducted to investigate the coverage probabilities of the GCI approach relative to the modified large sample (MLS) approach. Both methods tend to provide somewhat conservative coverage. Relative to the MLS approach, the GCI approach is closer to the correct (nominal) coverage for a two-sided interval, but farther to the correct coverage for a one-sided lower interval. Unlike the MLS approach, the GCI approach can also easily provide P-values. The fact that the GCI approach is suitable for confidence interval estimation and obtaining P-values makes the GCI approach a suitable candidate for making inference about interrater reliability.

Assessing the most powerful analysis method for school-based intervention studies with alcohol, tobacco, and other drug outcomes

This article compares four mixed-model analyses valid for group-randomized trials (GRTs) involving a nested cohort design with a single pretest and a single posttest, the most common design used in GRTs. This study makes estimates of intraclass correlations (ICCs) available to investigators planning GRTs with alcohol, tobacco, and other drug measures as the outcomes of interest. It also provides formulae demonstrating the potential benefits to the standard error of the intervention effect of both adjustments for fixed and time-varying covariates, as well as correlations over time. These estimates will allow other researchers using these variables to plan their studies by performing a priori power analyses for any of four common analytic options.

Design and analysis of group-randomized trials: a review of recent methodological developments.

We review recent developments in the design and analysis of group-randomized trials (GRTs). Regarding design, we summarize developments in estimates of intraclass correlation, power analysis, matched designs, designs involving one group per condition, and designs in which individuals are randomized to receive treatments in groups. Regarding analysis, we summarize developments in marginal and conditional models, the sandwich estimator, model-based estimators, binary data, survival analysis, randomization tests, survey methods, latent variable methods and nonlinear mixed models, time series methods, global tests for multiple endpoints, mediation effects, missing data, trial reporting, and software. We encourage investigators who conduct GRTs to become familiar with these developments and to collaborate with methodologists who can strengthen the design and analysis of their trials.

Methods to reduce the impact of intraclass correlation in group-randomized trials.

This study reports intraclass correlation (ICC) for dependent variables used in group-randomized trials (GRTs). The authors also document the effect of two methods suggested to reduce the impact of ICC in GRTs; these two methods are modeling time and regression adjustment for covariates. They coded and analyzed 1,188 ICC estimates from 17 published, in press, and unpublished articles representing 21 studies. Findings confirm that both methods can improve the efficiency of analyses shown to be valid across conditions common in GRTs. Investigators planning GRTs should obtain ICC estimates matched to their planned analysis so that they can size their studies properly.

Heritability estimates in dwarf population of sunflower - Helianthus annuus L.

A comparison of heritability estimates from regression and intra-class correlation analyses in full-sib and half-sib families showed that heritability estimates from regression analysis are lower than intra-class correlation estimates in both types of families. The heritability estimates from half-sib families are higher than from full-sibs for all the traits except number of leaves per plant and 100-seed weight. A comparison of heritability estimates from offspring- parents, bOP1, bOP2 and bOMP showed that heritability estimates from regression of offspring-mid parent are higher than offspring-one parents. Further, leaf length showed the lowest heritability in regression analysis. The overall results showed that plant height, 100-seed weight and number of leaves per plant showed higher heritability.

Intraclass correlation among measures related to cigarette use by adolescents: Estimates from an urban and largely African American cohort

This paper presents the first estimates of school-level intraclass correlation (ICC) for smoking-related variables from an urban and largely African American population. Seventh graders ( n=6967) from 39 middle schools in Memphis, TN, were measured at baseline in 1994 and annually through 1997. Mixed model regression methods were used to estimate variance components for school and residual error. School-level ICCs were large enough, if ignored, to substantially inflate the Type I error rate in an analysis of treatment effects. We show how those correlations can be reduced using regression adjustments and used to determine sample size for future school-based smoking prevention studies.

Intraclass correlation for measures from a middle school nutrition intervention study: estimates, correlates, and applications.

This article presents the first estimates of school-level intraclass correlation for dietary measures based on data from the Teens Eating for Energy and Nutrition at School study. This study involves 3,878 seventh graders from 16 middle schools from Minneapolis-St. Paul, Minnesota. The sample was 66.8% White, 11.2% Black, and 7.0% Asian; 48.8% of the sample was female. Typical fruit and vegetable intake was assessed with a modified version of the Behavior Risk Factor Surveillance System questionnaire. Twenty-four-hour dietary recalls were conducted by nutritionists using the Minnesota Nutrition Data System. Mixed-model regression methods were used to estimate variance components for school and residual error, both before and after adjustment for demographic factors. School-level intraclass correlations were large enough, if ignored, to substantially inflate the Type I error rate in an analysis of treatment effects. The authors show how to use the estimates to determine sample size requirements for future studies.