Abstract
The intraclass correlation coefficient (ICC) is a classical index of measurement reliability. With the advent of new and complex types of data for which the ICC is not defined, there is a need for new ways to assess reliability. To meet this need, we propose a new distance‐based ICC (dbICC), defined in terms of arbitrary distances among observations. We introduce a bias correction to improve the coverage of bootstrap confidence intervals for the dbICC, and demonstrate its efficacy via simulation. We illustrate the proposed method by analyzing the test‐retest reliability of brain connectivity matrices derived from a set of repeated functional magnetic resonance imaging scans. The Spearman‐Brown formula, which shows how more intensive measurement increases reliability, is extended to encompass the dbICC.
Highlights
With the increasing availability of new and complex forms of data, there is a corresponding need for new ways to assess measurement reliability
This article aims to help meet this need by reformulating the intraclass correlation coefficient (ICC), a standard index of reliability, in terms of distances between observations
This possibility, which is quite consistent with the real-data results in Figure 6, suggests that while longer functional magnetic resonance imaging (fMRI) scans might make correlation matrices more reliable as measures of functional connectivity, the improvement would likely be less dramatic than simulations might lead us to expect
Summary
With the increasing availability of new and complex forms of data, there is a corresponding need for new ways to assess measurement reliability. Reliability measures for more complex settings include replacing model (1) with the generalizability theory model of Cranford et al (2006), as well as generalizations of (2) to multivariate data (Alonso et al, 2010), including high-dimensional data (Shou et al, 2013) All of these extensions assume a model that is more complex than (1), but still of an additive (signal plus noise) form. 2. Distance-based reliability measurement A novel reliability index applicable to general data objects can be defined by re-deriving the ICC (2) in terms of squared distances among observations. The dbICC is an extension of these measures to more general distances and data types
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
