Assessing agreement with relative area under the coverage probability curve.

Abstract

There has been substantial statistical literature in the last several decades on assessing agreement, and coverage probability approach was selected as a preferred index for assessing and improving measurement agreement in a core laboratory setting. With this approach, a satisfactory agreement is based on pre-specified high satisfactory coverage probability (e.g., 95%), given one pre-specified acceptable difference. In practice, we may want to have quality control on more than one pre-specified differences, or we may simply want to summarize the agreement based on differences up to a maximum acceptable difference. We propose to assess agreement via the coverage probability curve that provides a full spectrum of measurement error at various differences/disagreement. Relative area under the coverage probability curve is proposed for the summary of overall agreement, and this new summary index can be used for comparison of different intra-methods or inter-methods/labs/observers' agreement. Simulation studies and a blood pressure example are used for illustration of the methodology. Copyright © 2016 John Wiley & Sons, Ltd.