Validation of a risk-assessment scale and a risk-adapted monitoring plan for academic clinical research studies — The Pre-Optimon study

Abstract

Context Good Clinical Practice regulates monitoring activities in clinical research. Due to question and design diversity, and limited resources, on-site monitoring is often less intensive in the academic context, and variable. Standardization is needed, and relies on definition and validation of tools accounting for risk. Objective To define, and validate tools, to implement a risk-based monitoring strategy for academic clinical research. Methods Working groups of experienced professionals searched the literature, and built a consensus risk-assessment scale (RAS), and a risk-adapted monitoring plan (RAMP). We allocated 200 protocols to 49 assessors. We assessed the RAS relevance vs. a visual analogue scale (VAS), and its reproducibility through Kraemer's kappa, and intraclass correlation coefficient (ICC) from a random proportional odds model. We identified sources of disagreement through a logistic regression. We described assessors' difficulties during assessment. We applied the RAMP to 10 protocols per risk level, and rated its feasibility (0 = easy to 4 = impossible). Results RAS and RAMP were defined in 4 levels. RAS relevance was good: RAS-risk levels were evenly distributed on VAS-risk (0.6, 2.6, 5.6, and 7.9). Reproducibility was moderate to good: kappa = 0.48, ICC = 0.70. Major disagreements (36%) arose from decision-makers, rather than hands-on managers. Most difficulties occurred in ill-written protocols (17%). RAMP was easily feasible for most protocols (mean score: 0.2 to 0.9). We proposed a standard synopsis for evaluation purpose. Conclusion We defined, and validated risk-based tools. This risk-adapted strategy will be compared to an intensive one in a randomized trial, Optimon, to define a standard of practice for academic clinical research.