Measuring clinical trial set-up complexity: development and content validation of a pharmacy scoring tool to support workload planning

Abstract

Abstract Introduction Clinical trials are essential for advancing medical knowledge and patient care. Pharmacy plays a vital role in research delivery, safeguarding participants, healthcare professionals and Trusts by ensuring Investigational Medicinal Products (IMPs) are procured, handled, stored, dispensed and used safely. Data have emerged indicating that clinical trial protocols are increasingly more complex for site personnel to execute. Whilst various tools have been developed to support workload planning and costing of clinical trials for clinicians and nursing personnel, the equivalent tool for pharmacy activities is not available.1,2 Aim This study aims to locally develop and validate a pharmacy scoring tool to assess clinical trial complexity at the set-up stage. Objectives are to develop pharmacy-scoring tool to quantify clinical trials execution complexity with respect to pharmacy activities and determine Content Validity Index (CVI) and Inter-Rater Agreement (IRA) of the scoring tool through an internal vetting process. Methods A three-stage process was used to develop and validate the scoring tool. Two researchers contextualised the pharmacy–specific items, organised the content and constructed the tool based on suitable elements for inclusion through a targeted literature search and outputs from an expert panel meeting. Four experts independently rated each items on its relevance, clarity and alignment with clinical trial complexity considerations at the set-up stage using a 4-point Likert scale. Following data analysis, researchers made a judgement on whether to retain, modify, omit or add new items to the tool. Where the desired level of content validity had not been achieved for an item, revisions were made following a group discussion and the tool was re-distributed to the same expert panel for a second round of assessment. Item level-CVI (I-CVI) and tool level-CVI (average-CVI) was calculated for each round. Four experts then utilised the revised tool to retrospectively review 20 random studies to evaluate clinical trial complexity scores independently. IRA among assessors was measured for each individual pharmacy-specific item by dividing the responses into dichotomous ratings (one or two vs. three or four). The study was deemed a service evaluation and approved by the Trust Pharmacy Research and Audit Group therefore ethical submission was waived. Results Fourteen pharmacy-specific items were identified for inclusion in the tool with a possible score of 0-3 points per item. The highest possible complexity score when adding up all items is 42 points with a scale of low 0-14, moderate >14-28, and high >28-42. Following two rounds of content validity assessment where clarity revisions were made to four items, 14 out of 14 scored I-CVI of 1 with overall tool level average-CVI score of 1. The IRA shows 100% agreement by item and tool. Discussion/Conclusion This clinical trial’s complexity-scoring tool is the first of its kind tailored for pharmacy set-up in the UK. Overall, the results from this study support validity and reliability (content and construct) of this locally developed tool. The formal agreement on the use of this tool with the Trust Research Lead has better facilitated workload and capacity planning as well as ensuing costings for Pharmacy clinical trial activities.