Applying modern measurement approaches to constructs relevant to evidence-based practice among Canadian physical and occupational therapists

Abstract

BackgroundEvidence-based practice (EBP) is a complex process. To quantify it, one has to also consider individual and contextual factors using multiple measures. Modern measurement approaches are available to optimize the measurement of complex constructs. This study aimed to develop a robust measurement approach for constructs around EBP including practice, individual (e.g. knowledge, attitudes, confidence, behaviours), and contextual factors (e.g. resources).MethodsOne hundred eighty-one items arising from 5 validated EBP measures were subjected to an item analysis. Nominal group technique was used to arrive at a consensus about the content relevance of each item. Baseline questionnaire responses from a longitudinal study of the evolution of EBP in 128 new graduates of Canadian physical and occupational therapy programmes were analysed. Principles of Rasch Measurement Theory were applied to identify challenges with threshold ordering, item and person fit to the Rasch model, unidimensionality, local independence, and differential item functioning (DIF).ResultsThe nominal group technique identified 70/181 items, and modified Delphi approach identified 68 items that fit a formative model (2 related EBP domains: self-use of EBP (9 items) and EBP activities (7 items)) or a reflective model (4 related EBP domains: attitudes towards EBP (17 items), self-efficacy (9 items), knowledge (11 items) and resources (15 items)). Rasch analysis provided a single score for reflective construct. Among attitudes items, 65% (11/17) fit the Rasch model, item difficulties ranged from − 7.51 to logits (least difficult) to + 5.04 logits (most difficult), and person separation index (PSI) = 0.63. Among self-efficacy items, 89% (8/9) fit the Rasch model, item difficulties ranged from − 3.70 to + 4.91, and PSI = 0.80. Among knowledge items, 82% (9/11) fit the Rasch model, item difficulties ranged from − 7.85 to 4.50, and PSI = 0.81. Among resources items, 87% (13/15) fit the Rasch model, item difficulties ranged from − 3.38 to 2.86, and PSI = 0.86. DIF occurred in 2 constructs: attitudes (1 by profession and 2 by language) and knowledge (1 by language and 2 by profession) arising from poor wording in the original version leading to poor translation.ConclusionsRasch Measurement Theory was applied to develop a valid and reliable measure of EBP. Further modifications to the items can be done for subsequent waves of the survey.