Abstract
BackgroundValid and reliable instruments are required to measure the effect of educational interventions to improve evidence-based practice (EBP) knowledge and skills in occupational therapy. The aims of this paper are to: 1) describe amendments to the Adapted Fresno Test of Competence in EBP (AFT), and 2) report the psychometric properties of the modified instrument when used with South African occupational therapists.MethodsThe clinical utility of the AFT was evaluated for use with South African occupational therapists and modifications made. The modified AFT was used in two studies to assess its reliability and validity. In Study 1 a convenience sample of 26 occupational therapists in private practice or government-funded health facilities in a South African province were recruited to complete the modified AFT on two occasions 1 week apart. Completed questionnaires were scored independently by two raters. Inter-rater, test-retest reliability and internal consistency were determined. Study 2 was a pragmatic randomised controlled trial involving occupational therapists in four Western Cape Department of Health district municipalities (n = 58). Therapists were randomised in matched pairs to one of two educational interventions (interactive or didactic), and completed the modified AFT at baseline and 12 weeks after the intervention. An intention-to-treat analysis was performed. Data were not normally distributed, thus non-parametric statistics were used.ResultsIn Study 1, 21 of 26 participants completed the questionnaire twice. Test-retest (ICC = 0.95, 95 % CI = 0.88–0.98) and inter-rater reliability (Time 1: ICC = 0.995, 95 % CI = 0.99–0.998; Time 2: ICC = 0.99, 95 % CI = 0.97–0.995) were excellent for total scores. Internal consistency based on time 1 scores was satisfactory (α = 0.70). In Study 2, 28 participants received an interactive educational intervention and completed the modified AFT at baseline and 12 weeks later. Median total SAFT scores increased significantly from baseline to 12-weeks (Z = −4.078, p < 0.001) with a moderate effect size (r = 0.55).ConclusionThe modified AFT has demonstrated validity for detecting differences in EBP knowledge between two groups. It also has excellent test-retest and inter-rater reliability. The instrument is recommended for contexts where EBP is an emerging approach and time is at a premium.Trial registrationPan African Controlled Trials Register PACTR201201000346141. Registered 31 January 2012.Clinical Trials NCT01512823. Registered 1 February 2012.South African National Clinical Trial Register DOH2710093067. Registered 27 October 2009.
Highlights
Valid and reliable instruments are required to measure the effect of educational interventions to improve evidence-based practice (EBP) knowledge and skills in occupational therapy
Evaluation of its applicability to occupational therapists in South Africa was based on anticipated EBP knowledge and skill levels as identified in the aforementioned South African occupational therapy survey [4]
The excellent scores for test-retest reliability indicate that the Shortened AFT (SAFT) is stable in the absence of change and able to describe EBP knowledge and skills [16, 19, 20, 27]
Summary
Valid and reliable instruments are required to measure the effect of educational interventions to improve evidence-based practice (EBP) knowledge and skills in occupational therapy. Educational interventions are commonly used to equip health professionals with the tools for EBP, but valid and reliable instruments are required to measure the effects of such interventions in improving knowledge and skills [1, 2]. Few had attended EBP training (25 %) and relied strongly on their clinical experience rather than research literature. Their limited success in finding and applying evidence may have been due to reported barriers such as time, knowledge, and convenient access to evidence sources. The survey findings suggested a need for further education, and a randomised controlled trial (RCT) was planned to test the effects of two educational interventions in this group and context
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have