Abstract
The use of simulated patients (SPs) in pharmacy practice research has become an established method to observe practice. The reliability of data reported using this method in comparison to pharmacy staff self-reported behaviour has yet to be ascertained. To compare the inter-rater agreement of pharmacy staff and SP-reported data to researcher-reported data from audio recordings of SP encounters. A dataset of 352 audio-recorded SP encounters was generated in March-October 2015 by 61 undergraduate pharmacy students completing SP visits to 36 community pharmacies in Sydney, Australia. Post-visit scores were recorded on data collection forms by SPs. Staff completed self-assessments on identical forms immediately after the encounter. Two-hundred-and-seventy visits were randomly selected as the sample for this study, where the researcher independently scored encounters via audio recordings. Inter-rater agreement was calculated through intra-class correlation (ICC) and weighted kappa analyses. Analysis of staff scores returned ICC values of 0.48 (95% CI:0.38-0.56; p<0.001) for information gathering and 0.63 (95% CI:0.55-0.70; p<0.001) for total score. Weighted kappa for information rating was 0.30 (95% CI:0.21-0.38; p<0.001) and 0.43 (95% CI:0.34-0.51; p<0.001) for overall outcome. ICC values for SPs were 0.91 (95% CI:0.88-0.93; p<0.001) and 0.90 (95% CI:0.87-0.92; p<0.001) for information gathering and total scores respectively. Weighted kappa values were 0.44 (95% CI:0.37-0.52; p<0.001) for information rating and 0.63 (95% CI:0.55-0.70; p<0.001) for overall outcome. Pharmacy staff self-reported their behaviour with a poor degree of reliability. Conversely, SPs had a high level of agreement with the researcher scoring from audio recordings. Disagreement for both groups of raters was most apparent in rating the information provided and overall appropriateness of outcome. Future research should investigate this discrepancy between staff-reported behaviour and actual behaviour and consider the implications of this discrepancy in the interpretation of self-reported data.
Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callMore From: Research in social & administrative pharmacy : RSAP
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
