Abstract
BackgroundThe Paediatric Residency Program (PRP) of Padua, Italy, developed a set of questionnaires to assess the quality of the training provided by each faculty member, the quality of the professional experience the residents experienced during the various rotations and the functioning of the Resident Affair Committee (RAC), named respectively: “Tutor Assessment Questionnaire” (TAQ), “Rotation Assessment Questionnaire” (RAQ), and RAC Assessment Questionnaire”. The process that brought to their validation are herein presented.MethodBetween July 2012 and July 2013, 51 residents evaluated 26 tutors through the TAQ, and 25 rotations through the RAQ. Forty-eight residents filled the RAC Assessment Questionnaire. The three questionnaires were validated through a many-facet Rasch measurement analysis.ResultsIn their final form, the questionnaires produced measures that were valid, reliable, unidimensional, and free from gender biases. TAQ and RAQ distinguished tutors and rotations into 5–6 levels of different quality and effectiveness. The three questionnaires allowed the identification of strengths and weaknesses of tutors, rotations, and RAC. The agreement observed among judges was coherent to the predicted values, suggesting that no particular training is required for developing a shared interpretation of the items.ConclusionsThe work herein presented serves to enrich the armamentarium of tools that resident medical programs can use to monitor their functioning. A larger application of these tools will serve to consolidate and refine further the results presented.Electronic supplementary materialThe online version of this article (doi:10.1186/s13052-014-0106-2) contains supplementary material, which is available to authorized users.
Highlights
The fundamental issue of monitoring and evaluating the quality of training provided by accredited post-graduate medical training programs remains widely unsolved, at least in Europe
The three questionnaires allowed the identification of strengths and weaknesses of tutors, rotations, and Resident Affair Committee (RAC)
The Paediatric Residency Program (PRP) of the University of Padua, in Italy, while waiting for having the proxyevaluation system implemented, addressed the issue of monitoring and evaluating the quality of the training by having the program going through a periodic, systematic ISO:9001 certification process [10]
Summary
Validation of the Tutor Assessment Questionnaire (TAQ) Fifty-one residents (47 females, No 14, 14, 10, 9, 4 for 1st to 5th year residents, respectively) evaluated 26 tutors. The step calibrations were ordered (τpoor-mediocre = − 1.42; τmediocre-respectable = − 1.15; τrespectable-good = − .05; Table 1 Validation of the Tutor Assessment Questionnaire (TAQ) - Locations of residents, tutors and items on the latent variable. The functioning of Item 14 (Scoring “the teaching attitude of the faculty”) changed with the gender of residents (i.e., males provided more severe evaluations than females; t(7) = 2.38, p < .05), whereas Item 5 (“[The RAC] It provides residents with individualized feedbacks regarding the evaluation they received”) exhibited misfit (infit = 1.64, outfit = 1.66). These two items were excluded and a new analysis was run.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
