EVALUATING INTER-RATER RELIABILITY OF THE MODIFIED GORDON SCORE FOR PIN SITE INFECTION

Abstract

There is no consensus on how to evaluate and grade pin site infection. A precise, objective and reliable pin site infectious score is warranted. The literature was reviewed for pin site infection classification systems, The Modified Gordon Score (MGS) grade 0-6 was used. The aim was to test the reliability of The Modified Gordon Infection Score. The observed agreement and inter-rater reliability were investigated between nurse and doctors.MGS was performed in the outpatient clinic at Aalborg University Hospital, Denmark on 1472 pin sites in 119 patients by one nurse and one of three orthopaedic surgeons blinded to each other's judgement. The data was stored in a Red Cap Database for further statistical analysis. The observed agreement between the nurse and the 3 orthopaedic surgeons was evaluated with a one-way random-effect model with interclass correlation with absolute agreement. Furthermore the observed agreement for each of the 3 surgeons with the nurse was calculated.The distribution of MGS infection grade in the 1472 pin sites was: Grade 0; n=1372, Grade 1; n=32, Grade 2; n=39, Grade 3; n=24, Grade 4; n=5, Grade 5; n=0, Grade 6; n=0.The observed agreement between the nurse and the surgeons was calculated as 98%. The ICC estimated between nurse and the surgeons was 0,8943 (ICC >0,85 = reliable). The grading was done by three different doctors with an agreement with the nurse as follows. Rater1 (n=416) =99,5 %, Rater2 (n=1440) =97,4%, Rater3 (n=1440) =96,6%.A limitation to this study is that the dataset represents mostly clean pin sites with MGS 0. Only 100 pin sites had signs of superficial infection MGS 1-4 none above 4. We found that the MGS infection score is highly reliable for low grade infections but we cannot conclude on reliability in severe infections.