Measuring impairment caused by leprosy: inter-tester reliability of the WHO disability grading system

Abstract

This paper reports the results of a study on the inter-tester reliability of the WHO disability grading system. The WHO disability grading system is the most frequently used method of grading impairment in leprosy patients. With this method, a grade of 0-2 is assigned to each of six individual body sites (both eyes, hands and feet). The maximum grade of any of these sites is used as an overall indicator of the person's impairment status. To date, the WHO disability grading scale has not been subjected to reliability testing. The reliability of the grading system depends on the operational definitions of the grades, the way the tester interprets these definitions and the skill of the tester. It is therefore important that the definitions are unambiguous and leave as little room as possible for multiple interpretations. Three testers with varying degrees of experience did paired assessments on a total of 150 leprosy patients in the Leprosy Mission Hospital Purulia, India, using recently published operational definitions of the WHO disability grades. For every patient, they determined the maximum grade (minimum 0, maximum 2), and calculated the impairment sum-score (EHF score), adding up the six grades for eyes, hands and feet (minimum 0, maximum 12). The weighted Kappa statistic (Kw) was used as the coefficient of inter-tester reliability. A kappa of 0 represents agreement no better than chance, and 1.0 complete (chance-corrected) agreement. Kw values of > or = 0.80 are considered very good and adequate for monitoring and research. Weighted Kappa analysis yielded a reliability coefficient of 0.89 (95%CI 0.84-0.94) for the maximum grade and a Kw of 0.97 (95%CI 0.96-0.98) for the EHF score. We concluded that, when using standard operational definitions, the WHO disability grading system can be used reliably in the hands of both experienced and inexperienced testers, provided adequate training has been given. Reliability should be evaluated further in a field setting, when used by primary health care workers. It is recommended that the 'WHO disability grading' be renamed 'WHO impairment grading', using the terminology as defined by the International Classification of Functioning, Disability and Health (ICF).