Interobserver variability when employing the IUGA/ICS classification system for complications related to prostheses and grafts in female pelvic floor surgery

Abstract

To unify and organize reporting, an International Urogynecological Association (IUGA)/International Continence Society (ICS) expert consortium published terminology guidelines with a classification system for complications related to implants used in female pelvic surgery. We hypothesize that the complexity of the codification system may be a hindrance to precision, especially with decreasing levels of postgraduate expertise. Residents, fellows, and attending physicians were asked to code seven test cases taken from published literature. Category, timing, and site components of the classification system were assessed independently and according to the level of training. Interobserver reliability was calculated as percent agreement and Fleiss' kappa statistic. A total of 24 participants (6 attending physicians, 3 fellows, and 15 residents) were tested. The percent agreement showed significant variation when classified by level of training. In all categories, attending physicians had the greatest percentage agreement and largest kappa. The most agreement was seen when attending physicians classified mesh complications by time, 71% agreement with kappa 0.73 [95% confidence interval (CI) 0.58-0.88]. For the same task, the percentage agreement for fellows was 57%, kappa 0.55 (95% CI 0.23-0.87) and with residents 57%, kappa 0.71([95% CI 0.64-0.78). Interestingly, the site component of the classification system had the least overall agreement and lowest kappa [0%, kappa 0.29 (95% CI 0.26-0.32)] followed by the category component [14%, kappa 0.48 (95% CI 0.46-0.5)]. The IUGA/ICS mesh complication classification system has poor interobserver reliability. This trended downward with decreasing postgraduate level; however, we did not have sufficient statistical power to show an association when stratifying by all training levels. This highlights the complex nature of the classification system in its current form and its limitation for widespread clinical and research application.