Exploring expert variability in defining pseudoparalysis: an international survey

Abstract

There is currently disagreement among experts in the field of shoulder surgery when attempting to define the term "pseudoparalysis." Multiple surgical techniques to address this condition have been investigated; however, many studies have recruited heterogeneous patient populations and have used varying definitions of pseudoparalysis. This makes it difficult to compare outcomes among various techniques. To our knowledge, no previous study has surveyed international experts regarding the definition of pseudoparalysis using a questionnaire and video-based patient assessment. The purpose of this study was to evaluate the level of agreement among shoulder surgeons in defining and applying the term "pseudoparalysis." We hypothesized that inter-rater agreement for classifying patients as having pseudoparalysis would be poor. Members of the American Shoulder and Elbow Surgeons, the European Society for Surgery of the Shoulder and the Elbow, and our national shoulder and elbow society were surveyed on 2 occasions using an electronic questionnaire. All surgeons were asked to identify their preferred definition of pseudoparalysis from 1 of 4 options. The surgeons then viewed video examinations of 10 patients and labeled them as having pseudoparalysis or not. Inter-rater reliability and intrarater reliability were calculated as κ coefficients. The Pearson χ2 test was used to detect associations between the preferred definition and demographic information. A total of 246 surgeons responded to at least 1 survey. Overall inter-rater agreement on classifying patients as having pseudoparalysis based on video consultation showed a κ value of 0.59 (95% confidence interval [CI], 0.58-0.60). The same verbal definition was selected by 56.1% of surgeons. The surgeons were not internally consistent in their choice of definition, with intrarater reliability showing a κ value of 0.64 (95% CI, 0.48-0.81). Intrarater reliability for classifying patients as having pseudoparalysis was better, with a κ value of 0.78 (95% CI, 0.72-0.83). An association was observed between how surgeons defined pseudoparalysis and their age (P = .03), as well as their shoulder caseload percentage (P = .04). Shoulder surgeons do not agree on how best to define pseudoparalysis of the shoulder. Inter-rater agreement based on video consultation was weak overall and improved with the elimination of an outlier video. Intrarater agreement was less frequent when selecting a preferred definition compared with classifying patients as having pseudoparalysis based on video examinations. Surgeons may rely less on explicit criteria and more on a conceptual framework when assigning a "pseudoparalytic" label. Care should be taken with use of the term "pseudoparalysis" in clinical outcome studies when there is clearly a lack of consensus among experts in defining this term.