Inter-rater reliability in the Testing methods of the International Dysphagia Diet Standardization Initiative (IDDSI) in Slovenian language

Abstract

Background: Texture modification of drinks and food is a widespread clinical intervention for managing dysphagia. The descriptive and experiential approach to measurement is mainly used by healthcare professionals, caregivers, and patients. This study examines the inter-rater reliability of the International Dysphagia Diet Standardization Initiative (IDDSI) Testing methods on different food samples from the hospital’s central kitchen.
 Methods: Nine previously trained healthcare professionals of different professional backgrounds and working experiences independently evaluated 34 food samples in IDDSI levels 4, 5, 6, and 7.1, using a Slovenian translation of the IDDSI Audit Tools. The 12 descriptive food characteristics were observed during the two consecutive days of data collection, representing 7 IDDSI Testing Methods. The Fleiss’ Kappa was used as a measure of agreement.
 Results: Perfect agreement among raters was found for the Fork drip test regarding both observed parameters: dripping and holding shape. All ratings of participants were completely the same. Therefore, Fleiss’ Kappa is considered (κ =1) in both cases. Moderate agreement (κ = 0.583) was found in particle size assessment. Fair agreement was observed in the Fork Pressure Test (κ = 0.361) and Finger Test, indicating bolus separation (κ = 0.328), pressure (κ = 0.305), and moisture (κ = 0.297). Fair agreement was also found for the Fork/Spoon Separation Test (κ = 0.265) and the Spoon Tilt Test in parameters of holding shape on the spoon (κ =0.233) and stickiness (κ = 0.327). No statistically significant results were found in the assessment of adhesiveness (κ = 0.059) in the Spoon Tilt Test and assessment of a typical pattern (κ = 0.039) in the Fork Pressure Test.
 Conclusions: The study proved the Fork Drip Test to be a comprehensive and highly reliable assessment tool, while the other IDDSI Testing Methods showed low inter-rater reliability. The study highlights the risks of low agreement among raters. It draws attention to the IDDSI descriptors, which should be improved with additional instructions or standardised training protocol for learning the IDDSI Testing Methods.