Interrater Reliability of the 2015 Pediatric Acute Lung Injury Consensus Conference Criteria for Pediatric ARDS

Abstract

Diagnostic guidelines for pediatric acute respiratory distress syndrome (PARDS) were developed at the 2015 Pediatric Acute Lung Injury Consensus Conference (PALICC). Although this was an improvement in creating pediatric-specific diagnostic criteria, there remains potential for variability in identification of PARDS. What is the inter-rater reliability of the 2015 PALICC criteria for diagnosing moderate-severe PARDS? What clinical criteria and patient factors are associated with diagnostic disagreements? Patients with acute hypoxic respiratory failure admitted from 2016-2021 who received invasive mechanical ventilation were retrospectively reviewed by two PICU physicians. Reviewers evaluated whether the patient met the 2015 PALICC definition of moderate-severe PARDS and rated their diagnostic confidence. Inter-rater reliability was measured using Gwet's Agreement Coefficient (AC1). 37 of 191 encounters had a diagnostic disagreement. Inter-rater reliability was "substantial" (AC1=0.74, 95% CI [0.65 - 0.83]). Disagreements were due to different interpretations of chest radiographs (56.8%), ambiguity in origin of pulmonary edema (37.8%), or lack of clarity if patient's current condition was significantly different from baseline (27.0%). Disagreement was more likely in patients who were chronically ventilated (OR 4.66, 95% CI [2.16-10.08], p<0.001), had a primary cardiac admission diagnosis (OR 3.36, 95% CI [1.18 - 9.53], p=0.02), or underwent cardiothoracic surgery during the admission (OR 4.90, 95% CI [1.60 - 15.00], p=0.005). Reviewers were at least moderately confident in their decision 73% of the time, however were less likely to be confident if the patient had cardiac disease or chronic respiratory failure. The inter-rater reliability of the 2015 PALICC criteria for diagnosing moderate-severe PARDS in this cohort was substantial, with diagnostic disagreements commonly due to differences in chest radiograph interpretations. Patients with cardiac disease or chronic respiratory failure were more vulnerable to diagnostic disagreements. More guidance is needed on interpreting chest radiographs and diagnosing PARDS in these subgroups.