Evaluation of severity scoring systems in ICUs—translation, conversion and definition ambiguities as a source of inter-observer variability in Apache II, SAPS and OSF

Abstract

To explore translation, conversion and definition ambiguities, when using severity scoring systems in patients admitted to intensive care units (ICUs). A prospective study of the prognosis of acute renal failure in ICUs. The study was conducted in 20 French ICUs. 360 patients presenting with severe acute renal failure were studied during their ICU stay. The inter-observer variability of Apache II (acute physiology and chronic health evaluation), SAPS (simplified acute physiology score), and OSF (organ-system failure) was considered. For Apache II, we explored the uncertainty of measurements arising from conversion into SI units, the rounding procedures used for the non-inclusive intervals defined for quantitative parameters such as age, mean arterial pressure (MAP) or serum creatinine, the absence of definition of acute renal failure (ARF) and its consequence on doubling serum creatinine values, and the absence of guidelines in the case of spontaneous ventilation when arterial blood gases (ABG) and forced inspiratory oxygen (FIO2) were not measured. The resulting variability was evaluated, calculating the lowest and the highest value of the scoring system for each patient. The mean difference by patient was greater than 1.5 (p < 0.0001). Other examples were presented and discussed for SAPS and OSF. Translation, conversion and definition ambiguities are a source of inter-observer variability and increase the risk of classification and/or selection biases. This gives rise to particular concern in the design and analysis of multicenter trials of meta-analysis, and improvement of these scoring systems should be envisaged in the future.