Multiple shunt failures: an analysis of relevant features.

Abstract

Sir: We read with considerable interest the article entitled “Multiple shunt failure: an analysis of relevant factors” by Jorge Lazareff et al. [2]. We think this is an extremely interesting and important area, and the authors have suggested some fascinating etiological possibilities for shunt failure. However, we would contend that their method of statistical analysis is so flawed that their results are suspect. Their whole analysis is predicated on the division of patients into four groups, depending on the number of shunt revisions (group 1: no failure, group 2: 1 failure, group 3: 2 failures, group 4: 3 or more failures). Their own analysis shows that shunt failure is a time-related event and the longer a patient is followed, the more likely a shunt failure, whether for a first, second or third or more time. The authors included patients entered continuously over a 6-year period, so that the group into which any patient falls is very much a function of how long they have been followed. A patient followed for 6 years has a very high probability of being in a group other than 1, and a patient followed for 6 months has a higher probability of being in group 1, regardless of any of the other factors they examined. While the authors do use survival analysis for time to shunt failure, their Fig. 3 demonstrates a very peculiar application of it. For some reason they exclude patients whose shunts have not failed, when the normal procedure would be to include all patients at risk and develop a curve for first shunt failure. In a similar way, one could develop a curve for second failure, again taking in all patients at risk, but also accounting for the fact that many of the patients from the first failure group are being included. We contend that the statistical analysis they perform on these curves is meaningless. When the authors look at factors related to surgical procedures (their Table 1), they again divide the patients into the same groups according to number of shunt revision. For patients in groups 2 through 4, these procedures are not each applied in a separate patient, but include multiple procedures on the same patient. In statistical terms, these events are not independent and should not be analyzed as such. The above leads to an overestimate of the hazard ratio (risk of failure) and an underestimate of the standard error, and thus of the P-value, leading to risk factors that appear unjustly significant. Furthermore, performance of multiple tests (i.e., evaluation of association between each of the risk factors and the shunt failure) without adjustment for the P-value predisposes to an increase in the type 1 error. With more than 20 covariates and only 108 events something is bound to appear significant when truly it is not. We would be the first to acknowledge that the proper statistical analysis of multiple shunt failures is complex, and it is in fact the subject of ongoing research [1, 3, 4]. However, we think the authors have not adhered to standard statistical methods and that their data analysis is flawed.