Statistical Simulations to Evaluate the Methods of the Construction of Injury Risk Curves

Abstract

Several statistical methods are currently used to build injury risk curves in the biomechanical field. These methods include the certainty method (Mertz et al. 1996), Mertz/Weber method (Mertz and Weber 1982), logistic regression (Kuppa et al. 2003, Hosmer and Lemeshow 2000), survival analysis with Weibull distribution (Kent et al. 2004, Hosmer and Lemeshow 2000), and the consistent threshold estimate (CTE) (Nusholtz et al. 1999, Di Domenico and Nusholtz 2005). There is currently no consensus on the most accurate method to be used and no guidelines to help the user to choose the more appropriate one. Injury risk curves built for the WorldSID 50th side impact dummy with these different methods could vary significantly, depending on the sample considered (Petitjean et al. 2009). As a consequence, further investigations were needed to determine the fields of application of the different methods and to recommend the best statistical method depending on the biomechanical sample considered. This study used statistical simulations on theoretical samples to address these questions. Two different theoretical distributions of injury thresholds were utilized to assess the five different methods of constructing injury risk curves. A normal distribution and a Weibull distribution, whose shape was not similar to a normal distribution, were selected. One hundred sets of "test subjects" were randomly chosen from each theoretical distribution, with sample sizes ranging from 10 to 50. A "stimulus value" was chosen for each "test subject." The stimulus values were equally spaced, distributed tightly or loosely about the theoretical mean injury threshold, concentrated below the mean value, or concentrated above the mean value. An adaptive method was also used to assign stimulus values, based on the proportion of uninjured and injured in an early subset of the test subjects. The influences of 10%, 25%, and 50% exact data were compared to stimulus values that were either right or left censored. The test subject was considered to be uninjured if the stimulus was less than the subject's threshold or injured if the stimulus was equal to or greater than the subject's threshold. In all, 12,800 simulated data sets with both normal and Weibull distributions were used to construct injury risk curves by each of the five statistical methods. Cumulative errors of the constructed injury risk curves, compared to the theoretical curves, were calculated across the whole curve, as well as the portion of the theoretical curve up to 25% risk of injury. P-values were used to assess the significance of the differences in the errors. The CTE and the survival analysis take into account the exact data whatever the theoretical distribution of injury threshold, while the logistic regression, the Mertz/Weber and the certainty methods do not. For left and right censored data, the logistic regression and/or the survival analysis lead to the lowest error. The survival analysis leads to the lowest error whatever the sample size, the level of censoring and the theoretical distribution evaluated. Increasing the sample size generally decreased the error. However, the benefit from increasing the sample size decreased when the sample size was already high. For the survival analysis, increasing the proportion of exact data decreased the error. The same way, the benefit from increasing the proportion of exact data decreased when the proportion of exact data was already high. Survival analysis may not converge for small sample size with left and right censored data. The number of simulations for which the survival analysis did not converge highly decreases with the increase of proportion of exact data and the increase of the sample size. Therefore, it is recommended to use survival analysis with Weibull distribution to build risk curves compared to the four other statistical methods evaluated. The accuracy of survival analysis with other distributions (log-normal, log-logistic, etc) was not studied. There is no recommendation for the method to be used when survival analysis does not converge. The balance between maximal acceptable error and the need for an injury risk curve, even for a small dataset of poor quality, is not addressed.