Simulation-based assessment of model selection criteria during the application of benchmark dose method to quantal response data

Keita Yoshii,Takayuki Yamaguchi,Hiroshi Nishiura,Akihiko Hirose,Kaoru Inoue

doi:10.1186/s12976-020-00131-w

Abstract

BackgroundTo employ the benchmark dose (BMD) method in toxicological risk assessment, it is critical to understand how the BMD lower bound for reference dose calculation is selected following statistical fitting procedures of multiple mathematical models. The purpose of this study was to compare the performances of various combinations of model exclusion and selection criteria for quantal response data.MethodsSimulation-based evaluation of model exclusion and selection processes was conducted by comparing validity, reliability, and other model performance parameters. Three different empirical datasets for different chemical substances were analyzed for the assessment, each having different characteristics of the dose-response pattern (i.e. datasets with rich information in high or low response rates, or approximately linear dose-response patterns).ResultsThe best performing criteria of model exclusion and selection were different across the different datasets. Model averaging over the three models with the lowest three AIC (Akaike information criteria) values (MA-3) did not produce the worst performance, and MA-3 without model exclusion produced the best results among the model averaging. Model exclusion including the use of the Kolmogorov-Smirnov test in advance of model selection did not necessarily improve the validity and reliability of the models.ConclusionsIf a uniform methodological suggestion for the guideline is required to choose the best performing model for exclusion and selection, our results indicate that using MA-3 is the recommended option whenever applicable.

Highlights

To employ the benchmark dose (BMD) method in toxicological risk assessment, it is critical to understand how the BMD lower bound for reference dose calculation is selected following statistical fitting procedures of multiple mathematical models
Because of the statistical estimation that we performed, we considered that we knew the unbiased BMD10 and unbiased BMDL10 values that should be recovered by the BMD method using the simulated datasets
The lowest Benchmark dose lower bound (BMDL) or lowest BMD yielded the best validity results, except when the exclusion using the Kolmogorov-Smirnov test (KS test) and Benchmark dose upper bound (BMDU)/BMDL ratio was applied in advance of model selection

Summary

Introduction

To employ the benchmark dose (BMD) method in toxicological risk assessment, it is critical to understand how the BMD lower bound for reference dose calculation is selected following statistical fitting procedures of multiple mathematical models. The BMD method determines the threshold dose by fitting various statistical models to the dose-response curve, which addresses the problems surrounding the use of NOAEL because it can account for the response data across different doses and can help in objectively calculating the point of departure. To employ the BMD method, it is critical to select the best performing BMDL by following the statistical fitting procedures of multiple mathematical models. The BMD method uses a specified percentile point (e.g. 10% of the benchmark response, abbreviated as BMD10) as the threshold for the reference value, but the 10% percentile point is never strictly objective [10,11,12]. Quantitative guides for such restrictions can be complicated for non-expert users

Objectives

Methods

Results

Conclusion