Decision theory and linear sequential unmasking in forensic fire debris analysis: A proposed workflow

Abstract

The ASTM E1618-19 standard method for fire debris analysis does not endorse the use of statistical analysis to establish error rates or the use of probabilistic statements about the strength of the evidence. The method requires the analyst to report in categorical terms with the possibility of introducing disclaimers. This report introduces a new workflow that extends the data analysis and interpretation methods of ASTM E1618-19. In the new workflow, analysts are asked to assign evidentiary values to fire debris samples. The assigned values are combined with decision theory to determine an optimal decision threshold for ascribing a sample as positive or negative for ignitable liquid residue in categorical terms. The new workflow incorporates linear sequential unmasking with the data analysis methods of ASTM E1618-19, machine learning likelihood ratios and the use of ignitable liquid and substrate databases. The workflow is applied to 20 known ground truth samples by each of three analysts to generate validated personal receiver operating characteristic (ROC) curves and optimal decision thresholds. The ROC curve provides a direct link between perceived evidentiary value and decision thresholds based on a defined ratio of costs for false positive and false negative decisions. The new workflow is applied to casework-relevant large-scale burn data for which the ground truth is unknown.