Firearm Evidence Research Articles

Combined interpretation of objective firearm evidence comparison algorithms using Bayesian networks.

Traditionally, firearm and toolmark examiners manually evaluate the similarity of features on two bullets using comparison microscopy. Advances in microscopy have made it possible to collect 3D topographic data, and several automated comparison algorithms have been introduced for the comparison of bullet striae using these data. In this study, open-source approaches for cross-correlation, congruent matching profile segments, consecutive matching striations, and a random forest model were evaluated. A statistical characterization of these automated approaches was performed using four datasets of consecutively manufactured firearms to provide a challenging comparison scenario. Each automated approach was applied to all samples in a pairwise fashion, and classification performance was compared. Based on these findings, a Bayesian network was empirically learned and constructed to leverage the strengths of each individual approach, model the relationship between the automated results, and combine them into a posterior probability for the given comparison. The network was evaluated similarly to the automated approaches, and the results were compared. The developed Bayesian network classified 99.6% of the samples correctly, and the resultant probability distributions were significantly separated more so than the automated approaches when used in isolation.

Overall Proportion of Total DNA Consistent with an Individual Briefly Handling a Firearm.

In forensic investigations, DNA profiles are routinely obtained from firearms evidence and alternative hypotheses may be proposed for consideration on the activity level. DNA profiles found to be consistent with the DNA profile of a specific individual could be a result of directly handling the firearm or other modes of transfer of DNA. Sixteen law-enforcement-owned firearms were evaluated with samples collected from the frame and slide area, the trigger and trigger guard area, and the front and rear sights after brief handling by laboratory personnel. Twenty-two out of forty-eight samples resulted in DNA profiles suitable for comparison, of which six resulted in likelihood ratios (LR) that demonstrated support for the hypothesis that included the brief handler as a contributor to the DNA profile obtained from the sample. Five of these samples were obtained from the frame and slide and one was from the trigger and trigger guard area. None of the DNA profiles obtained from the sights supported the inclusion of the brief handler as a contributor to the DNA profile. Gaining knowledge and supporting data on the nature of DNA profiles typically obtained from both owners and brief handlers can be useful for the purposes of evaluative reporting when considering results obtained from firearm evidence.

A Study in Reproducibility: The Congruent Matching Cells Algorithm and cmcR Package

Scientific research is driven by our ability to use methods, procedures, and materials from previous studies and further research by adding to it. As the need for computationally-intensive methods to analyze large amounts of data grows, the criteria needed to achieve reproducibility, specifically computational reproducibility, have become more sophisticated. In general, prosaic descriptions of algorithms are not detailed or precise enough to ensure complete reproducibility of a method. Results may be sensitive to conditions not commonly specified in written-word descriptions such as implicit parameter settings or the programming language used. To achieve true computational reproducibility, it is necessary to provide all intermediate data and code used to produce published results. In this paper, we consider a class of algorithms developed to perform firearm evidence identification on cartridge case evidence known as the *Congruent Matching Cells* (CMC) methods. To date, these algorithms have been published as textual descriptions only. We introduce the first open-source implementation of the Congruent Matching Cells methods in the R package cmcR. We have structured the cmcR package as a set of sequential, modularized functions intended to ease the process of parameter experimentation. We use cmcR and a novel variance ratio statistic to explore the CMC methodology and demonstrate how to fill in the gaps when provided with computationally ambiguous descriptions of algorithms.

An Introduction to the Automatic and Objective Firearm Evidence Identification and Report of Error Rate

An Introduction to the Automatic and Objective Firearm Evidence Identification and Report of Error Rate

Reporting Likelihood Ratio (LR) for Case Work in Firearm Evidence Identifications

Reporting Likelihood Ratio (LR) for Case Work in Firearm Evidence Identifications

Reporting likelihood ratio for casework in firearm evidence identification

Likelihood ratio (LR) plays an important role in estimating the weight of evidence in firearm evidence identifications. LR is computed from a statistical model including the distribution of the known-matching (KM or within) and known-nonmatching (KNM or between) comparison scores. Current LR procedures rely on KM/KNM scores from existing reference firearm toolmark data sets or alternatively from generating a set of test fires using multiple firearms. Both procedures may contain theoretical or practical issues which may hinder the LR procedures from reporting an unbiased LR estimation in casework. In this paper, a reference data set was established from a set of firearms, each test-fired two cartridge cases, resulting in a basic data set and a control data set. The congruent matching cells (CMC) method was used to generate CMC scores that are used to fit in the KM/KNM statistical distributions for LR estimation. In the initial test, 130 firearms from eight manufacturers were used for generating a reference data set consisting of 260 cartridge cases representing 130 KM and 8385 KNM pairwise breech face images. Test results showed that the KM and KNM distribution intersect at CMC=2, which is equivalent to LR=1 (equally to support both the prosecutor and the defense propositions). When the CMC threshold is increased to 6 or more, the LR values are higher than a million, which can provide extremely strong support to the conclusion of the same firearm (or the prosecutor's proposition) in the casework of firearm evidence identification.

Virtual image standard (VIS) for performance evaluation of the congruent matching cells (CMC) algorithms in firearm evidence identifications

The congruent matching cells (CMC) method was invented at the National Institute of Standards and Technology (NIST) in 2012 for automatic and objective firearm evidence identifications and estimation of the weight of evidence in firearm evidence identifications. Since 2013, five CMC algorithms have been developed at NIST. In this paper, the virtual image standard (VIS) is proposed through trimming and stitching KNM images for quantitative performance evaluations of different CMC algorithms. The evaluation criteria include the correlation accuracy (both the CMC numbers and distribution pattern), correlation efficiency, false positive (FP) error rate, and the maximum separation of known matching (KM) and known non-matching (KNM) image pairs. The VIS composes correlation cells from different KNM images, which can provide a ground truth for verifying the CMC numbers, distribution patterns, and FP errors. By identifying three groups of VIS, the Convergence CMC algorithm showed superior performances for the future casework in firearm evidence identifications. Lastly, the success of this study suggests that the VIS could also be used to optimize the correlation parameters, to develop and test new CMC algorithms, and evaluate the performance before it is put into use for firearm examiner's casework.

Statistical models for firearm and tool mark image comparisons based on the congruent matching cells (CMC) method

In the branch of forensic science known as firearm evidence identification, various similarity scores have been proposed to compare firearm marks. Some similarity score comparisons, for example, congruent matching cells (CMC) method, are based on pass-or-fail tests. The CMC method compares the pairwise topography images of breech face impressions, from which the similarity score is derived for quantifying their topography similarity. For an image pair, the CMC method determines a certain number of correlated cell pairs. Next, each correlated pair is determined to be a congruent match cell (CMC) pair, or not based on several identification parameters. The number of CMC pairs as a threshold is required so that the two images of surface topographies can be either identified as matching or determined to be non-matching. To reliably estimate error rates or evaluate likelihood ratio (LR), the key is to find an appropriate probability distribution for the frequency distribution of the observed CMC results. This paper discusses four statistical models for CMC measurements, which are binomial and three binomial-related probability distributions. In previous studies, for a sequence of binomial distributed or other binomial-related distributed random variables (r.v.), the number of Bernoulli trials N for each r.v. is assumed to be the same. However, in practice, N(the number of cell pairs in an image pair) varies from one r.v. (or one image pair) to another. In that case, the term, frequency function, of the CMC results is not appropriate. In this paper, the generalized frequency function is introduced to depict the behavior of the CMC values and its limiting distribution is provided. Based on that, nonlinear regression models are used to estimate the model parameters. The methodology is applied to a set of actual CMC values of fired cartridge cases.

Brief for Indian Law and Policy Professors as Amici Curiae Supporting Petitioner in United States v. Cooley

Amici curiae are 25 scholars who teach, write, and/or practice in the area of federal Indian law and federal Indian policy. The amici are concerned with the Ninth Circuit Court’s decision in United States v. Cooley, 919 F.3d 1135 (9th Cir. 2019). There, the circuit court held that tribal police officers lack authority to briefly detain and search a non-Indian on a public highway running through the Crow Reservation unless it is either “apparent” or “obvious” that the non-Indian has violated state or federal law. Under that theory, the court affirmed the district court’s decision to suppress drug and firearm evidence obtained from a non-Indian defendant by a tribal law enforcement officer. The amici urge the Supreme Court to reverse the Ninth Circuit decision as the circuit court’s legal theory contradicts the custom, tradition, and policy of policing, as well as the treaties between the United States and indigenous nations. This all demonstrates that the United States has long recognized tribal authority to detain and search non-Indians.

Evaluating Likelihood Ratio (LR) for firearm evidence identifications in forensic science based on the Congruent Matching Cells (CMC) method

Firearm evidence identification has been challenged by the 2008 and 2009 National Research Council (NRC) reports and by legal proceedings on its fundamental assumptions, its procedure involving subjective interpretations, and the lack of a statistical foundation for evaluation of error rates or other measures for the weight of evidence. To address these challenges, researchers of the National Institute of Standards and Technology (NIST) recently developed a Congruent Matching Cells (CMC) method for automatic and objective firearm evidence identification and quantitative error rate evaluation. Based on the CMC method, a likelihood ratio (LR) procedure is proposed in this paper aiming to provide a scientific basis for firearm evidence identification and a method for evaluation of the weight of evidence. The initial LR evaluations using two sets of 9mm cartridge cases’ breech face impression images with different sample sizes, imaging methods and ammunition showed that for all the declared identifications of the tested 2D and 3D image pairs, the evaluated LRs for the least favorable scenario were well above an order of 106, which provides Extremely Strong Support for a prosecution proposition (e.g. a same-source proposition) in a Bayesian frame. The LR evaluations also showed that for all the declared exclusions of the tested 3D image pairs, the evaluated LRs for the least favorable scenario were above an order of 102, which provides Moderately Strong Support for a defense proposition (e.g. a different-source proposition) in a Bayesian frame.

The Use of Correlated Binomial Distribution in Estimating Error Rates for Firearm Evidence Identification

The Use of Correlated Binomial Distribution in Estimating Error Rates for Firearm Evidence Identification

The Use of Correlated Binomial Distribution in Estimating Error Rates for Firearm Evidence Identification

The Use of Correlated Binomial Distribution in Estimating Error Rates for Firearm Evidence Identification

The Use of Correlated Binomial Distribution in Estimating Error Rates for Firearm Evidence Identification

In the branch of forensic science known as firearm evidence identification, estimating error rates is a fundamental challenge. Recently, a new quantitative approach known as the congruent matching cells (CMC) method was developed to improve the accuracy of ballistic identifications and provide a basis for estimating error rates. To estimate error rates, the key is to find an appropriate probability distribution for the relative frequency distribution of observed CMCs overlaid on a relevant measured firearm surface such as the breech face of a cartridge case. Several probability models based on the assumption of independence between cell pair comparisons have been proposed, but the assumption of independence among the cell pair comparisons from the CMC method may not be valid. This article proposes statistical models based on dependent Bernoulli trials, along with corresponding methodology for parameter estimation. To demonstrate the potential improvement from the use of the dependent Bernoulli trial model, the methodology is applied to an actual data set of fired cartridge cases.

A Normalized Congruent Matching Area Method for the Correlation of Breech Face Impression Images.

The congruent matching cells (CMC) method was invented at the National Institute of Standards and Technology (NIST) for firearm evidence identification and error rate estimation. The CMC method divides the correlated image pairs into cells and uses four parameters to quantify topography similarity and pattern congruency of the correlated cell pairs in firearm breech face impressions on fired cartridge cases. A preliminary conservative numerical identification criterion of C = 6 CMCs was suggested for identifying images of cartridge cases fired from the same firearm. The CMC method was validated by correlations using both three-dimensional (3D) topography images and two-dimensional (2D) optical images from a set of 40 cartridge cases fired from a firearm set composed of 10 consecutively manufactured pistol slides. However, in the original CMC method, due to the difference in the effective data area of the correlated cells, final CMCs obtained from an image pair presented different data quantity (or validity level), and thus the empirical criterion C = 6 CMCs did not remain optimal for identification when the correlated cell size changed. In this study, a normalized congruent matching area (NCMA) method that considers the difference in the data area in each correlated cell pair was developed. Based on the NCMA method, an optimal range of cell sizes for breech face identification with granular characteristics was determined. A binomial model was used to fit the known nonmatching NCMA probability distribution Ψ NCMA, and a beta-binomial model was used to fit the known matching NCMA probability distribution Φ NCMA. An experimental improvement in the normalized identification criterion C of around 6 % was observed in the validation tests when the cell sizes were in the optimal range.

Using statistical models in estimating error rates for forensic firearm identification

Estimating error rates for firearm evidence identification is a fundamental challenge in forensic science. The recently developed Congruent Matching Cells (CMC) method provides applications to firearm evidence identification. To estimate error rates, appropriate statistical models are needed for the CMC values. In this paper, in addition to the binomial probability distribution the correlated binomial distribution is proposed. For an image comparison, the correlated binomial distribution can be applied to the cell pairs from CMC method. An application to an actual data set demonstrates that the correlated binomial distribution fits the relative frequency distribution of CMC values much better than the binomial distribution.

Pilot study of feature-based algorithm for breech face comparison

A novel feature-based method, which is scale invariant feature transform (SIFT) and RANdom SAmple Consensus (RANSAC) integration algorithm, is introduced to promote the automated identification of the breech face impression, the most common mark left on the cartridge used for firearm evidence. SIFT algorithm is employed to extract the local extrema from examined impression as keypoints representing its invariant features, and to build the feature descriptor for each keypoint based on its local gradients in neighborhood. RANSAC is used to improve the matching performance among these keypoints and feature descriptors. With hypothesize-and-verify methods, RANSAC is able to construct the best model fitting initial matching pairs of keypoints and to guarantee the robust comparison result. Validation tests using 40 cartridge cases fired from pistols with 10 consecutively manufactured slides yielded a clear separation result, which strongly supports the effectiveness of the ensemble algorithm of SIFT and RANSAC. This application indicates the practical feasibility of feature-based algorithm and image processing technique in forensic science.

Estimating error rates for firearm evidence identifications in forensic science

Estimating error rates for firearm evidence identification is a fundamental challenge in forensic science. This paper describes the recently developed congruent matching cells (CMC) method for image comparisons, its application to firearm evidence identification, and its usage and initial tests for error rate estimation. The CMC method divides compared topography images into correlation cells. Four identification parameters are defined for quantifying both the topography similarity of the correlated cell pairs and the pattern congruency of the registered cell locations. A declared match requires a significant number of CMCs, i.e., cell pairs that meet all similarity and congruency requirements. Initial testing on breech face impressions of a set of 40 cartridge cases fired with consecutively manufactured pistol slides showed wide separation between the distributions of CMC numbers observed for known matching and known non-matching image pairs. Another test on 95 cartridge cases from a different set of slides manufactured by the same process also yielded widely separated distributions. The test results were used to develop two statistical models for the probability mass function of CMC correlation scores. The models were applied to develop a framework for estimating cumulative false positive and false negative error rates and individual error rates of declared matches and non-matches for this population of breech face impressions. The prospect for applying the models to large populations and realistic case work is also discussed. The CMC method can provide a statistical foundation for estimating error rates in firearm evidence identifications, thus emulating methods used for forensic identification of DNA evidence.

A convergence algorithm for correlation of breech face images based on the congruent matching cells (CMC) method

The Congruent Matching Cells (CMC) method was invented at the National Institute of Standards and Technology (NIST) for accurate firearm evidence identification and error rate estimation. The CMC method is based on the principle of discretization. The toolmark image of the reference sample is divided into correlation cells. Each cell is registered to the cell-sized area of the compared image that has maximum surface topography similarity. For each resulting cell pair, one parameter quantifies the similarity of the cell surface topography and three parameters quantify the pattern congruency of the registration position and orientation. An identification (declared match) requires a significant number of CMCs, that is, cell pairs that meet both similarity and pattern congruency requirements. The use of cell correlations reduces the effects of “invalid regions” in the compared image pairs and increases the correlation accuracy. The identification accuracy of the CMC method can be further improved by considering a feature named “convergence,” that is, the tendency of the x–y registration positions of the correlated cell pairs to converge at the correct registration angle when comparing same-source samples at different relative orientations. In this paper, the difference of the convergence feature between known matching (KM) and known non-matching (KNM) image pairs is characterized, based on which an improved algorithm is developed for breech face image correlations using the CMC method. Its advantage is demonstrated by comparison with three existing CMC algorithms using four datasets. The datasets address three different brands of consecutively manufactured pistol slides, with significant differences in the distribution overlap of cell pair topography similarity for KM and KNM image pairs. For the same CMC threshold values, the convergence algorithm demonstrates noticeably improved results by reducing the number of false-positive or false-negative CMCs in a comparison.

Statistical Approaches to Type Determination of the Ejector Marks on Cartridge Cases.

While type determination on bullets has been performed for over a century, type determination on cartridge cases is often overlooked. Presented here is an example of type determination of ejector marks on cartridge cases from Glock and Smith & Wesson Sigma series pistols using Naïve Bayes and Random Forest classification methods. The shapes of ejector marks were captured from images of test-fired cartridge cases and subjected to multivariate analysis. Naïve Bayes and Random Forest methods were used to assign the ejector shapes to the correct class of firearm with success rates as high as 98%. This method is easily implemented with equipment already available in crime laboratories and can serve as an investigative lead in the form of a list of firearms that could have fired the evidence. Paired with the FBI's General Rifling Characteristics (GRC) database, this could be an invaluable resource for firearm evidence at crime scenes.

A Review of NIST Projects in Surface and Topography Metrology for Firearm Evidence Identification in Forensic Science

This is a review of the National Institute of Standards and Technology’s (NIST) efforts in surface metrology and topography measurements for firearm evidence identifications in forensic science. Based on the research projects in surface metrology and standardization, NIST researchers have developed Standard Reference Material (SRM) Bullets and Cartridge Cases (Certain commercial equipment, instruments, or materials are identified in this paper to specify adequately the experimental procedure. Such identification does not imply recommendation or endorsement by the National Institute of Standards and Technology, nor does it imply that the materials or equipment identified are necessarily the best available for the purpose) and established a 2D/3D Ballistics Topography Measurement System. They formulated a Traceability and Quality System to support nationwide ballistics identifications within the National Integrated Ballistics Information Network (NIBIN) in the United States. They have recently proposed a Congruent Matching Cells (CMC) method for accurate ballistic identification and error rate estimation, which provides a statistical foundation and a practical method to promote firearm evidence identifications from qualitative image comparisons to quantitative topography measurements.