Firearm Examination Research Articles

A preliminary study of the manufacturing of breech faces.

Firearm examiners may face challenges with subclass characteristics. Subclass characteristics have the potential to be confused with individual characteristics if a careful analysis for the presence of subclass characteristics is not performed. Several case studies are reported in the literature that discuss the presence of subclass characteristics and advise firearm examiners to disregard these characteristics for identification purposes. As far as can be determined, no fundamental study has been performed that explores the various methods used in firearm manufacturing and assess the potential of subclass characteristics being produced. In this study broaching, plunge milling, and lathe turning were used to produce consecutively manufactured breech faces for a Thompson/Center® (T/C®) Contender® G2 pistol. The breech faces were finished by tumbling or glass bead blasting. The 3D topographical scans of the breech faces were collected using a confocal microscope and then compared using the National Institute of Standards and Technology (NIST) congruent matching cells (CMC) algorithm. The results of this study demonstrated the presence of subclass characteristics after manufacturing by broaching and plunge milling. Lathe turning did not produce any subclass characteristics. ANOVA and Kruskal-Wallis H tests were used to determine significant differences between finishing methods within each manufacturing method. Significant differences were found in glass bead blasting versus tumbling and glass bead blasting versus no finish. No significant differences were found in no finish versus tumbling.

Why the post-identification era is long overdue: Commentary on the current controversy over forensic feature comparison as applied to forensic firearms examination

In this commentary, we critically review recurring arguments for and against the discipline of forensic feature comparison as applied to firearms examination from various commentators within and outside forensic science. One of the mainstream criticisms that we address, among others, is that the field cannot demonstrate sufficient proficiency and robustness based on empirical (i.e., black-box) studies. While the lack of empirically demonstrated examiner proficiency is a valid concern and a powerful concept in the short term (e.g., in admissibility proceedings), many critics reduce their discussion of forensic feature comparison solely to the need to measure and demonstrate proficiency through error rates. However, the exclusive focus on aggregate expert performance metrics, here referred to as examiner diagnosticism, remains a surface-level perspective. It provides an incomplete account of the field because these metrics do not represent—but are often confused with—the notion of the evidentiary value of findings, i.e., observations made on examined items in individual cases. We argue that examiner diagnosticism should be contrasted and complemented with the notion of feature selectivity, i.e., the diagnostic capacity of observed marks and features on examined items. We argue that forensic scientists should report and be probed on their ability to quantify feature selectivity (i.e., the probative value of findings). By ceasing to express source attribution opinions (identification/individualisation), which are now widely exposed as unscientific, the forensic feature comparison disciplines could move further into the long-awaited post-identification era pioneered by other fields such as forensic genetics.

The effect of human decomposition on bullet examination

Most firearm related homicides involve the deceased being forensically examined within a day or two, however, there are times when bodies have been examined and the fired components removed several days or weeks after death, when the body is in an active or advanced state of decomposition. In these cases, ballistic investigation has been found to be complicated due to the damage to the bullets, however the extent of this is not yet known. To date, there have been no studies investigating the effect of human decomposition and the subsequent analysis of bullets lodged in the body in an Australian context. Herein, seven fired copper jacketed bullets were manually inserted into three specific tissue types; lungs, abdomen and leg muscle (twenty-one bullets in total), of human donors in both cool and warm conditions at the Australian Facility for Taphonomic Experimental Research (AFTER). Bullets were removed every three days for a period of twenty-one days, and each bullet underwent manual microscopic examinations by firearms examiners across Australia. Results have indicated that the bullets corrode quickly in warm conditions, compared to bullets exposed to decomposition in cooler conditions. The results of this study will inform investigators and pathologists of the need to remove and examine fired bullets from decomposed bodies as soon as possible, especially in warm conditions to provide firearms examiners with the best opportunity to link fired bullets to a common source.

A call for open science in forensics

The modern canon of open science consists of five “schools of thought” that justify unfettered access to the fruits of scientific research: i) public engagement, ii) democratic right of access, iii) efficiency of knowledge gain, iv) shared technology, and v) better assessment of impact. Here, we introduce a sixth school: due process. Due process under the law includes a right to “discovery” by a defendant of potentially exculpatory evidence held by the prosecution. When such evidence is scientific, due process becomes a Constitutional mandate for open science. To illustrate the significance of this new school, we present a case study from forensics, which centers on a federally funded investigation that reports summary statistics indicating that identification decisions made by forensic firearms examiners are highly accurate. Because of growing concern about validity of forensic methods, the larger scientific community called for public release of the complete analyzable dataset for independent audit and verification. Those in possession of the data opposed release for three years while summary statistics were used by prosecutors to gain admissibility of evidence in criminal trials. Those statistics paint an incomplete picture and hint at flaws in experimental design and analysis. Under the circumstances, withholding the underlying data in a criminal proceeding violates due process. Following the successful open-science model of drug validity testing through “clinical trials,” which place strict requirements on experimental design and timing of data release, we argue for registered and open “forensic trials” to ensure transparency and accountability.

The false promise of firearms examination validation studies: Lay controls, simplistic comparisons, and the failure to soundly measure misidentification rates.

Several studies have recently attempted to estimate practitioner accuracy when comparing fired ammunition. But whether this research has included sufficiently challenging comparisons dependent upon expertise for accurate conclusions regarding source remains largely unexplored in the literature. Control groups of lay people comprise one means of vetting this question, of assessing whether comparison samples were at least challenging enough to distinguish between experts and novices. This article therefore utilizes such a group, specifically 82 attorneys, as a post hoc control and juxtaposes their performance on a comparison set of cartridge case images from one commonly cited study (Duez etal. in J Forensic Sci. 2018;63:1069-1084) with that of the original participant pool of professionals. Despite lacking the kind of formalized training and experience common to the latter, our lay participants displayed an ability, generally, to distinguish between cartridge cases fired by the same versus different guns in the 327 comparisons they performed. And while their accuracy rates lagged substantially behind those of the original participant pool of professionals on same-source comparisons, their performance on different-source comparisons was essentially indistinguishable from that of trained examiners. This indicates that although the study we vetted may provide useful information about professional accuracy when performing same-source comparisons, it has little to offer in terms of measuring examiners' ability to distinguish between cartridge cases fired by different guns. If similar issues pervade other accuracy studies, then there is little reason to rely on the false-positive rates they have generated.

Forensic document examination: where to from here?

ABSTRACT In late 2022, the Australia New Zealand Policing Advisory Agency National Institute of Forensic Science (ANZPAA NIFS) Document Specialist Advisory Group (DocSAG) began a capability mapping project with the broad aim to consider, in the Australian and New Zealand context, forensic document examination: where are we now, and where to from here? Our capability mapping project is based largely on the approach used for the 2022 Forensic Capability Mapping – Firearm Examination Project, led by ANZPAA NIFS. A survey was developed and disseminated to capture information addressing key topics: current and emerging risks, improvement areas, current and future capability, and business continuity. This paper outlines the project process and summarizes key findings of the first round of data collection.

The use of machine learning for the determination of a type/model of firearms by the characteristics on cartridge cases

Cartridge cases are commonly collected at crime scenes involving firearms. One of the stages in forensic examination is the determination of the type and model of firearms based on the class characteristics of these cartridge cases. A firearm examiner evaluates the class characteristics on the basis of their knowledge and experience, and by referring to collections of cartridge cases representing class characteristics of different firearms, special databases and reference books. However, this process is highly subjective. The novelty of this research is in developing objective methods of firearms determination by applying a machine learning approach. In this study, several Convolutional Neural Networks from Keras programming package were trained to determine the type/model of a firearm based on the class characteristics observed on cartridge cases from seven different categories of firearms. The prediction accuracies received by this method range from 71 to 81 percent for models based on different Convolutional Neural Networks, while using an ensemble of the machine learning models increased the accuracy to 88 %. The research demonstrates the efficacy of machine learning in enhancing accuracy and reducing subjectivity in firearm identification, highlighting its significant potential in forensic science applications.

The relevance of current forensic firearms examination techniques when applied to 3D printed firearms.

This research project had two major objectives. The first was to successfully print and fire the Liberator, a 3D-printed firearm, to assess its feasibility as a lethal weapon. The second objective was to identify any individual characteristics that might be deposited during the firing process by the firearm. The Liberator was printed using unchanged files downloaded from the internet using PLA and ABS filament. The Liberator was fired remotely into newspapers at the Allegheny County Medical Examiner's Office. The printing of the Liberator was both inexpensive and relatively quick with only minor hand modifications made after printing. The Liberator fractured beyond repair after firing but successfully fired and penetrated three newspapers. Neither the bullet nor the cartridge case exhibited any individual characteristics that could be used for identification purposes. Suspected thermoplastic deposits were identified on both the bullet and cartridge case, but additional testing must be done for confirmation purposes. In conclusion, the Liberator can be used reliably for one shot and will not yield any evidence for Firearms and Toolmark Examination.

“The Smoking Gun” Evidence in the Tragic Death of Vincent Van Gogh

It has been widely accepted in the past that Vincent Van Gogh died of a self-inflicted gunshot wound in a field outside of Auvers-sur-Oise, France. However, much of the new evidence points to another theory: Van Gogh didn’t kill himself but was murdered. The updated forensic research into this cold case is significant for the art community and art historians interested in the truth. Interviews, letters, and records were compiled and reviewed in detail, including evidence regarding the potential weapon and possible persons of interest. Van Gogh’s wound was described as pea sized, surrounded by discoloration with no exit wound or powder burn. The records of these details have been reviewed by a forensic pathologist, who concluded the unlikelihood of self-infliction. No potentially related gun, bullet, or other weapon were found immediately following his death. However, 60 years later a rusted and significantly damaged non-functioning Lefaucheux pinfire revolver was found in a wheat field, coincidentally and somewhat suspiciously, as the movie, LUST FOR LIFE, was being filmed on site. Whether this weapon, recently auctioned for approximately $183,000 USD, was related to his death is debated. Further investigation by a qualified gunsmith or forensic firearms examiner needs to be undertaken to determine if this gun was actually functional when it was “dropped” in the field or placed there to create additional buzz for BIG budget Hollywood production. Vincent van Gogh was allegedly shot in the abdomen on July 27, 1890, allegedly as critically there is no definitive evidence to support a gunshot wound. There is no smoking gun in evidence, no bullet, no suicide note, no exit wound, no black powder, no autopsy, no known crime scene, no witnesses, and no solid evidence of any kind to support a gun shot. All that is known for certain is that he sustained a penetrating wound to his abdomen. All one has to go on to make any kind of forensic analysis, or even confirm the possibility of a gunshot wound, is a description of the wound 26 years later.

The systemic error in the vertical component of handgun bullet trajectory reconstructions.

Establishing the path or trajectory of a fired bullet is an often recurring part of shooting incident reconstruction. The current study describes how gravitational pull causes a systemic error on the vertical component of a trajectory reconstruction. Bullet drop, drop angle, and vertical offset are explained and calculated for 10 different handgun/ammunition combinations over a range of distances up to 100 m. The presented results are intended to provide forensic firearm examiners with a reference frame for the magnitude of error introduced on handgun bullet trajectory reconstructions over distance. Threshold values of 20 and 30 m are proposed as conservative distances up to where bullet trajectories can be modeled as straight lines with subsonic/transonic handgun bullets and with supersonic handgun bullets respectively. Both the bullet drop and vertical offset will be below 5 cm at these distances for those categories. The drop angle will be below 0.3°.

Classification of firing pin impressions using HOG-SVM.

Crimes, such as robbery and murder, often involve firearms. In order to assist with the investigation into the crime, firearm examiners are asked to determine whether cartridge cases found at a crime scene had been fired from a suspect's firearm. This examination is based on a comparison of the marks left on the surfaces of cartridge cases. Firing pin impressions can be one of the most commonly used of these marks. In this study, a total of nine Ruger model 10/22 semiautomatic rifles were used. Fifty cartridges were fired from each rifle. The cartridge cases were collected, and each firing pin impression was then cast and photographed using a comparison microscope. In this paper, we will describe how one may use a computer vision algorithm, the Histogram of Orientated Gradient (HOG), and a machine learning method, Support Vector Machines (SVMs), to classify images of firing pin impressions. Our method achieved a reasonably high accuracy at 93%. This can be used to associate a firearm with a cartridge case recovered from a scene. We also compared our method with other feature extraction algorithms. The comparison results showed that the HOG-SVM method had the highest performance in this classification task.

Three-Way ROCs for Forensic Decision Making

Firearm examiners use a comparison microscope to judge whether bullets or cartridge cases were fired by the same gun. Examiners can reach one of three possible conclusions: Identification (a match), Elimination (not a match), or Inconclusive. Numerous error rate studies report that firearm examiners commit few errors when they conduct these examinations. However, the studies also report many inconclusive judgments (> 50%), and how to score these responses is controversial. There have recently been three Signal Detection Theory (SDT) primers in this domain. Unfortunately, these analyses rely on hypothetical data and fail to address the inconclusive response issue adequately. This article reports an SDT analysis using data from a large error rate study of practicing firearm examiners. First, we demonstrate the problem of relying on the traditional two-way SDT model, which either drops or combines inconclusive responses; in addition to lacking ecological validity, this approach leads to implausible results. Second, we introduce readers to the three-way SDT model. We demonstrate this approach in the forensic firearms domain. While the three-way approach is statistically complicated, it is well suited to evaluate performance for any forensic domain in which three possible decision categories exist.

Repeatability and reproducibility of comparison decisions by firearms examiners.

In a comprehensive study to assess various aspects of the performance of qualified forensic firearms examiners, volunteer examiners compared both bullets and cartridge cases fired from three different types of firearms. They rendered opinions on each comparison according to the Association of Firearm & Tool Mark Examiners (AFTE) Range of Conclusions, as Identification, Inconclusive (A, B, or C), Elimination, or Unsuitable. In this part of the study, comparison sets used previously to characterize the overall accuracy of examiners were blindly resubmitted to examiners to assess the repeatability (105 examiners; 5700 comparisons of bullets and cartridge cases) and reproducibility (191 examiners of bullets, 193 of cartridge cases; 5790 comparisons) of firearms examinations. Data gathered using the prevailing AFTE Range were also recategorized into two hypothetical scoring systems. Consistently positive differences between observed agreement and expected agreement indicate that the repeatability and reproducibility of examiners exceed chance agreement. When averaged over bullets and cartridge cases, the repeatability of comparison decisions (involving all five levels of the AFTE Range) was 78.3% for known matches and 64.5% for known nonmatches. Similarly averaged reproducibility was 67.3%% for known matches and 36.5% for known nonmatches. For both repeatability and reproducibility, many of the observed disagreements were between a definitive and inconclusive category. Examiner decisions are reliable and trustworthy in the sense that identifications are unlikely when examiners are comparing non-matching items, and eliminations are unlikely when they are comparing matching items.

Investigation of the variability in the tool marks produced by modified blank firing pistols on fired 7.65 mm projectiles

Firearms identification has an important place in forensic ballistic investigations since the weapons are widely used in criminal offences. Firearm examiners resolve many case files, through the use of automatic systems or comparison microscopes. Advanced forensic technologies like BALİSTİKA helps to record and analyze non standard ballistic evidence. In today's world, with the ease of access to materials and production technique information, pistols modified from blank firers are frequently encountered as crime tools. In this study, the characteristics of 7.65 mm fired cartridge cases obtained by controlled shots from blank firing modified pistols were examined, and their detection performances were compared by means of the Balistika system. Although distinctive differences are not expected after successive test shootings, balistically important changes were seen after the use of blank firing modified pistols and the 3D imaging system proved to be useful in observing such differences. The analyses showed that the modifications in weapons lead to variation in the ballistic characteristics and reduce the accuracy of the detection performance, which may result in flawed forensic decisions. It was also found that the deviations in ballistic impressions of modified blank firing pistols were greater than that of standard fabricated and hand-made pistols. This unique study contributed to the forensic sciences literature by focusing on the impact of modified weapons on ballistic characteristics.

The inconclusive category, entropy, and forensic firearm identification

There has been extensive recent discussion of the difficulty in estimating meaningful error rates in forensic firearms examinations, and other areas of pattern evidence. The 2016 President’s Council of Advisors on Science and Technology (PCAST) report was clear in criticizing many forensic disciplines as lacking the types of studies that would provide error rate measurements seen in other scientific fields. However, there is a substantial lack of consensus on the approach to measuring an “error rate” for fields such as forensic firearm examination that include in the conclusion scale the “inconclusive” category, as occurs in the Association of Firearm and Tool Mark Examiners (AFTE) Range of Conclusions and many other such fields. Many authors appear to assume the error rate calculated in the binary decision model is the only appropriate way to report errors, but there have been attempts made to adapt the error rate from the binary decision model to scientific fields in which the inconclusive category is viewed as a meaningful outcome of the examination process. In this study we present three neural networks of differing complexity and performance trained to classify the outlines of ejector marks on cartridge cases fired from different firearm models, as a model system for examining the performance of various metrics of error in systems using the inconclusive category. We also discuss an entropy, or information, based method to assess the similarity of classifications to ground truth that is applicable to range of conclusion scales, even when the inconclusive category is used.

Validity of forensic cartridge-case comparisons

This article presents key findings from a research project that evaluated the validity and probative value of cartridge-case comparisons under field-based conditions. Decisions provided by 228 trained firearm examiners across the US showed that forensic cartridge-case comparison is characterized by low error rates. However, inconclusive decisions constituted over one-fifth of all decisions rendered, complicating evaluation of the technique's ability to yield unambiguously correct decisions. Specifically, restricting evaluation to only the conclusive decisions of identification and elimination yielded true-positive and true-negative rates exceeding 99%, but incorporating inconclusives caused these values to drop to 93.4% and 63.5%, respectively. The asymmetric effect on the two rates occurred because inconclusive decisions were rendered six times more frequently for different-source than same-source comparisons. Considering probative value, which is a decision's usefulness for determining a comparison's ground-truth state, conclusive decisions predicted their corresponding ground-truth states with near perfection. Likelihood ratios (LRs) further showed that conclusive decisions greatly increase the odds of a comparison's ground-truth state matching the ground-truth state asserted by the decision. Inconclusive decisions also possessed probative value, predicting different-source status and having a LR indicating that they increase the odds of different-source status. The study also manipulated comparison difficulty by using two firearm models that produce dissimilar cartridge-case markings. The model chosen for being more difficult received more inconclusive decisions for same-source comparisons, resulting in a lower true-positive rate compared to the less difficult model. Relatedly, inconclusive decisions for the less difficult model exhibited more probative value, being more strongly predictive of different-source status.

Hierarchical Bayesian non-response models for error rates in forensic black-box studies.

Forensic science plays a critical role in the United States criminal legal system. Historically, however, most feature-based fields of forensic science, including firearms examination and latent print analysis, have not been shown to be scientifically valid. Recently, black-box studies have been proposed as a means of assessing whether these feature-based disciplines are valid, at least in terms of accuracy, reproducibility and repeatability. In these studies, forensic examiners frequently either do not respond to every test item or select an answer equivalent to 'don't know'. Current black-box studies do not account for these high levels of missingness in statistical analyses. Unfortunately, the authors of black-box studies typically do not share the data necessary to meaningfully adjust estimates for the high proportion of missing responses. Borrowing from work in the context of small area estimation, we propose the use of hierarchical Bayesian models that do not require auxiliary data to adjust for non-response. Using these models, we offer the first formal exploration of the impact that missingness is playing in error rate estimations reported in black-box studies. We show that error rates currently reported as low as 0.4% could actually be at least 8.4% in models accounting for non-response where inconclusive decisions are counted as correct, and over 28% when inconclusives are counted as missing responses. These proposed models are not the answer to the missingness problem in black-box studies. But with the release of auxiliary information, they can be the foundation for new methodologies to adjust for missingness in error rate estimations. This article is part of the theme issue 'Bayesian inference: challenges, perspectives, and prospects'.

A Re-Analysis of Repeatability and Reproducibility in the Ames-USDOE-FBI Study

Forensic firearms identification, the determination by a trained firearms examiner as to whether or not bullets or cartridges came from a common weapon, has long been a mainstay in the criminal courts. Reliability of forensic firearms identification has been challenged in the general scientific community, and, in response, several studies have been carried out aimed at showing that firearms examination is accurate, that is, has low error rates. Less studied has been the question of consistency, of whether two examinations of the same bullets or cartridge cases come to the same conclusion, carried out by an examiner on separate occasions—intrarater reliability or repeatability—or by two examiners—interrater reliability or reproducibility. One important study, described in a 2020 Report by the Ames Laboratory-USDOE to the Federal Bureau of Investigation, went beyond considerations of accuracy to investigate firearms examination repeatability and reproducibility. The Report’s conclusions were paradoxical. The observed agreement of examiners with themselves or with other examiners appears mediocre. However, the study concluded repeatability and reproducibility are satisfactory, on grounds that the observed agreement exceeds a quantity called the expected agreement. We find that appropriately employing expected agreement as it was intended does not suggest satisfactory repeatability and reproducibility, but the opposite.

Accuracy of comparison decisions by forensic firearms examiners.

This black box study assessed the performance of forensic firearms examiners in the United States. It involved three different types of firearms and 173 volunteers who performed a total of 8640 comparisons of both bullets and cartridge cases. The overall false-positive error rate was estimated as 0.656% and 0.933% for bullets and cartridge cases, respectively, while the rate of false negatives was estimated as 2.87% and 1.87% for bullets and cartridge cases, respectively. The majority of errors were made by a limited number of examiners. Because chi-square tests of independence strongly suggest that error probabilities are not the same for each examiner, these are maximum-likelihood estimates based on the beta-binomial probability model and do not depend on an assumption of equal examiner-specific error rates. Corresponding 95% confidence intervals are (0.305%, 1.42%) and (0.548%, 1.57%) for false positives for bullets and cartridge cases, respectively, and (1.89%, 4.26%) and (1.16%, 2.99%) for false negatives for bullets and cartridge cases, respectively. The results of this study are consistent with prior studies, despite its comprehensive design and challenging specimens.

Examination of the possibility to use Siamese networks for the comparison of firing pin marks.

One of the most discussed issues in forensic firearms identification is the subjectivity of conclusions. The main part of firearms examiners' work is to make a microscopic comparison of the marks on cartridge cases and bullets. In this process, examiners have to decide if the quantity and the quality of the observed characteristics are sufficient for identification. This decision is based on the personal experience of an examiner, so examiners with different backgrounds can come to different conclusions, and this fact presents a problem. Besides, the calculation of the error rate for this type of examination is a debatable issue. Different mathematical and statistical models were proposed, and computer-based algorithms were developed in order to avoid subjectivity and to determine error rates. This article investigates the possibility to use methods of machine learning for the comparison of marks of the firing pin impressions on cartridge cases. In the research, the Siamese network model, which included two similar Convolutional Neural Networks, was prepared and trained. For the training and validation of the model, the database of firing pin impressions was prepared. This database included images of cartridge cases discharged from 300 firearms that came from regular casework and clone images used for data augmentation. The model was trained and examined using the validation part of the database. The metrics, such as accuracy, sensitivity, and specificity were calculated. The results of the research show the possibility of using the Siamese network for building an objective forensic firearms examination system with a known error rate.