Inconclusive Decisions Research Articles

How often do fingerprint examiners disagree in routine casework?

During routine casework, fingerprint examiners are required to make decisions pertaining to the sufficiency of friction ridge skin impressions. Prior experimental research has established that differences of opinion between examiners are expected, though it is uncertain if these findings are representative of the decisions made during casework. In this study, 5000 job-cards completed by fingerprint experts of the NSW Police Force were scrutinised to track the differences of opinion that occurred between examiners. Experts recorded 19,491 casework decisions, which resulted in 8964 reported identification and inconclusive determinations. Expert decision making was found to be unanimous in 94.8 % of these determinations; 4.6 % involved one expert-to-expert disagreement; and 0.5 % involved two expert-to-expert disagreements. Nil determinations featured more than two expert-to-expert disagreements. Expert-to-expert disagreements occurred in 3.7 % of all identification and inconclusive casework verification decisions. However, verifying experts were more likely to agree with a prior expert’s identification decision, than a prior expert’s inconclusive decision. The observed expert-to-expert identification disagreement rate was 2.0 %, whereas the observed expert-to-expert inconclusive disagreement rate was 12.5 %. Overall, most casework disagreements arose due to subjective differences concerning the suitability of friction ridge skin information for comparison or sufficiency for identification. Experts were more concordant in their decision-making with other experts than with trainees, and approximately three times more likely to disagree with a prior trainees’ identification or inconclusive decision than a prior experts’ identification or inconclusive decision. We assume these differences reflect trainees’ developing proficiencies in assessing the suitability or sufficiency of friction ridge skin impression information. Differences of opinion in casework are expected, which exposes the subjective nature of fingerprint decision-making. Computer-based quality metric and likelihood ratio tools should be considered for use in casework to guide examiner evaluations and mitigate examiner disagreements.

Comparative Analysis of Rainfall Data Sources in Agro-Ecological Zones of Sri Lanka: Expensive Meteorological Measurements vs Freely Available NASA Data

Rainfall information contributes significantly to weather-related research as it can be directly applied to many theories and models such as climate change, disaster management, environmental assessments. Therefore, the availability of long-term meteorological data and their accuracy are crucial. At the same time the access to these data from different sources have their own drawbacks. For example, meteorological department of Sri Lanka is responsible of collecting rainfall data. However, their data are very expensive, and their data are not available for all desired locations. On the other hand, the freely available satellite data sources have a question of reliability of the data although no missing data are found. Thus, a comparative assessment of rainfall data was carried out from two distinct sources: meteorological department of Sri Lanka and freely available National Aeronautics and Space Administration (NASA) satellite-based data. The objective of this study was to compare and discern the distinctions in rainfall data obtained from these sources for the benefit of future usage of the data in various studies. The study was carried out in different agro-ecological Zones (wet, dry, and intermediate) of Sri Lanka for the period of 1981-2021. The stations were selected randomly to represent the zones. Identified missing values in the meteorological dataset were subsequently addressed using the mean imputation method. Statistical procedures were conducted on the non-normal rainfall data to identify similarities between the sources within individual agro-ecological zones. Results revealed that there is a statistically significant difference at 5% between the sources (NASA and meteorological measurements) in wet and intermediate zones. However, the comparison of the dry zone resulted in inconclusive decisions both at 5% and 1% significance levels. In conclusion, it is statistically advisable to use meteorological rainfall measurements over NASA data for the wet and intermediate zones.
 
 Keywords: Rainfall, Agro-ecological zones, Meteorological data, NASA data, Non-normal data

Do inconclusive forensic decisions disadvantage the innocent?

Two experiments examined the potential for inconclusive forensic decisions to disadvantage the innocent. Both experiments tested the hypothesis that inconclusive decisions produce more incriminating legal judgments than do clearly exculpatory forensic decisions. Experiment 2 also examined whether this hypothesized effect conformed to a confirmation bias, a communication error, or perceptual accuracy. In Experiment 1 (N = 492), a forensic expert testified that physical evidence recovered from a crime scene either matched or did not match a suspect's evidence or produced an inconclusive result. In Experiment 2 (N = 1,002), a forensic expert testified that physical evidence recovered from a crime scene either matched or did not match a suspect's evidence, produced an inconclusive result, or was unsuitable for analysis. A fifth condition omitted the forensic evidence and expert testimony. The inconclusive decision produced less incriminating legal judgments than did the match forensic decision (|d|average = 0.96), more incriminating legal judgments than did the no-match forensic decision (|d|average = 0.62), and equivalent legal judgments to the unsuitable decision (|d|average = 0.12) and to legal judgments made in the absence of forensic evidence (|d|average = 0.07). These results suggest that participants interpreted the inconclusive decision to be forensically neutral, which is consistent with a communication error. The findings provide preliminary support for the idea that inconclusive decisions can put the innocent at risk of wrongful conviction by depriving them of a clearly exculpatory forensic decision. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Inconclusive decisions and error rates in forensic science

In recent years, there has been discussion and controversy relating to the treatment of inconclusive decisions in forensic feature comparison disciplines when considering the reliability of examination methods and results. In this article, we offer a brief review of the various viewpoints and suggestions that have been recently put forth, followed by a solution that we believe addresses the treatment of inconclusive decisions. We consider the issues in the context of method conformance and method performance as two distinct concepts, both of which are necessary for the determination of reliability. Method conformance relates to an assessment of whether the outcome of a method is the result of the analyst's adherence to the procedures that define the method. Method performance reflects the capacity of a method to discriminate between different propositions of interest (e.g., mated and non-mated comparisons). We then discuss implications of these issues for the forensic science community.

Validity of forensic cartridge-case comparisons

This article presents key findings from a research project that evaluated the validity and probative value of cartridge-case comparisons under field-based conditions. Decisions provided by 228 trained firearm examiners across the US showed that forensic cartridge-case comparison is characterized by low error rates. However, inconclusive decisions constituted over one-fifth of all decisions rendered, complicating evaluation of the technique's ability to yield unambiguously correct decisions. Specifically, restricting evaluation to only the conclusive decisions of identification and elimination yielded true-positive and true-negative rates exceeding 99%, but incorporating inconclusives caused these values to drop to 93.4% and 63.5%, respectively. The asymmetric effect on the two rates occurred because inconclusive decisions were rendered six times more frequently for different-source than same-source comparisons. Considering probative value, which is a decision's usefulness for determining a comparison's ground-truth state, conclusive decisions predicted their corresponding ground-truth states with near perfection. Likelihood ratios (LRs) further showed that conclusive decisions greatly increase the odds of a comparison's ground-truth state matching the ground-truth state asserted by the decision. Inconclusive decisions also possessed probative value, predicting different-source status and having a LR indicating that they increase the odds of different-source status. The study also manipulated comparison difficulty by using two firearm models that produce dissimilar cartridge-case markings. The model chosen for being more difficult received more inconclusive decisions for same-source comparisons, resulting in a lower true-positive rate compared to the less difficult model. Relatedly, inconclusive decisions for the less difficult model exhibited more probative value, being more strongly predictive of different-source status.

Hierarchical Bayesian non-response models for error rates in forensic black-box studies.

Forensic science plays a critical role in the United States criminal legal system. Historically, however, most feature-based fields of forensic science, including firearms examination and latent print analysis, have not been shown to be scientifically valid. Recently, black-box studies have been proposed as a means of assessing whether these feature-based disciplines are valid, at least in terms of accuracy, reproducibility and repeatability. In these studies, forensic examiners frequently either do not respond to every test item or select an answer equivalent to 'don't know'. Current black-box studies do not account for these high levels of missingness in statistical analyses. Unfortunately, the authors of black-box studies typically do not share the data necessary to meaningfully adjust estimates for the high proportion of missing responses. Borrowing from work in the context of small area estimation, we propose the use of hierarchical Bayesian models that do not require auxiliary data to adjust for non-response. Using these models, we offer the first formal exploration of the impact that missingness is playing in error rate estimations reported in black-box studies. We show that error rates currently reported as low as 0.4% could actually be at least 8.4% in models accounting for non-response where inconclusive decisions are counted as correct, and over 28% when inconclusives are counted as missing responses. These proposed models are not the answer to the missingness problem in black-box studies. But with the release of auxiliary information, they can be the foundation for new methodologies to adjust for missingness in error rate estimations. This article is part of the theme issue 'Bayesian inference: challenges, perspectives, and prospects'.

Inconclusive conclusions in forensic science: rejoinders to Scurich, Morrison, Sinha and Gutierrez

To the Editor, We thank professors Scurich, Morrison, Sinha and Mr Gutierrez for their thoughtful comments on our article (Arkes and Koehler, 2021). We agree with Scurich (2023) that when an examiner knows that he or she is being tested, the results of such a test are highly suspect. If an examiner can avoid making errors by deeming a comparison to be inconclusive and if inconclusives are never deemed to be indicative of an error, then a ‘strategic’ examiner can inflate accuracy levels by rendering an inconclusive decision for any difficult test. Such a test will not provide an unbiased measure of an examiner’s accuracy. But this is not reason enough to change the way accuracy is measured. In support of a different view on the matter, Scurich (2023) provides an analogy offered by Kaye et al. (2022) in which a student answers ‘I don’t know’ to a true–false test question. If the student should know the answer, then Kaye et al. (2022) say the ‘I don’t know’ answer should be counted as an error. We don’t think that Kaye’s analogy is apt in this context. Teachers are charged with determining whether a student should know the answer to a question. For example, if the topic was covered in the required reading or in a lecture, then the student should know the answer. In a forensic test, one cannot say whether the examiner should know the answer. Dror and Scurich (2020) suggested strategies that might help one determine whether the examiner should know the answer, but in our response to Dror and Scurich (Arkes and Koehler, 2022) we offered reasons why we think those strategies are inadequate.

Signal detection theory fails to account for real-world consequences of inconclusive decisions

Signal detection theory fails to account for real-world consequences of inconclusive decisions

A method for quantifying individual decision thresholds of latent print examiners

In recent years, ‘black box’ studies in forensic science have emerged as the preferred way to provide information about the overall validity of forensic disciplines in practice. These studies provide aggregated error rates over many examiners and comparisons, but errors are not equally likely on all comparisons. Furthermore, inconclusive responses are common and vary across examiners and comparisons, but do not fit neatly into the error rate framework. This work introduces Item Response Theory (IRT) and variants for the forensic setting to account for these two issues. In the IRT framework, participant proficiency and item difficulty are estimated directly from the responses, which accounts for the different subsets of items that participants often answer. By incorporating a decision-tree framework into the model, inconclusive responses are treated as a distinct cognitive process, which allows inter-examiner differences to be estimated directly. The IRT-based model achieves superior predictive performance over standard logistic regression techniques, produces item effects that are consistent with common sense and prior work, and demonstrates that most of the variability among fingerprint examiner decisions occurs at the latent print evaluation stage and as a result of differing tendencies to make inconclusive decisions.

The use and abuse of the elusive construct of inconclusive decisions

The use and abuse of the elusive construct of inconclusive decisions

Influence of governance bundles and directors' social capital on cash holding in foreign cross‐listed firms

AbstractThis paper provides new evidence on how both governance bundles and directors' social capital together can help to determine cash holdings for foreign cross‐listed firms. Using a large cross‐country sample of 1677 publicly listed firms from 32 countries during the period of 2004–2015, we find a positive relationship between governance bundles and cash holdings for foreign cross‐listed firms with higher directors' social capital. We address potential issue of endogeneity. Therefore, our findings are robust to alternative model specifications and instrumentations and alternative measure of social capital. The findings of our study contribute to the inconclusive decision in the academic literature related to cash holdings, governance bundles, and directors' social capital, especially related to the foreign cross‐listed firms. In addition, the findings can assist the stakeholders of foreign cross‐listed firms to understand the intention of the firms' cash holdings and allow policy makers to identify the need of modification for governance structure by controlling the opportunistic behaviour of the firm manager.

Treatment of inconclusives in the AFTE range of conclusions

Abstract In the past decade, and in response to the recommendations set forth by the National Research Council Committee on Identifying the Needs of the Forensic Sciences Community (2009), scientists have conducted several black-box studies that attempt to estimate the error rates of firearm examiners. Most of these studies have resulted in vanishingly small error rates, and at least one of them (D. P. Baldwin, S. J. Bajic, M. Morris, and D. Zamzow. A Study of False-Positive and False-Negative Error Rates in Cartridge Case Comparisons. Technical report, Ames Lab IA, Performing, Fort Belvoir, VA, April 2014.) was cited by the President’s Council of Advisors in Science and Technology (PCAST) during the Obama administration, as an example of a well-designed experiment. What has received little attention, however, is the actual calculation of error rates and in particular, the effect of inconclusive findings on those error estimates. The treatment of inconclusives in the assessment of errors has far-reaching implications in the legal system. Here, we revisit several black-box studies in the area of firearms examination, investigating their treatment of inconclusive results. It is clear that there are stark differences in the rate of inconclusive results in regions with different norms for training and reporting conclusions. More surprisingly, the rate of inconclusive decisions for materials from different sources is notably higher than the rate of inconclusive decisions for same-source materials in some regions. To mitigate the effects of this difference we propose a unifying approach to the calculation of error rates that is directly applicable in forensic laboratories and in legal settings.

Forensic science and the principle of excluded middle: “Inconclusive” decisions and the structure of error rate studies

In a paper published recently in this journal, Dror and Scurich (2020) [20] critically discuss the notions of “inconclusive evidence” (i.e., test items for which it is difficult to render a categorical response) and “inconclusive decisions” (i.e., experts’ conclusions or responses) in the context of forensic science error rate studies. They expose several ways in which the understanding and use of “inconclusives” in current forensic science research and practice can adversely affect the outcomes of error rate studies. A main cause of distortion, according to Dror and Scurich, is what they call “erroneous inconclusive” decisions, in particular the lack of acknowledgment of this type of erroneous conclusion in the computation of error rates. To overcome this complication, Dror and Scurich call for a more explicit monitoring of “inconclusives” using a modified error rate study design. Whilst we agree with several well-argued points raised by the authors, we disagree with their framing of “inconclusive decisions” as potential errors. In this paper, we argue that referring to an “inconclusive decision” as an error is a contradiction in terms, runs counter to an analysis based on decision logic and, hence, is questionable as a concept. We also reiterate that the very term “inconclusive decision” disregards the procedural architecture of the criminal justice system across modern jurisdictions, especially the fact that forensic experts have no decisional rights in the criminal process. These positions do not ignore the possibility that “inconclusives” – if used excessively – do raise problems in forensic expert reporting, in particular limited assertiveness (or, overcautiousness). However, these drawbacks derive from inherent limitations of experts rather than from the seemingly erroneous nature of “inconclusives” that needs to be fixed. More fundamentally, we argue that attempts to score “inconclusives” as errors amount to philosophical claims disguised as forensic methodology. Specifically, these attempts interfere with the metaphysical substrate underpinning empirical research. We point this out on the basis of the law of the excluded middle, i.e. the principle of “no third possibility being given” (tertium non datur).

Ultimate limits of approximate unambiguous discrimination

Quantum hypothesis testing is an important tool for quantum information processing. Two main strategies have been widely adopted: in a minimum error discrimination strategy, the average error probability is minimized; while in an unambiguous discrimination strategy, an inconclusive decision (abstention) is allowed to vanish any possibility of errors when a conclusive result is obtained. In both scenarios, the testing between quantum states are relatively well-understood, for example, the ultimate limits of the performance are established decades ago; however, the testing between quantum channels is less understood. Although the ultimate limit of minimum error discrimination between channels has been explored recently, the corresponding limit of unambiguous discrimination is unknown. In this paper, we formulate an approximate unambiguous discrimination scenario, and derive the ultimate limits of the performance for both states and channels. In particular, in the channel case, our lower bound of the inconclusive probability holds for arbitrary adaptive sensing protocols. For the special class of `teleportation-covariant' channels, the lower bound is achievable with maximum entangled inputs and no adaptive strategy is necessary.

Quality of Laypersons' Assessment of Forensically Relevant Stimuli.

The current study examined the potential for cognitive bias in lay examiners' comparisons of footwear impressions within the technical review process while addressing limitations of previous research. Prior research has found inconsistent results regarding the extent to which cognitive bias may influence forensic comparisons, often asking non-experts to review forensic stimuli above their competency level. Furthermore, past research has largely ignored the potential for cognitive bias during the technical review process. In collaboration with the Miami-Dade Police Department's Forensic Services Bureau, we examined the effects of previous examiners' level of experience and prior knowledge of the previous examiner's decision on the technical review stage of footwear impression stimuli. Before lay examiners were presented with pairs of known match and nonmatch footwear impressions, they were either told that an expert or a novice had previously examined them and determined them to be either a match, nonmatch, or inconclusive (plus a no-information condition). Participants then evaluated each pair of footwear impressions to make their own determinations of match, nonmatch, or inconclusive. Results support the technical review process for all decision types, as known nonmatch stimuli were generally more difficult for lay examiners to assess than known match stimuli. Knowledge of a prior examiner's decision and status was observed only when the prior decision was inconclusive, suggesting the need for inclusion of inconclusive decisions in future research examining cognitive bias in forensic examination.

(Mis)use of scientific measurements in forensic science

Forensic science error rate studies have not given sufficient attention or weight to inconclusive evidence and inconclusive decisions. Inconclusive decisions can be correct decisions, but they can also be incorrect decisions. Errors can occur when inconclusive evidence is determined as an identification or exclusion, or conversely, when same- or different-source evidence is incorrectly determined as inconclusive. We present four common flaws in error rate studies: 1. Not including test items which are more prone to error; 2. Excluding inconclusive decisions from error rate calculations; 3. Counting inconclusive decisions as correct in error rate calculations; and 4. Examiners resorting to more inconclusive decisions during error rate studies than they do in casework. These flaws seriously undermine the credibility and accuracy of error rates reported in studies. To remedy these shortcomings, we present the problems and show the way forward by providing a corrected experimental design that quantifies error rates more accurately.

Do Oil Price Shocks Affect Household Consumption?

The paper addresses whether international oil price change has any impact on consumer spending. The study is conducted using Organisation for Economic Co-operation and Development nations, which have been chosen deliberately based on their economic importance and classifying each into oil importing and exporting countries: Canada, Germany, the UK and the USA. Applying the empirical methodology of the vector autoregressive model, we find evidence that international oil price shocks have a significant impact on consumer spending. The analysis is performed with two sets of specification for oil (‘Oil price change’ and ‘Net oil price increase’) and the main tools used for diagnosis are forecast error variance decomposition and impulse–response functions.The results are strongly significant for Canada and the USA. The results for Germany and the UK are mixed, which leads us to an inconclusive decision about the impact on these countries. However, in general, our empirical work supports the evidence that oil prices have some predictive power in influencing consumption decisions across oil-importing and oil-exporting countries.

Are Inconclusive Decisions in Forensic Science as Deficient as They Are Said to Be?

Many quarters of forensic science use reporting formats such as “identification,” “inconclusive,” and “exclusion.” These types of conclusions express opinions as to whether or not a particular person or object is the source of the material or traces of unknown source that is of interest in a given case. Rendering an “inconclusive” conclusion is sometimes criticized as being inadequate because—supposedly—it does not provide recipients of expert information with helpful directions. In this paper, we critically examine this claim using decision theory. We present and defend the viewpoint according to which deciding to render an “inconclusive” conclusion is, on a formal account, not as inadequate as may commonly be thought. Using elements of decision theory from existing accounts on the topic, we show that inconclusive conclusions can actually be viable alternatives with respect to other types of conclusions, such as “identification.”

"Cannot Decide": The Fine Line Between Appropriate Inconclusive Determinations Versus Unjustifiably Deciding Not To Decide.

Inconclusive decisions, deciding not to decide, are decisions. We present a cognitive model which takes into account that decisions are an outcome of interactions and intersections between the actual data and human cognition. Using this model it is suggested under which circumstances inconclusive decisions are justified and even warranted (reflecting proper caution and meta-cognitive abilities in recognizing limited abilities), and, conversely, under what circumstances inconclusive decisions are unjustifiable and should not be permitted. The model further explores the limitations and problems in using categorical decision-making when the data are actually a continuum. Solutions are suggested within the forensic fingerprinting domain, but they can be applied to other forensic domains, and, with modifications, may also be applied to other expert domains.

Fuzzy bilevel programming with multiple non-cooperative followers: model, algorithm and application

In centralized decision problems, it is not complicated for decision-makers to make modelling technique selections under uncertainty. When a decentralized decision problem is considered, however, choosing appropriate models is no longer easy due to the difficulty in estimating the other decision-makers’ inconclusive decision criteria. These decision criteria may vary with different decision-makers because of their special risk tolerances and management requirements. Considering the general differences among the decision-makers in decentralized systems, we propose a general framework of fuzzy bilevel programming including hybrid models (integrated with different modelling methods in different levels). Specially, we discuss two of these models which may have wide applications in many fields. Furthermore, we apply the proposed two models to formulate a pricing decision problem in a decentralized supply chain with fuzzy coefficients. In order to solve these models, a hybrid intelligent algorithm integrating fuzzy simulation, neural network and particle swarm optimization based on penalty function approach is designed. Some suggestions on the applications of these models are also presented.