Decision Threshold Research Articles

P-435 Diagnosing ectopic pregnancy using the Bayes theorem, a classic that is good as new: a retrospective cohort study

Abstract Study question What is the accuracy of an algorithm using Bayes' theorem for diagnosing ectopic pregnancy (EP) having hCG, ultrasound, and clinical data in a real cohort? Summary answer The accuracy of the algorithm is 98.9% (95%CI=98.4% to 99.3%) What is known already The Bayes' theorem describes the probability of an event after the input of prior knowledge of conditions that might be related to the event. Ectopic pregnancy has been diagnosed using clinical aspects, transvaginal ultrasound (TVUS) and serial measurement of hCGs Study design, size, duration The Bayes' theorem describes the probability of an event after the input of prior knowledge of conditions that might be related to the event. Ectopic pregnancy has been diagnosed using clinical aspects, transvaginal ultrasound (TVUS) and serial measurement of hCGs. Participants/materials, setting, methods First-trimester patients who attended the Gynaecologic Emergency Unit were included. Clinical signs and symptoms, the presence of risk factors for ectopic pregnancy (EP), ultrsound findings and hCG levels were independent variables. Pre-test probability, based on clinical presentation and risk factors, likelihood ratio (LR) of variable were calculated, yielding a post-test probability (PTP). EP was the main outcome. EP was excluded with PTP <1%, while a PTP ≥ 95% warranted a treatment. Main results and the role of chance The incidence of ectopic pregnancy was 8.5% (212/2495;95%CI=7.4% to 9.6%); 310 patients were excluded because they were submitted to surgery with decision thresholds <95%. The algorithm was applied in 2185 patients. Just one case remained inconclusive after 3 consultations, and it was considered as an error in prediction. The sensitivity, specificity, and accuracy values (95%CI) of the algorithm were 98.9% (96.1% to 99.8%), 98.9% (98.3% to 99.2%), and 98.9% (98.3% to 99.2%), respectively. Limitations, reasons for caution These results were based on a single centre, and β-hCG were measured using the 4th International Standard in Cobas HCG+ β Elecsys 2010 Modular Analytics E170 machine Wider implications of the findings An online and customizable algorithm was developed to be used by different centers Trial registration number not applicable

Evaluation of performance and precision of a radiochemical procedure for quantification of low concentrations of 234U and 238U in water samples

ABSTRACT Uranium isotopes 238U, 234U, and 235U are alpha-particle emitters and can be directly measured by alpha spectrometry. To evaluate the accuracy of the determination of low concentration of U isotopes in water samples, six spiked water samples were analysed by extraction chromatography using uranium selective resin. The separated uranium isotopes were electrodeplated on a disc of stainless steel and were measured by an alpha spectrometer for the determination of radiochemical yield using 232U isotopic tracer. Three certified IAEA reference water samples (IAEA-Cu-2010-03, IAEA-420 and IAEA-421) have been analysed for method validation. The measurement uncertainty sources were identified, and the combined standard uncertainty of the measurement has been determined. Although the activity of uranium isotopes in the certified reference samples was very low, the obtained results were reliable and were within the certified range with 95% confidence level. The characteristic limits (the decision threshold and detection limit) were calculated. The combined uncertainty of the results was mostly attributed to the count rates in the region of interest of the analyte and the tracer in the blank as well as in the sample spectra. Based on the results of the acceptance criteria including Z-score, u-test, trueness and precision, the used radiochemical procedure was found accurate and precise as well.

Algoritmid museaale uurimas: masinõppemudeli Sälli kasutamine objektide säilivuse prognoosimiseks

The primary task of museums is to preserve museum objects in the form of physical objects. Despite its apparent simplicity and comprehensibility, damage to man-made objects – artefacts – is a complex and complicated field. Damage processes are grouped as being physical, chemical, mechanical, and biological. In most cases, different processes work together, damaging the materials and structure of the artefacts. A number of factors, the most important of which are the composition and structure of materials, environmental conditions, and human impacts, affect damage processes. It is very difficult, and in most cases impossible, to take all these factors into account. At the same time, modelling the aging of museum objects is especially important for their successful preservation. Modelling of damage processes makes it possible to assess the extent of such processes (which objects have been damaged and what the degree of damage is), the speed of damage processes, and thereby changes in the number of damaged objects over time, and finally, the effectiveness of possible management measures.
 In this article, we discuss the machine learning model Sälli, which predicts the durability of museum objects. For this purpose, the machine learning model uses data from MuIS (Estonian Museum Information System). The condition of objects is assessed in MuIS with four values: ‘good’, ‘satisfactory’, ‘poor’, and ‘very poor’. Almost 3.7 million condition assessments have been entered into MuIS. The development of a condition prediction model based on these data requires at least pairs of consecutive condition assessments in order to attempt to determine what correlates with the change in condition, whether it be one or another event, or a property (nature, material, age, techniques) of a museum object, or some combination of such factors. There are more than 1.4 million such pairs among the museum objects with several condition assessments. Almost 32,000 of them, or a little over 2%, consist of two different condition assessments, i.e., they indicate a change in condition. According to the data entered in MuIS, almost 30,000 museum objects, i.e., less than one percent of all museum objects, have been subject to a change in condition.
 As data points, we used at least two condition assessments for each museum object, to which we added the characteristics of the respective museum object and other features that help to predict the deterioration of the condition of the museum object. These data included static data related to the museum object: museum, museum collection, nature, material, material group, technology, exhibitability, and dating. As additional information, we used the history of the museum object, i.e., a summary of the events related to the museum object, taking into account only the events that took place during the condition assessment (because we do not have information on the future). The model finds the probability that the condition of the museum object will deteriorate in the next n years. If the probability of deterioration is greater than or equal to a set threshold, the model responds with ‘deterioration’. In finding the optimal decision threshold, we used a 10-year forecast period, i.e., we trained the decision-makers to predict deterioration over the next 10 years.
 The best results were obtained using the decision forest algorithm, which was able to identify 92% of deteriorating museum objects with 50% accuracy. This model was also used to create the Sälli prototype. The task of the Kratt Sälli prototype is to draw the attention of museum staff to museum objects, the condition of which may deteriorate in the next 10 years and the situation of which should therefore be reviewed. For testing, a prototype of the 1,000 highest-risk museum objects from that museum was added to each test museum. To test the usefulness and usability of the machine learning model predictions, we created a simple web application that was tested in pilot museums. We found that the available data have the potential to predict deterioration, but the data still need to be improved and the model trained on them is not yet mature enough.

Evaluation of the probability distribution of radioactivity estimated by inverse problem solution using Monte Carlo Method

In this study, we derived the posterior probability distribution of the total activity estimated by inverse problem solution. The posterior probability distribution was derived by applying the Monte Carlo approach of the GUM Supplement 1 to the model of evaluation. The decision threshold, the detection limit, and the limits of the coverage interval for the results, all of which are defined in ISO 11929–2 as characteristic limits, were also derived.

Incentive motivation improves numerosity discrimination in children and adolescents

We recently showed that incentive motivation improves the precision of the Approximate Number System (ANS) in young adults. To shed light on the development of incentive motivation, the present study investigated whether this effect and its underlying mechanisms may also be observed in younger samples. Specifically, seven-year-old children (n = 23; 12 girls) and 14-year-old adolescents (n = 30; 15 girls) performed a dot comparison task with monetary reward incentives. Both age groups showed higher accuracy in a reward compared to a neutral condition and, similarly, higher processing efficiency as revealed by the drift rate parameter of the EZ-diffusion model. Furthermore, in line with the Incentive Salience Hypothesis, phasic pupil dilations—indicating the activation of the brain’s salience network—were greater in incentivized trials in both age groups. Together these finding suggest that incentive modulation improves numerosity discrimination in children and adolescents by enhancing the perceptual saliency of numerosity information. However, the observed reward anticipation effects were less pronounced in children relative to adolescents. Furthermore, unlike previous findings regarding young adults, the decision thresholds of children and adolescents were not raised by the monetary reward, which may indicate a more protracted development of incentive regulation of response caution than perceptual evidence accumulation.

Time-frequency Analysis of Electroencephalogram Signals in a Perceptual Decision-Making Task of Random Dot Kinematograms

Purpose: Perceptual decision-making is the act of choosing one option from a set of alternatives based on available sensory information. Regarding the serious role of this act in human personal and social lives, the neurophysiological analysis of the brain during this type of decisions is of great interest. In this research, the underlying neural mechanism of these decisions is investigated using a perceptual decision-making Electroencephalogram (EEG) dataset with a perceptual discrimination task.
 Materials and Methods: An online available dataset containing the pre-processed EEG signals of 24 healthy participants during the perceptual decision-making task of Random Dot Kinematograms was used. After a secondary pre-processing stage, clean EEG signal was divided into 1.3-second segments and averaged for Event-Related Potential (ERP) and Event-Related Spectral Perturbation (ERSP) calculations. The task engagement index was also calculated and averaged among all participants.
 Results: According to the results, the amplitude of the N200 component in O1 and O2 channels was larger for correct choices than incorrect ones. Furthermore, in the O2 channel, it was observed that the average alpha power near 200 milli-seconds after stimulus onset was slightly higher in high and low confidence choices than medium confidence choices. The beta band power in the PO2 channel was also higher for correct choices rather than incorrect ones in this interval. Moreover, the results represented that the task engagement index was higher in medium confidence choices, especially in occipital and parieto-occipital channels.
 Conclusion: The larger N200 amplitude and the higher beta power for correct choices, and the lower alpha power for medium confidence choices may be due to more attention of the individuals to the stimuli. This phenomenon can be observed in the task engagement indices as well. This could be because the user expended more effort in medium confidence to bring one of the choices to the decision threshold.

Attention-deficit/hyperactivity disorder (ADHD) in cultural context: Do parents in Hong Kong and the United Kingdom adopt different thresholds when rating symptoms, and if so why?

ObjectivesAttention‐deficit/hyperactivity disorder (ADHD) prevalence is similar across world regions. However, because informants' decision thresholds may vary between regions, these similarities may mask regional variations in actual ADHD behaviours. We tested this by comparing the relationship between informant's ratings and children's measured activity in United Kingdom (UK) and Hong Kong (HK) and then explored whether any national differences in endorsement thresholds discovered are linked to cultural variations in parenting factors.MethodsParents rated the 18 ADHD symptoms in 112 three‐to‐five‐year‐old children stratified for ADHD symptom levels (49 girls and 63 boys; 55 from the UK and 57 from HK) and completed some parenting questionnaires. Children's task‐related activity was measured using actometers.ResultsIn both groups, measured activity was positively correlated with hyperactivity/impulsivity (r = 0.44HK; r = 0.41UK). While HK children were less active than UK children (p < 0.01), HK parents rated their children as more hyperactive/impulsive and inattentive (ps < 0.05). The lower rating threshold indicated by this pattern in HK parents were explained by their higher child‐related stress levels.ConclusionsUK and HK parents operated different ADHD symptom endorsement thresholds. The link between these and child‐related stress may mark a more general role of cultural pressure for child conformity and school achievement in HK.

Machine learning generalizability across healthcare settings: insights from multi-site COVID-19 screening

As patient health information is highly regulated due to privacy concerns, most machine learning (ML)-based healthcare studies are unable to test on external patient cohorts, resulting in a gap between locally reported model performance and cross-site generalizability. Different approaches have been introduced for developing models across multiple clinical sites, however less attention has been given to adopting ready-made models in new settings. We introduce three methods to do this—(1) applying a ready-made model “as-is” (2); readjusting the decision threshold on the model’s output using site-specific data and (3); finetuning the model using site-specific data via transfer learning. Using a case study of COVID-19 diagnosis across four NHS Hospital Trusts, we show that all methods achieve clinically-effective performances (NPV > 0.959), with transfer learning achieving the best results (mean AUROCs between 0.870 and 0.925). Our models demonstrate that site-specific customization improves predictive performance when compared to other ready-made approaches.

Multiantenna Spectrum Sensing for Correlated Signal in Spatially Correlated Noise Environments

The multiantenna spectrum sensing (MSS) problem for correlated signal is addressed in this correspondence. First, the zero-lag and non-zero-lag covariance matrices are established, respectively, to capture the correlation feature of primary signals. Then, two scenarios are considered: 1) the noises at different antennas are spatially white; 2) the noises at different antennas are spatially correlated. Afterward, two variants of the covariance detection method are devised to detect the presence of primary user. Also, we derive the approximate closed-form expressions to calculate the decision thresholds of the proposed two methods. Simulation results are presented to demonstrate the superiority of the proposed two methods over the conventional MSS methods.

A novel framework for online transaction fraud detection system based on deep neural network

Since the introduction of online payment systems, people have started doing online transactions which has also led to the rise of fraudulent transactions causing loss of money to the users and created distrust in the usage of online payment systems. Hence, fraud detection systems are the need of the hour. Among the transactions occurring on daily basis, frauds are less in number as compared to the genuine transactions, so class imbalance naturally exists in fraud detection systems. In this research work, a novel framework for online transaction fraud detection system based on Deep Neural Network (DNN) has been proposed by utilizing algorithm-level method capable to detect frauds from imbalanced data and to maintain the overall performance of the model as well. The proposed system optimizes the decision threshold by utilizing the validation data efficiently for deciding whether an incoming transaction is a Fraud or not. For demonstration of the efficiency of our proposed system, three class imbalanced publicly available datasets have been used. Proposed system has shown better performance than data-level method. The results produced by the proposed fraud detection system have also been compared with existing machine learning techniques-based fraud detection systems. The experimental results show that the deep learning-based fraud detection system is more efficient for detecting frauds from imbalanced datasets having large number of input features as compared to the machine learning models.

QoE-Aware Cross-Layer Adaptation for D2D Video Communication in Cooperative Cognitive Radio Networks

In this article, a cross-layer parameter optimization is considered for H.264 scalable video transmission in cognitive radio enabled device-to-device networks. Adaptive modulation and coding is considered at the physical layer along with video layer aware automatic repeat request (VLA-ARQ) at the link layer. The VLA-ARQ scheme is aware of video characteristics. Scalable video coding (SVC) based video streaming is carried out at the application layer. A novel cognitive radio network frame structure is proposed accommodating cooperative spectrum sensing (CSS), relaying primary user’s (PU) transmission, and secondary users’ (SU) opportunistic transmission to maximize quality of experience (QoE) and to minimize total power consumption. In SVC-based video streaming, successful reception of a particular video layer is dependent upon the successful reception of its higher layers, which influences the eventual QoE at the user end. A set of optimal values of CSS time duration, sensing decision threshold, cooperation, and transmission power control are obtained under the constraints of detection reliability, PU spectrum efficiency, and SU power budget. Extensive simulation results with respect to the system parameters show the efficacy of the proposed system. The numerical results demonstrate that the proposed technique offers more than 14% gain in overall utility with the proposed CSS and cross-layer parameter optimization strategy.

Performance metric curve analysis framework to assess impact of the decision variable threshold, disease prevalence, and dataset variability in two-class classification.

Purpose: The aim of this study is to (1)demonstrate a graphical method and interpretation framework to extend performance evaluation beyond receiver operating characteristic curve analysis and (2)assess the impact of disease prevalence and variability in training and testing sets, particularly when a specific operating point is used. Approach: The proposed performance metric curves (PMCs) simultaneously assess sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV), and the 95% confidence intervals thereof, as a function of the threshold for the decision variable. We investigated the utility of PMCs using six example operating points associated with commonly used methods to select operating points (including the Youden index and maximum mutual information). As an example, we applied PMCs to the task of distinguishing between malignant and benign breast lesions using human-engineered radiomic features extracted from dynamic contrast-enhanced magnetic resonance images. The dataset had 1885 lesions, with the images acquired in 2015 and 2016 serving as the training set (1450 lesions) and those acquired in 2017 as the test set (435 lesions). Our study used this dataset in two ways: (1)the clinical dataset itself and (2)simulated datasets with features based on the clinical set but with five different disease prevalences. The median and 95% CI of the number of type I (false positive) and type II (false negative) errors were determined for each operating point of interest. Results: PMCs from both the clinical and simulated datasets demonstrated that PMCs could support interpretation of the impact of decision threshold choice on type I and type II errors of classification, particularly relevant to prevalence. Conclusion: PMCs allow simultaneous evaluation of the four performance metrics of sensitivity, specificity, PPV, and NPV as a function of the decision threshold. This may create a better understanding of two-class classifier performance in machine learning.

BER performance of nonzero dead time SPAD-based UWOC with signal accumulation over generalized gamma turbulence channels

Underwater wireless optical communication (UWOC) becomes a hot topic in current researches. It suffers from severe absorption, scattering and turbulence, which result in a low reception power even in a short communication distance. To address this challenge, a single-photon avalanche diode (SPAD) based UWOC with signal accumulation method is proposed in this paper. Bit error rate (BER) performance of above method considering nonzero SPAD dead time over generalized gamma turbulence channels is investigated. For this goal, we first derive the expression for the instantaneous BER of above UWOC system. Then we obtain the optimal decision threshold corresponding to the minimum average BER. Based on those obtain expressions, we construct a useful closed form expression for the average BER corresponding to the minimum BER. With those expressions, the effects of the system parameters on the BER performance under turbulence channels are studied. The simulations show that a poor BER performance of 0.0094 for the system without using the signal accumulation under turbulence σ2 = 0.31 can be improved to 7.5 × 10−7 by using 10 accumulation times, which indicates that the signal accumulation can overcome the adverse turbulence effect. Besides, we also find that even with quiet low receive power of 1 photon/bit, the UWOC system can still obtain a good BER performance of smaller than 10−6.

The influence of reward anticipation on conflict control in children and adolescents: Evidences from hierarchical drift-diffusion model and event-related potentials

Reward is deemed a performance reinforcer. The current study investigated how social and monetary reward anticipation affected cognitive control in 39 children, 40 adolescents, and 40 adults. We found that cognitive control performance improved with age in a Simon task, and the reaction time (RT) was modulated by the reward magnitude. The conflict monitoring process (target N2 amplitudes) of adolescents and the attentional control processes (target P3 amplitudes) of adolescents and adults could be adjusted by reward magnitude, suggesting that adolescents were more sensitive to rewards compared to children. Reward magnitudes influenced the neural process of attentional control with larger P3 in congruent trails than that in incongruent trials only in low reward condition. The result of hierarchical drift-diffusion model indicated that children had slower drift rates, higher decision threshold, and longer non-decision time than adolescents and adults. Adolescents had faster drift rates in monetary task than in social task under the high reward condition, and they had faster drift rates under high reward condition than no reward condition only in the monetary task. The correlation analysis further showed that adults’ non-decision time and decision threshold correlated with conflict monitoring process (N2 responses) and attentional control process on conflicts (P3 responses). Adolescents’ drift rates associated with neural process of attentional control. The current study reveals that reward magnitude and reward type can modulate cognitive control process, especially in adolescents.

Development of a multivariable prediction model to identify dairy calves too young to be transported to auction markets in Canada using simple physical examination and body weight

Calves born on Eastern Canadian dairy farms that are not kept in the herds are traditionally sold through auction markets and are raised for meat purposes such as veal calves. Since February 2020, a new Canadian federal regulation has forbidden calves <9 d old to be sold through auction markets. However, in the absence of a real-time birth registry consultation system, it would be of interest to look for predictors that could be associated with age to allow identification of calves too young to be transported. In the current retrospective cross-sectional study, 1,178 calves with a declared birth date (411 calves aged <9 d old; 34.9%) were assessed in 2 large Québec auction sites. Easy-to-record covariates [body weight (BW), breed phenotype, and presence of an umbilical cord remnant] as well as other clinical signs (umbilical swelling, enlargement, umbilical pain, wet umbilicus, skin tent, sunken eyes, ocular and nasal secretion, and hide cleanliness) were assessed. Two logistic regression models using age as a dichotomous dependent variable (<9 d old vs ≥9 d old) were built. The first model (model 1) considered all covariates, which were selected after univariable analyses and a backward stepwise selection process, whereas a more pragmatic model (model 2) only included the 3 easy-to-record variables (i.e., BW, breed, umbilical cord). Both models had similar accuracy to detect calves <9 d old (sensitivity of 38.4 and 37.5%, and specificity of 85.7 and 84.6% for model 1 and 2, respectively). Model 2 was subsequently more specifically studied as it employs a faster and easier assessment. Decision thresholds were tested for their robustness based on misclassification cost term (MCT) analysis with various prevalence of calves <9 d old and various costs of false-negative:false-positive ratio. Despite statistical significance, model accuracy, even if refined with MCT analysis, was limited at the individual level, showing the limits of using physical signs and BW or their combination as a reliable proxy of age. The sensitivity of these models to find calves <9 d old was not to be used for monitoring compliance with the Canadian federal regulation. The relatively high model specificity may help to use this model as a rule-in test (i.e., targeting positive calves for further investigation) rather than a rule-out test (due to its low sensitivity).

A Novel Sequential Three-Way Decision Model for Medical Diagnosis

In the sequential three-way decision model (S3WD), conditional probability and decision threshold pair are two key elements affecting the classification results. The classical model calculates the conditional probability based on the strict equivalence relationship, which limits its application in reality. In addition, little research has studied the relationship between the threshold change and its cause at different granularity levels. To deal with these deficiencies, we propose a novel sequential three-way decision model and apply it to medical diagnosis. Firstly, we propose two methods of calculating conditional probability based on similarity relation, which satisfies the property of symmetry. Then, we construct an S3WD model for a medical information system and use three different kinds of cost functions as the basis for modifying the threshold pair at each level. Subsequently, the rule of the decision threshold pair change is explored. Furthermore, two algorithms used for implementing the proposed S3WD model are introduced. Finally, extensive experiments are carried out to validate the feasibility and effectiveness of the proposed model, and the results show that the model can achieve better classification performance.

Hybrid localization scheme using K‐fold optimization with machine learning in WSN

SummaryNode localization technology can identify and track nodes, making observing data more relevant; for example, information received at the sink node would be useless to the client if node localization data from the sensor region were not included. Localization is described as determining the location of unknown sensor nodes named destination nodes applying the recognized location of anchor nodes based on measurements such as time difference of occurrence, time of occurrence, angle of occurrence, triangulation, and maximum probability. The purpose of node localization is to assign coordinates points to all sensor nodes arbitrarily put in the monitoring region and have an unknown location. Localization of nodes is essential to account for the cause of events that help group sensor querying, routing, and network coverage. In this paper, data transmission among the nodes is done by comparing the received signal strength indicator (RSSI) value with the supervised learning value. If the RSSI value is less than the supervised learning value, the data transmission takes place; else, no transmission. This paper proposes a hybrid localization scheme that effectively uses K‐fold optimization with supervised learning and gives good results for distance error and RSSI/energy efficiency. The proposed scheme can effectively detect the optimal path for data transmission, node localization for the destination, and overall performance enhancement using threshold decision making.

Supplemental Material for Speed-Accuracy Tradeoffs in Decision Making: Perception Shifts and Goal Activation Bias Decision Thresholds

Supplemental Material for Speed-Accuracy Tradeoffs in Decision Making: Perception Shifts and Goal Activation Bias Decision Thresholds

Siting of Petrol Stations and Compliance to Environmental Regulations in Jalingo Metropolis, Taraba State, Nigeria

Environmental regulation is a complex interdisciplinary effort involving ethical principles, political interests, scientific knowledge, and technological capacities, that covers environmental media and protection and conservation of wildlife, fulfilling environmental legislation requirement. Compliance is a substantial element of environmental regulation. This study appraised the siting of petrol stations and compliance to environmental regulations in Jalingo metropolis. Specifically, it assessed the extent of compliance in the siting of petrol stations in Jalingo metropolis and compared the compliance level of the three categories of petrol stations-Nigeria National Petroleum Corporation (NNPC), major oil marketers and the independent oil marketers. The study adopted the survey research design. Data was obtained from twenty (20) petrol stations, two hundred (200) copies of questionnaires using 3-likert scale and published documents. Both inferential and descriptive statistics was used. The mean cut off of 2.5 was taken as threshold for decision, and any item below 2.5 falls below average and considered negative. Chi-square was used for test of hypothesis which was rejected and the alternative hypothesis was accepted. The findings revealed that major marketers complied more than the independent marketers and the NNPC. It further revealed that the environmental regulations followed by a successful implementation and monitoring, will improve petrol stations level of compliance with environmental laws and regulations. The study recommends the need to reassure the petrol station operators that environmental legislation and environmental regulation requirements are not punitive, but guidelines to help conserve the environment.

Decision theory and linear sequential unmasking in forensic fire debris analysis: A proposed workflow

The ASTM E1618-19 standard method for fire debris analysis does not endorse the use of statistical analysis to establish error rates or the use of probabilistic statements about the strength of the evidence. The method requires the analyst to report in categorical terms with the possibility of introducing disclaimers. This report introduces a new workflow that extends the data analysis and interpretation methods of ASTM E1618-19. In the new workflow, analysts are asked to assign evidentiary values to fire debris samples. The assigned values are combined with decision theory to determine an optimal decision threshold for ascribing a sample as positive or negative for ignitable liquid residue in categorical terms. The new workflow incorporates linear sequential unmasking with the data analysis methods of ASTM E1618-19, machine learning likelihood ratios and the use of ignitable liquid and substrate databases. The workflow is applied to 20 known ground truth samples by each of three analysts to generate validated personal receiver operating characteristic (ROC) curves and optimal decision thresholds. The ROC curve provides a direct link between perceived evidentiary value and decision thresholds based on a defined ratio of costs for false positive and false negative decisions. The new workflow is applied to casework-relevant large-scale burn data for which the ground truth is unknown.