False-positive Misclassification Research Articles

The International Standards on Auditing and the Accuracy of Auditors’ Going Concern Disclosures: Evidence for Private Firms in a Low Litigious Environment

ABSTRACT This paper examines the effect of adopting the International Standard on Auditing (ISA) 570 on going concern (GC) disclosures in the audit report in a European environment characterized by low litigation and reputation risk. We exploit a natural experiment in Belgium, where the global GC standard ISA 570 replaced the local GC standard. We examine whether this shift in standards changed the accuracy of auditors’ GC disclosures. Using a sample of 30,339 stressed private companies with audited financial statements for periods ending on or after 15 December 2012 till 14 December 2016, our findings suggest that the adoption of the ISA 570 improves the accuracy of auditors’ GC disclosures. Specifically, we find that the implementation of the ISA 570 is associated with an 8 percent decrease in false positive misclassifications (GC opinion without subsequent bankruptcy), while the rate of false negative misclassifications (bankruptcy without prior GC opinion) remains unchanged.

Analysis of agreement between measures of subjective cognitive impairment and probable dementia in the National Health and Aging Trends Study.

Subjective cognitive impairment (SCI) measures in population-based surveys offer potential for dementia surveillance, yet their validation against established dementia measures is lacking. We assessed agreement between SCI and a validated probable dementia algorithm in a random one-third sample (n=1936) of participants in the 2012 National Health and Aging Trends Study (NHATS). SCI was more prevalent than probable dementia (12.2%vs 8.4%). Agreement between measures was 90.0% and of substantial strength. Misclassification rates were higher among older and less-educated subgroups due to higher prevalence of false-positive misclassification but did not vary by sex or race and ethnicity. SCI sensitivity (63.4%) and specificity (92.5%) against dementia were comparable with similar metrics for the NHATS probable dementia measure against the "gold-standard" Aging, Demographics, and Memory Study-based dementia criteria, implying that population-based surveys may afford cost-effective opportunities for dementia surveillance to assess risk and inform policy. The prevalence of subjective cognitive impairment (SCI) is generally higher than that of a validated measure of probable dementia, particularly within the youngest age group, females, Whites, and persons with a college or higher degree. Percent agreement between SCI and a validated measure of probable dementia was 90.0% and of substantial strength (prevalence- and bias-adjusted kappa, 0.80). Agreement rates were higher in older and less-educated subgroups, driven by the higher prevalence of false-positive disagreement, but did not vary significantly by sex or race and ethnicity. SCI's overall sensitivity and specificity were 63.4% and 92.5%, respectively, against a validated measure of probable dementia, suggesting utility as a low-cost option for dementia surveillance. Heterogeneity in agreement quality across subpopulations warrants caution in its use for subgroup analyses.

Confidence intervals for a proportion using a fixed-inverse double sampling scheme when the data are subject to false-positive misclassification

Of interest in this paper is the development of a model that uses fixed, then inverse sampling of binary data that is subject to false-positive misclassification in an effort to estimate a proportion. From this model, both the proportion of success and false-positive misclassification rate may be estimated. Also, three first-order likelihood-based confidence intervals for the proportion of success are mathematically derived and studied via a Monte Carlo simulation. The simulation results indicate that the likelihood ratio interval is generally preferable over the Wald and score interval. Lastly, the model is applied to two different real-world medical data sets.

A - 183 Novel Performance Validity Indicators for Frequently Used Memory Tests Using Infrequent Patterns of Memory Performance.

Determining performance validity is an imperative part of neuropsychological assessment in concussion management. Several existing embedded performance validity tests (PVTs) have been developed for popular measures of memory, Hopkins Verbal Learning Test Revised (HVLT-R) and Brief Visuospatial Memory Test Revised (BVMT-R). However, many PVTs utilize cognitive performance measures to classify performance invalidity, which increases the likelihood of false positives, especially in clinical populations. To reduce this possibility, a novel embedded PVT was developed utilizing infrequent patterns of performance in a primarily amnestic, dementia population. Neuropsychological data for 50 adult outpatients (Mean Age = 73, Range = 50-97) who met criteria for Major Neurocognitive Disorder (MND) were reviewed. Infrequent patterns of performance (IPOP) were identified on the HVLT-R and BVMT-R, including inconsistency between recall and recognition, unusually high intrusions, exceedingly poor learning, and lack of primary effect. Scaled scores were developed from the IPOP scores (0 = 0-74%, 1 = 75-94%, 2 = 95-99%). Total IPOP scaled scores were compared to existing embedded PVTs, which utilized recognition discrimination and percent retention scores, to examine classification rates. Failure rates on the embedded PVTs for the HVLT-R were 16-42% and BVMT-R 38-58% whereas elevated IPOP scaled scores were identified in less than 15% of the sample. High frequencies of embedded PVT failures were found in a clinical sample of patients with MND, which is concerning for false positive misclassification. Infrequent patterns of memory performances identified in a dementia population were less frequent and could potentially be utilized to examine performance validity in persons suspected of producing invalid memory performances.

A - 184 Validity of the Infrequent Patterns of Performance Embedded Performance Validity Indicators in a Mixed Clinical Sample.

Novel performance validity tests (PVTs) for the Hopkins Verbal Learning Test-Revised (HVLT-R) and Brief Visuospatial Memory Test-Revised (BVMT-R), derived from infrequent patterns of performance (IPOP), were developed to examine invalid memory performance while reducing false positive misclassification. To examine their classification accuracy, these indicators were compared to several existing embedded PVTs in a mixed clinical sample. A chart review was completed for 75 adult outpatients (Mean Age = 52, Range = 19-85) referred to an academic medical center outpatient clinic for neuropsychological evaluation. Twenty-eight percent of the sample failed the Medical Symptom Validity Test (MSVT), which was used as the criterion to indicate invalid performance. Several established embedded validity indicators were calculated from HVLT-R and BVMT-R scores, including the recognition discrimination index (RDI) and percent retention (PR), as well as the Reliable Digit Span (RDS) validity indicator. Classification statistics, including sensitivity, specific, likelihood ratios, predictive values, and overall accuracy, were calculated and compared. The HVLT-R IPOP had a higher overall classification accuracy (72%) compared to the RDI indicator (68%) while maintaining excellent specificity (97%). The BVMT-R IPOP had identical classification accuracy (75%) compared to the RDI and PR indicators with very high specificity (96%). Both indicators were more accurate than RDS (72%). The HVLT-R and BVMT-R IPOP performance validity indicators produced adequate classification statistics when compared to existing embedded PVTs. Both indicators maintained excellent specificity, which is crucial in reducing false positive classification. Future research will focus on further examining the validity of these indicators in other clinical groups and known groups, including medico-legal populations.

An international multi-institutional validation study of the algorithm for prostate cancer detection and Gleason grading

Pathologic examination of prostate biopsies is time consuming due to the large number of slides per case. In this retrospective study, we validate a deep learning-based classifier for prostate cancer (PCA) detection and Gleason grading (AI tool) in biopsy samples. Five external cohorts of patients with multifocal prostate biopsy were analyzed from high-volume pathology institutes. A total of 5922 H&E sections representing 7473 biopsy cores from 423 patient cases (digitized using three scanners) were assessed concerning tumor detection. Two tumor-bearing datasets (core n = 227 and 159) were graded by an international group of pathologists including expert urologic pathologists (n = 11) to validate the Gleason grading classifier. The sensitivity, specificity, and NPV for the detection of tumor-bearing biopsies was in a range of 0.971–1.000, 0.875–0.976, and 0.988–1.000, respectively, across the different test cohorts. In several biopsy slides tumor tissue was correctly detected by the AI tool that was initially missed by pathologists. Most false positive misclassifications represented lesions suspicious for carcinoma or cancer mimickers. The quadratically weighted kappa levels for Gleason grading agreement for single pathologists was 0.62–0.80 (0.77 for AI tool) and 0.64–0.76 (0.72 for AI tool) for the two grading datasets, respectively. In cases where consensus for grading was reached among pathologists, kappa levels for AI tool were 0.903 and 0.855. The PCA detection classifier showed high accuracy for PCA detection in biopsy cases during external validation, independent of the institute and scanner used. High levels of agreement for Gleason grading were indistinguishable between experienced genitourinary pathologists and the AI tool.

Development and Optimization of a Machine-Learning Prediction Model for Acute Desquamation After Breast Radiation Therapy in the Multicenter REQUITE Cohort

PurposeSome patients with breast cancer treated by surgery and radiation therapy experience clinically significant toxicity, which may adversely affect cosmesis and quality of life. There is a paucity of validated clinical prediction models for radiation toxicity. We used machine learning (ML) algorithms to develop and optimise a clinical prediction model for acute breast desquamation after whole breast external beam radiation therapy in the prospective multicenter REQUITE cohort study. Methods and MaterialsUsing demographic and treatment-related features (m = 122) from patients (n = 2058) at 26 centers, we trained 8 ML algorithms with 10-fold cross-validation in a 50:50 random-split data set with class stratification to predict acute breast desquamation. Based on performance in the validation data set, the logistic model tree, random forest, and naïve Bayes models were taken forward to cost-sensitive learning optimisation. ResultsOne hundred and ninety-two patients experienced acute desquamation. Resampling and cost-sensitive learning optimisation facilitated an improvement in classification performance. Based on maximising sensitivity (true positives), the “hero” model was the cost-sensitive random forest algorithm with a false-negative: false-positive misclassification penalty of 90:1 containing m = 114 predictive features. Model sensitivity and specificity were 0.77 and 0.66, respectively, with an area under the curve of 0.77 in the validation cohort. ConclusionsML algorithms with resampling and cost-sensitive learning generated clinically valid prediction models for acute desquamation using patient demographic and treatment features. Further external validation and inclusion of genomic markers in ML prediction models are worthwhile, to identify patients at increased risk of toxicity who may benefit from supportive intervention or even a change in treatment plan.

Improved radiation expression profiling in blood by sequential application of sensitive and specific gene signatures

Purpose. Combinations of expressed genes can discriminate radiation-exposed from normal control blood samples by machine learning based signatures (with 8 to 20% misclassification rates). These signatures can quantify therapeutically-relevant as well as accidental radiation exposures. The prodromal symptoms of Acute Radiation Syndrome (ARS) overlap those present in Influenza and Dengue Fever infections. Surprisingly, these human radiation signatures misclassified gene expression profiles of virally infected samples as false positive exposures. The present study investigates these and other confounders, and then mitigates their impact on signature accuracy. Methods. This study investigated recall by previous and novel radiation signatures independently derived from multiple Gene Expression Omnibus datasets on common and rare non-malignant blood disorders and blood-borne infections (thromboembolism, S. aureus bacteremia, malaria, sickle cell disease, polycythemia vera, and aplastic anemia). Normalized expression levels of signature genes are used as input to machine learning-based classifiers to predict radiation exposure in other hematological conditions. Results. Except for aplastic anemia, these blood-borne disorders modify the normal baseline expression values of genes present in radiation signatures, leading to false-positive misclassification of radiation exposures in 8 to 54% of individuals. Shared changes, predominantly in DNA damage response and apoptosis-related gene transcripts in radiation and confounding hematological conditions, compromise the utility of these signatures for radiation assessment. These confounding conditions (sickle cell disease, thromboembolism, S. aureus bacteremia, malaria) induce neutrophil extracellular traps, initiated by chromatin decondensation, DNA damage response and fragmentation followed by programmed cell death. Riboviral infections (for example, Influenza or Dengue fever) have been proposed to bind and deplete host RNA binding proteins, inducing R-loops in chromatin. R-loops that collide with incoming replication forks can result in incompletely repaired DNA damage, inducing apoptosis and releasing mature virus. To mitigate the effects of confounders, we evaluated predicted radiation-positive samples with novel gene expression signatures derived from radiation-responsive transcripts encoding secreted blood plasma proteins whose expression levels are unperturbed by these conditions. Conclusions. This approach identifies and eliminates misclassified samples with underlying hematological or infectious conditions, leaving only samples with true radiation exposures. Diagnostic accuracy is significantly improved by selecting genes that maximize both sensitivity and specificity in the appropriate tissue using combinations of the best signatures for each of these classes of signatures.

The Key Reason of False Positive Misclassification for Accurate Large-Area Mangrove Classifications

Accurate large-area mangrove classification is a challenging task due to the complexity of mangroves, such as abundant species within the mangrove category, and various appearances resulting from a large latitudinal span and varied habitats. Existing studies have improved mangrove classifications by introducing time series images, constructing new indices sensitive to mangroves, and correcting classifications by empirical constraints and visual inspections. However, false positive misclassifications are still prevalent in current classification results before corrections, and the key reason for false positive misclassification in large-area mangrove classifications is unknown. To address this knowledge gap, a hypothesis that an inadequate classification scheme (i.e., the choice of categories) is the key reason for such false positive misclassification is proposed in this paper. To validate this hypothesis, new categories considering non-mangrove vegetation near water (i.e., within one pixel from water bodies) were introduced, which is inclined to be misclassified as mangroves, into a normally-used standard classification scheme, so as to form a new scheme. In controlled conditions, two experiments were conducted. The first experiment using the same total features to derive direct mangrove classification results in China for the year 2018 on the Google Earth Engine with the standard scheme and the new scheme respectively. The second experiment used the optimal features to balance the probability of a selected feature to be effective for the scheme. A comparison shows that the inclusion of the new categories reduced the false positive pixels with a rate of 71.3% in the first experiment, and a rate of 66.3% in the second experiment. Local characteristics of false positive pixels within 1 × 1 km cells, and direct classification results in two selected subset areas were also analyzed for quantitative and qualitative validation. All the validation results from the two experiments support the finding that the hypothesis is true. The validated hypothesis can be easily applied to other studies to alleviate the prevalence of false positive misclassifications.

Confidence Intervals for a Proportion Using Inverse Sampling when the Data is Subject to False-positive Misclassification

Of interest in this paper is the development of a model that uses inverse sampling of binary data that is subject to false-positive misclassification in an effort to estimate a proportion. From this model, both the proportion of success and false- positive misclassification rate may be estimated. Also, three first-order likelihood- based confidence intervals for the proportion of success are mathematically derived and studied via a Monte Carlo simulation. The simulation results indicate that the score and likelihood ratio intervals are generally preferable over the Wald interval. Lastly, the model is applied to a medical data set.

Application of Machine Learning in Perovskite Solar Cell Crystal Size Distribution Analysis

AbstractThis research automates edge detection for perovskite crystal grains using machine learning (ML). Once the edges of the crystal grains are located, a flood-fill algorithm can be used to find the distribution of crystal grain areas. The ML algorithm uses GNU Octave to run a regularized logistic regression algorithm that classifies each pixel of an input image as part of an edge or not part of an edge. The basic features used for the algorithm are each pixel’s grayscale intensity, its Sobel derivative. Higher order Sobel derivatives, higher degree polynomial terms, and intensities convolved by various kernels were used as additional features to improve the program’s accuracy and true-positive rate. Training data is obtained by using non-ML Canny Edge Detection to annotate the edges an SEM image of a pure perovskite solar cell (PSC). The classifier exhibits an 85.58% accuracy and produces an edge mask that clearly outlines the crystals visually. The ML edge mask exhibits far fewer false-positive mis-classifications for pixels in the middle of the crystals than Canny. However, the ML mask’s edges are fainter, owing to a lower true-positive classification rate. Using more kernels, higher order derivatives, and higher degree polynomial terms all significantly increased the true positive rate of the classifier, leading to thicker edges. This algorithm can greatly accelerate perovskite solar cell research (and potentially any research requiring particle size analysis), automating a process scientists previously had to perform by hand. This will facilitate the search for a solution for the world’s growing demands for renewable energy.

Normal Values for Left Ventricular Mass in Relation to Lean Body Mass in Child and Adolescent Athletes

It has been demonstrated that regular sport activity in children leads to physiological changes in the heart including increased left ventricular (LV) myocardial thickness and mass (LVM). The aim of the study was to establish the first specific normal values of LVM for child and adolescent athletes. Parasternal long-axis, 2D-guided echocardiographic measurements were obtained from a group of 791 Caucasian child athletes (age 5–18 years, 58.7% boys). For the preparation of normative data, LVM-for-lean body mass (LBM) reference curves were constructed using the LMS method. Then, a simple correlation plot was constructed to analyse the concordant and discordant indications of left ventricular hypertrophy (LVH), defined as LVM-for-LBM above the 95th percentile, according to the newly created and previously published normative data on LVM-for-LBM in the general population of children. Reference scatter plots of LVM-for-LBM for boys and girls in the analysed group of children practicing sports were presented, showing mean values of LVM and z-scores. The application to the studied group of reference centiles established for the general population of children would lead to false positive misclassification of increased LVH in 5.8% of the girls and 17.0% of the boys. We present the first specific normative data for LV mass in relation to lean body mass in Caucasian children and adolescents engaged in regular sport activities. The application of specific normative data for LV mass results in fewer false positive findings of left ventricular hypertrophy in this group than that of reference values for general paediatric population.

Bayesian Credible Sets for Odds Ratio of Two Binomial Proportions Using Double Sampling with False-Positive Misclassification

SYNOPTIC ABSTRACTA Bayesian approach is considered for a point and interval estimation for odds ratios of two binomial proportion parameters using double-sampling data subject to false-positive misclassification. For such data, we derive an easy-to-implement, closed-form algorithm for drawing from the posterior distributions. Application of this algorithm is illustrated using data collected about student laboratory grades from a course.

Maximum Likelihood Estimation for a Binomial Parameter Using Double Sampling with one Type of Misclassification

SYNOPTIC ABSTRACTA Maximum Likelihood Estimator (MLE) approach is considered for the estimation of a binomial proportion parameter in doubly sampled data subject to false positive misclassification. We assume that an inexpensive, error-prone device is used on a large main study and an expensive, error-free device is utilized on a smaller substudy. This double sample allows identifiability of all unknown parameters, because by incorporating additional information (data) via double sampling, the dimension of sufficient statistics is greater than or equal to the numbers of parameters; hence, the model becomes identifiable. Additionally, we derive two confidence intervals (CIs): a naïve Wald CI and a modified Wald CI, and we compare the performance of these two CIs in terms of coverage probability and average length, via a Monte Carlo simulation. We then apply the two newly derived estimator and confidence intervals to a real data problem.

Accuracy of Body Mass Index-defined Obesity Status in US Firefighters.

Obesity is a significant problem affecting United States (US) firefighters. While body mass index (BMI) is widely used to diagnose obesity, its use for this occupational group has raised concerns about validity. We examined rates and types of misclassification of BMI-based obesity status compared to body fat percentage (BF%) and waist circumference (WC). Male career firefighters (N = 994) from 20 US departments completed all three body composition assessments. Mean BMI, BF%, and WC were 29 kg/m2, 23%, and 97 cm, respectively. Approximately 33% and 15% of BF%- and WC-defined obese participants were misclassified as non-obese (false negatives) using BMI, while 8% and 9% of non-obese participants defined by BF% and WC standards were identified as obese (false positives) using BMI. When stratified by race/ethnicity, Pacific Islanders showed high rates of false positive misclassification. Precision in obesity classification would be improved by using WC along with BMI to determine firefighters' weight status.

Parent and Child Acceptable Symptom State in Juvenile Idiopathic Arthritis

To explore the parent and child acceptable symptom state in juvenile arthritis (JA-PASS and JA-CASS, respectively) and estimate the JA-PASS and JA-CASS cutoff values for outcome measures. Children with juvenile idiopathic arthritis (JIA) and their parents completed a multi-dimensional questionnaire that included parent-reported and child-reported outcomes and a question about whether they considered the disease state as satisfactory. Additional assessments included demographic data, physician-reported outcomes, and acute-phase reactant levels. Stepwise logistic regression was used to assess contributors to JA-PASS and JA-CASS. Cutoff values of outcome measures that defined JA-PASS and JA-CASS were determined using both 75th percentile and receiver-operating characteristic (ROC) curve methods. Testing procedures included evaluation of discriminative and construct validity of the satisfaction question and assessment of reliability of JA-PASS and JA-PASS cutoffs. Of 584 parents, 385 (65.9%) considered their child in JA-PASS. Of 343 children, 236 (68.8%) considered themselves in JA-CASS. Significant contributors to being in either JA-PASS or JA-CASS were absence of active joints, better rating of overall well-being, and better physical function or health. Cutoff values yielded by 75th percentile and ROC curve methods were similar. Parent, child, and physician global ratings yielded the lowest percentage of false-positive misclassification and the best tradeoff between sensitivity and specificity. The satisfaction question showed good discriminative and construct validity and the JA-PASS and JA-PASS cutoffs were found to be stable over time. The acceptable symptom state is a relevant concept for children with JIA and their parents and constitutes a valid outcome measure that is potentially applicable in routine practice and clinical trials.

Credible sets for risk ratios in over-reported two-sample binomial data using the double-sampling scheme

We consider point and interval estimation for risk ratios based on two independent samples of binomial data subject to false positive misclassification. For such data it is well known that the model is unidentifiable. We consider incorporating training data obtained by using a double-sampling scheme to make the model identifiable. In this identifiable model, we propose a Bayesian method to make statistical inferences. In particular, we derive an easy-to-implement closed-form algorithm for drawing from the posterior distributions. The algorithm is illustrated using a real data example and further examined via Monte Carlo simulation studies.

A Bayesian approach to obtain confidence intervals for binomial proportion in a double sampling scheme subject to false-positive misclassification

The construction of a confidence interval based on a Bayesian approach is considered for the population proportion with double sampled data subject to false-positive misclassification. The Bayesian confidence intervals are compared with a standard frequentist confidence interval in terms of coverage probability and expected width. It is shown that a noninformative Bayes approach provides a relatively simple but effective confidence interval.

Divergence-based confidence intervals in false-positive misclassification model

In this article, we introduce minimum divergence estimators of parameters of a binary response model when data are subject to false-positive misclassification and obtained using a double-sampling plan. Under this set up, the problem of goodness-of-fit is considered and divergence-based confidence intervals (CIs) for a population proportion parameter are derived. A simulation experiment is carried out to compare the coverage probabilities of the new CIs. An application to real data is also given.

Potential for misclassification of mild cognitive impairment: A study of memory scores on the Wechsler Memory Scale-III in healthy older adults

The psychometric criterion of mild cognitive impairment (MCI) generally involves having an unusually low score on memory testing (i.e., -1.5 SDs). However, healthy older adults can obtain low scores, particularly when multiple memory measures are administered. In turn, there is a substantial risk of psychometrically misclassifying MCI in healthy older adults. This study examined the base rates of low memory scores in older adults (55-87 years; n = 550) from the Wechsler Memory Scale-Third Edition (WMS-III; Wechsler, 1997b) standardization sample. The WMS-III consists of four co-normed episodic memory tests (i.e., Logical Memory, Faces, Verbal Paired Associates, and Family Pictures) that yield eight age- and demographically-adjusted standard scores (Auditory Recognition and Working Memory tests not included). When the eight age-adjusted scores were examined simultaneously, 26% of older adults had one or more scores at or below the 5th percentile (i.e., -1.5 SDs). On the eight demographically- adjusted scores, 39% had at least one score at or below the 5th percentile. There was an inverse relationship between intellectual abilities and prevalence of low memory scores, particularly with the age-adjusted WMS-III scores. Understanding the base rates of low scores can reduce the overinterpretation of low memory scores and minimize false-positive misclassification.