High False Positive Error Rates Research Articles

Deep Learning for Protein-Protein Interaction Site Prediction.

Protein-protein interactions (PPIs) are central to cellular functions. Experimental methods for predicting PPIs are well developed but are time and resource expensive and suffer from high false-positive error rates at scale. Computational prediction of PPIs is highly desirable for a mechanistic understanding of cellular processes and offers the potential to identify highly selective drug targets. In this chapter, details of developing a deep learning approach to predicting which residues in a protein are involved in forming a PPI-a task known as PPI site prediction-are outlined. The key decisions to be made in defining a supervised machine learning project in this domain are here highlighted. Alternative training regimes for deep learning models to address shortcomings in existing approaches and provide starting points for further research are discussed. This chapter is written to serve as a companion to developing deep learning approaches to protein-protein interaction site prediction, and an introduction to developing geometric deep learning projects operating on protein structure graphs.

Opiate analgesics and testing of post traumatic amnesia in school-aged children

ABSTRACT Objective To assess the validity of the Westmead PTA scale in school-aged children treated with opiate analgesics. Method Twenty-eight hospitalized children without brain injury, aged between 8 and 16 years treated with opiate analgesics for pain relief following surgery were tested on the Westmead PTA scale. Pain and stress levels were also self-reported each day. Results Only 29% (n = 7) of children assessed over four days obtained a maximum score of 12/12 on three consecutive days, thus 71% would have been deemed to have been in PTA when they were not. The percentage of children who obtained a maximum score significantly decreased over consecutive days of assessment, due to an increase in error rate on picture memory items. Self-reported pain and stress ratings were not correlated with PTA scores. Conclusions Opiate analgesia can disrupt performance on the Westmead PTA scale in school-aged children resulting in a high false-positive error rate. It is therefore important to record pain medication schedules and interpret results cautiously when opiate analgesia is used following a TBI. Alteration of the method of administration of the memory items should be researched as this may increase the validity of the scale for children with TBI treated with opiate analgesics.

Use of the Rey 15-Item Test as a performance validity test in an elderly population

ABSTRACTThe objective of this study was to determine the effectiveness of the Rey 15-Item Test in elderly individuals with and without cognitive impairment and to develop new indices to function with this population, if needed. The sample consisted of 185 individuals referred for outpatient neuropsychological evaluation. All were clinically evaluated and administered the Rey 15-Item Test (RFIT) with recognition as part of this procedure. Significant differences were present between those passing and failing the RFIT on referral question; working status; age; and diagnosis of cognitive impairment. Individuals age 60+ failed the test at rates in excess of 35%. Diagnosis also played a role with those with more severe cognitive diagnosis failing at higher rates; even in those with no diagnosis, however, 19% of elderly individuals did not produce a passing score. The extant cutoff scores commonly used with the RFIT produce unacceptably high false positive error rates to be a useful freestanding Performance Validity Test (PVT) with individuals above age 59. The introduction of a new combination score and use of rare scores and patterns of scores improves the psychometric properties of the RFIT when used with elderly adults. Cautious use of PVTs is warranted until they have been validated with aged populations.

Evaluation of a Myopic Normative Database for Analysis of Retinal Nerve Fiber Layer Thickness.

Analysis of retinal nerve fiber layer (RNFL) abnormalities with optical coherence tomography in eyes with high myopia has been complicated by high rates of false-positive errors. An understanding of whether the application of a myopic normative database can improve the specificity for detection of RNFL abnormalities in eyes with high myopia is relevant. To evaluate the diagnostic performance of a myopic normative database for detection of RNFL abnormalities in eyes with high myopia (spherical equivalent, -6.0 diopters [D] or less). In this cross-sectional study, 180 eyes with high myopia (mean [SD] spherical equivalent, -8.0 [1.8] D) from 180 healthy individuals were included in the myopic normative database. Another 46 eyes with high myopia from healthy individuals (mean [SD] spherical equivalent, -8.1 [1.8] D) and 74 eyes from patients with high myopia and glaucoma (mean [SD] spherical equivalent, -8.3 [1.9] D) were included for evaluation of specificity and sensitivity. The 95th and 99th percentiles of the mean and clock-hour circumpapillary RNFL thicknesses and the individual superpixel thicknesses of the RNFL thickness map measured by spectral-domain optical coherence tomography were calculated from the 180 eyes with high myopia. Participants were recruited from January 2, 2013, to December 30, 2015. The following 6 criteria of RNFL abnormalities were examined: (1) mean circumpapillary RNFL thickness below the lower 95th or (2) the lower 99th percentile; (3) one clock-hour or more for RNFL thickness below the lower 95th or (4) the lower 99th percentile; and (5) twenty contiguous superpixels or more of RNFL thickness in the RNFL thickness map below the lower 95th or (6) the lower 99th percentile. Specificities and sensitivities for detection of RNFL abnormalities. Of the 46 healthy eyes and 74 eyes with glaucoma studied (from 39 men and 38 women), the myopic normative database showed a higher specificity (63.0%-100%) than did the built-in normative database of the optical coherence tomography instrument (8.7%-87.0%) for detection of RNFL abnormalities across all the criteria examined (differences in specificities between 13.0% [95% CI, 1.1%-24.9%; P = .01] and 54.3% [95% CI, 37.8%-70.9%; P < .001]) except for the criterion of mean RNFL thickness below the lower 99th percentile, in which both normative databases had the same specificities (100%) but the myopic normative database exhibited a higher sensitivity (71.6% vs 86.5%; difference in sensitivities, 14.9% [95% CI, 4.6%-25.1%; P = .002]). The application of a myopic normative database improved the specificity without compromising the sensitivity compared with the optical coherence tomography instrument's built-in normative database for detection of RNFL abnormalities in eyes with high myopia. Inclusion of myopic normative databases should be considered in optical coherence tomography instruments.

A Prototype for Network Intrusion Detection System using Danger Theory

Network Intrusion Detection System (NIDS) is considered as one of the last defense mechanisms for any organization. NIDS can be broadly classified into two approaches: misuse-based detection and anomaly-based detection. Misuse-based intrusion detection builds a database of the well-defined patterns of the attacks that exploit weaknesses in systems and network protocols, and uses that database to identify the intrusions. Although this approach can detect all the attacks included in the database, it leads to false negative errors where any new attack not included in that database can’t be detected. The other approach is the anomaly-based NIDS which is developed to emulate the Human Immune System (HIS) and overcome the limitation of the misuse-based approach. The anomaly-based detection approach is based on Negative Selection (NS) mechanism. NS is based on building a database of the normal self patterns, and identifying any pattern not included in that database as a non-self pattern and hence the intrusion is detected. Unfortunately, NS concept has also its drawbacks. Although any attack pattern can be detected as a non-self pattern and this leads to low false negative rate, non-self patterns would not necessarily indicate the existence of intrusions. So, NS has a high false positive error rate caused from that assumption. Danger Theory (DT) is a new concept in HIS, which shows that the response mechanism in HIS is more complicated and beyond the simple NS concept. So, is it possible to utilize the DT to minimize the high false positive detection rate of NIDS? This paper answers this question by developing a prototype for NIDS based on DT and evaluating that prototype using DARPA99 Intrusion Detection dataset.

Sex-biased dispersal produces high error rates in mitochondrial distance-based and tree-based species delimitation

Abstract Species delimitation using mitochondrial sequences aims to identify species as morphological expertise and biodiversity both decline. Species delimitation in animals relies completely or in part on 2 criteria: genetic distance and reciprocal monophyly. Using coalescent simulations of populations experiencing continuous and interrupted gene flow we show that these commonly applied criteria incur both high false-positive and high false-negative error rates in species delimitation when dispersal is sex biased, as it is in most mammals. The combination of distance- and topology-based criteria will incur false-positive error rates well above 5% when ancestral effective population sizes are large and when population structure has been in place for 10,000 years or more. This effect persists even with a subtle bias in dispersal of 1 : 4 female : males dispersing in each generation. High false-positive error rates in genetic distances can be overcome by correcting for within-population sequence divergence...

Performance on Standard Indexes of Effort Among Patients with Dementia

Assessment of response validity is an integral part of neuropsychological practice. Although many studies have demonstrated the efficacy of stand alone and embedded effort measures in a variety of medical and compensation-seeking contexts, much less is known about the robustness of these measures in elderly populations, particularly in patients with dementia. Although older adults may be viewed as less likely to intentionally feign symptoms for an external gain, there are a variety of other factors that could result in suboptimal effort, including fatigue, lack of interest or cooperation in the testing process, or failure to fully appreciate the implications of the assessment on treatment care and outcome. The current study examined the clinical utility of several stand alone and embedded effort measures including the Repeatable Battery for the Assessment of Neuropsychological Status Effort Index, Trail-Making Test Ratio, Rey 15-Item Test, and the Test of Memory Malingering in a sample of patients with dementia. Results found that the majority of effort indexes demonstrated unacceptably high false-positive error rates with specificity levels as high as 83%. These findings demonstrate the need for caution in interpreting effort measure performance in dementia samples due to the fact that despite their best effort, many patients with dementia fail effort measures and are at risk for being misclassified.

A novel approach for leveraging co-occurrence to improve the false positive error in signature files

Signature file is a well-studied method in information retrieval for indexing large text databases. Because of the small index size in this method, it is a good candidate for environments where memory is scarce. This small index size, however, comes at the cost of high false positive error rate. In this paper we address the problem of high false positive error rate of signature files by introducing COCA filters, a new variation of Bloom filters which exploits the co-occurrence probability of words in documents to reduce the false positive error. We show experimentally that by using this technique in real document collections we can reduce the false positive error by up to 21 times, for the same index size. It is also shown that in some extreme cases this technique is even able to completely eliminate the false positive error. COCA filters can be considered as a good replacement for Bloom filters wherever the co-occurrence of any two members of the universe is identifiable.

A pilot study of a test for visual recognition memory in adults with moderate to severe intellectual disability

Objective assessment of memory functioning is an important part of evaluation for Dementia of Alzheimer Type (DAT). The revised Picture Recognition Memory Test (r-PRMT) is a test for visual recognition memory to assess memory functioning of persons with intellectual disabilities (ID), specifically targeting moderate to severe ID. A pilot study was performed to investigate whether the r-PRMT could differentiate DAT-related memory decline from pre-existing poor memory functioning of persons with moderate to severe ID. The r-PRMT scores were compared between 26 participants with DAT and moderate to severe ID and 33 controls with similar levels of ID. The results revealed that the controls with DS showed uniformly high scores in contrast to those with DAT on the r-PRMT and the score distributions of two groups were distinctly different with no overlap. On the other hand, the controls with non-DS etiologies scored much lower with a wider score spread, resulting in significant overlap with the score distribution of the participants with DAT. In conclusion, the r-PRMT may be effective in identifying persons with DAT among persons with moderate to severe ID from DS. However, the r-PRMT may result in a high false positive error rate in discriminating those with DAT among persons with moderate to severe ID from non-DS etiologies, if the judgment is based on a single point assessment.

Determinants of the accuracy of rapid diagnostic tests in malaria case management: evidence from low and moderate transmission settings in the East African highlands

BackgroundThe accuracy of malaria diagnosis has received renewed interest in recent years due to changes in treatment policies in favour of relatively high-cost artemisinin-based combination therapies. The use of rapid diagnostic tests (RDTs) based on histidine-rich protein 2 (HRP2) synthesized by Plasmodium falciparum has been widely advocated to save costs and to minimize inappropriate treatment of non-malarial febrile illnesses. HRP2-based RDTs are highly sensitive and stable; however, their specificity is a cause for concern, particularly in areas of intense malaria transmission due to persistence of HRP2 antigens from previous infections.MethodsIn this study, 78,454 clinically diagnosed malaria patients were tested using HRP2-based RDTs over a period of approximately four years in four highland sites in Kenya and Uganda representing hypoendemic to mesoendemic settings. In addition, the utility of the tests was evaluated in comparison with expert microscopy for disease management in 2,241 subjects in two sites with different endemicity levels over four months.ResultsRDT positivity rates varied by season and year, indicating temporal changes in accuracy of clinical diagnosis. Compared to expert microscopy, the sensitivity, specificity, positive predictive value and negative predictive value of the RDTs in a hypoendemic site were 90.0%, 99.9%, 90.0% and 99.9%, respectively. Corresponding measures at a mesoendemic site were 91.0%, 65.0%, 71.6% and 88.1%. Although sensitivities at the two sites were broadly comparable, levels of specificity varied considerably between the sites as well as according to month of test, age of patient, and presence or absence of fever during consultation. Specificity was relatively high in older age groups and increased towards the end of the transmission season, indicating the role played by anti-HRP2 antibodies. Patients with high parasite densities were more likely to test positive with RDTs than those with low density infections.ConclusionRDTs may be effective when used in low endemicity situations, but high false positive error rates may occur in areas with moderately high transmission. Reports on specificity of RDTs and cost-effectiveness analyses on their use should be interpreted with caution as there may be wide variations in these measurements depending upon endemicity, season and the age group of patients studied.

Immune Memory and Gene Library Evolution in the Dynamic Clonal Selection Algorithm

This paper describes two extensions to the original DynamiCS: (1) the deletion of memory detectors that are no longer valid and (2) the simulation of gene library evolution. Firstly, DynamiCS is extended in order to decrease the false positive (FP) error rates caused by memory detectors. The extended DynamiCS eliminates memory detectors when they show a poor degree of self-tolerance to new antigens. This system is tested to determine whether surviving memory detectors no longer cause high FP error rates. The results show a marked decrease in FP errors produced by the system but an increase in the amount of costimulation required. The large amount of costimulation can render the system weak for intrusion detection. The second extension to DynamiCS is proposed to resolve this problem. It employs the use of hypermutation to produce the effect of gene library evolution. This is designed to fine-tune generated memory detectors so that the system obtains higher true positive (TP) detection rates without increasing the amount of co-stimulation. The new extension is tested to determine whether it gains high TP detection rates without increasing the amount of costimulation as the result of gene library evolution. The test results prove that hypermutation leads the progress of gene library evolution and thus produces immature detectors that are more tuned to cover existing non-self antigens.

Using the Wisconsin Card Sorting Test to Detect Malingering: An Analysis of the Specificity of Two Methods in Nonmalingering Normal and Patient Samples

This study sought to examine the specificity of two sets of equations designed to differentiate honest from malingered performance on the WCST. Data were provided by several samples of nonmalingering normal college students and neurological patients. High false positive error rates were observed in most samples. It was argued that these equations performed poorly because they reflect valid WCST response profiles. The importance of learning to identify malingering on standard neuropsychological tests is recognized, and an alternative approach to identifying such patterns of malingered performance is proposed.

Multidimensional DIF Analyses: The Effects of Matching on Unidimensional Subtest Scores

Popular techniques for assessing differential item functioning (DIF) assume that the test under study is unidimensional. When this assumption is tenable, number-correct score is a reasonable matching criterion. When a test is intentionally multidimensional, matching on a single test score does not ensure comparability and may result in inflated error rates. An alternate approach is to match on all relevant traits simultaneously, using a procedure such as logistic regression. In this study, data were generated to simulate two-dimensional tests. The dimensional structure of the tests, the discrimination levels of the items, and the correlation between the traits measured by the test were varied. Standard DIF analyses were conducted using total test score as the matching variable. High false-positive error rates were found. Items were divided into subtests using nonlinear factor analysis and DIF analyses were repeated with subtest scores as the matching criteria. False-positive error rates were reduced for most datasets. The dimensional structure of the test and the discrimination level of the items influenced false-positive rates for both sets of DIEF analyses. The findings suggest that assessing the dimensional structure of a test can be an important first step in DIF analysis. If a dataset is intentionally multidimensional, conditioning on scores reflecting each dimension can enhance the validity of the analyses.

The Clinical Prediction of Dangerousness

Laws committing dangerous sexual offenders indefinitely are legitimate only insofar as "dangerousness" can be predicted and treated. The court has relied on the clinician to identify and measure the crucial variables in violent behavior and assign a probability of occurrence. Yet most critical studies have concluded that not only has predictive accuracy not been demonstrated but also that such accuracy cannot be achieved. We argue that any conclusion at this time is premature. There are data indicating the soundness of clinical as opposed to statistical prediction of dangerous ness. Both approaches show high false positive error rates, but the im provement in accuracy of clinical studies over statistical studies cannot be denied. In examining the success of treatment, the authors present data indi cating that a program of psychological rehabilitation can have a profound effect on the majority of dangerous sexual offenders. The authors conclude that the indefinite commitment of dangerous offenders is legitimate and appropriate, but the policy carries with it heavy responsibilities. Enforcing such an approach means that the clinician must play an important role in his relationship to the court.