Drop In Recall Research Articles

Improving dictionary-based named entity recognition with deep learning.

Dictionary-based named entity recognition (NER) allows terms to be detected in a corpus and normalized to biomedical databases and ontologies. However, adaptation to different entity types requires new high-quality dictionaries and associated lists of blocked names for each type. The latter are so far created by identifying cases that cause many false positives through manual inspection of individual names, a process that scales poorly. In this work, we aim to improve block list s by automatically identifying names to block, based on the context in which they appear. By comparing results of three well-established biomedical NER methods, we generated a dataset of over 12.5 million text spans where the methods agree on the boundaries and type of entity tagged. These were used to generate positive and negative examples of contexts for four entity types (genes, diseases, species, and chemicals), which were used to train a Transformer-based model (BioBERT) to perform entity type classification. Application of the best model (F1-score = 96.7%) allowed us to generate a list of problematic names that should be blocked. Introducing this into our system doubled the size of the previous list of corpus-wide blocked names. In addition, we generated a document-specific list that allows ambiguous names to be blocked in specific documents. These changes boosted text mining precision by ∼5.5% on average, and over 8.5% for chemical and 7.5% for gene names, positively affecting several biological databases utilizing this NER system, like the STRING database, with only a minor drop in recall (0.6%). All resources are available through Zenodo https://doi.org/10.5281/zenodo.11243139 and GitHub https://doi.org/10.5281/zenodo.10289360.

A cardiologist-like computer-aided interpretation framework to improve arrhythmia diagnosis from imbalanced training datasets

Arrhythmias can pose a significant threat to cardiac health, potentially leading to serious consequences such as stroke, heart failure, cardiac arrest, shock, and sudden death. In computer-aided electrocardiogram interpretation systems, the inclusion of certain classes of arrhythmias, which we term "aggressive" or "bullying," can lead to the underdiagnosis of other "vulnerable" classes. To address this issue, a method for arrhythmia diagnosis is proposed in this study. This method combines morphological-characteristic-based waveform clustering with Bayesian theory, drawing inspiration from the diagnostic reasoning of experienced cardiologists. The proposed method achieved optimal performance in macro-recall and macro-precision through hyperparameter optimization, including spliced heartbeats and clusters. In addition, with increasing bullying by aggressive arrhythmias, our model obtained the highest average recall and the lowest average drop in recall on the nine vulnerable arrhythmias. Furthermore, the maximum cluster characteristics were found to be consistent with established arrhythmia diagnostic criteria, lending interpretability to the proposed method.

Effect of CT reconstruction settings on the performance of a deep learning based lung nodule CAD system

PurposeTo study the effect of different reconstruction parameter settings on the performance of a commercially available deep learning based pulmonary nodule CAD system. Materials and methodsWe performed a retrospective analysis of 24 chest CT scans, reconstructed at 16 different reconstruction settings for two different iterative reconstruction algorithms (SAFIRE and ADMIRE) varying in slice thickness, kernel size and iterative reconstruction level strength using a commercially available deep learning pulmonary nodule CAD system. The DL-CAD software was evaluated at 25 different sensitivity threshold settings and nodules detected by the DL-CAD software were matched against a reference standard based on the consensus reading of three radiologists. ResultsA total of 384 CT reconstructions was analysed from 24 patients, resulting in a total of 5786 found nodules. We matched the detected nodules against the reference standard, defined by a team of thoracic radiologists, and showed a gradual drop in recall, and an improvement in precision when the iterative strength levels were increased for a constant kernel size. The optimal DL-CAD threshold setting for use in our clinical workflow was found to be 0.88 with an F2 of 0.73 ± 0.053. ConclusionsThe DL-CAD system behaves differently on IR data than on FBP data, there is a gradual drop in recall, and growth in precision when the iterative strength levels are increased. As a result, caution should be taken when implementing deep learning software in a hospital with multiple CT scanners and different reconstruction protocols. To the best of our knowledge, this is the first study that demonstrates this result from a DL-CAD system on clinical data.

Extracting chemical reactions from text using Snorkel

BackgroundEnzymatic and chemical reactions are key for understanding biological processes in cells. Curated databases of chemical reactions exist but these databases struggle to keep up with the exponential growth of the biomedical literature. Conventional text mining pipelines provide tools to automatically extract entities and relationships from the scientific literature, and partially replace expert curation, but such machine learning frameworks often require a large amount of labeled training data and thus lack scalability for both larger document corpora and new relationship types.ResultsWe developed an application of Snorkel, a weakly supervised learning framework, for extracting chemical reaction relationships from biomedical literature abstracts. For this work, we defined a chemical reaction relationship as the transformation of chemical A to chemical B. We built and evaluated our system on small annotated sets of chemical reaction relationships from two corpora: curated bacteria-related abstracts from the MetaCyc database (MetaCyc_Corpus) and a more general set of abstracts annotated with MeSH (Medical Subject Headings) term Bacteria (Bacteria_Corpus; a superset of MetaCyc_Corpus). For the MetaCyc_Corpus, we obtained 84% precision and 41% recall (55% F1 score). Extending to the more general Bacteria_Corpus decreased precision to 62% with only a four-point drop in recall to 37% (46% F1 score). Overall, the Bacteria_Corpus contained two orders of magnitude more candidate chemical reaction relationships (nine million candidates vs 68,0000 candidates) and had a larger class imbalance (2.5% positives vs 5% positives) as compared to the MetaCyc_Corpus. In total, we extracted 6871 chemical reaction relationships from nine million candidates in the Bacteria_Corpus.ConclusionsWith this work, we built a database of chemical reaction relationships from almost 900,000 scientific abstracts without a large training set of labeled annotations. Further, we showed the generalizability of our initial application built on MetaCyc documents enriched with chemical reactions to a general set of articles related to bacteria.

PointNet and geometric reasoning for detection of grape vines from single frame RGB-D data in outdoor conditions

In this paper we present the usage of PointNet, a deep neural network that consumes raw un-ordered point clouds, for detection of grape vine clusters in outdoor conditions. We investigate the added value of feeding the detection network with both RGB and depth, contradictory to common practice in agricultural robotics of relying on RGB only. A total of 5057 pointclouds (1033 manually annotated and 4024 annotated using geometric reasoning) were collected in a field experiment conducted in outdoor conditions on 9 grape vines and 5 plants. The detection results show overall accuracy of 91% (average class accuracy of 74%, precision 53% recall 48%) for RGBXYZ data and a significant drop in recall for RGB or XYZ data only. These results suggest the usage of depth cameras for vision in agricultural robotics is crucial for crops where the color contrast between the crop and the background is complex. The results also suggest geometric reasoning can be used for increased training set size, a major bottleneck in the development of agricultural vision systems.

ArtiFuse-computational validation of fusion gene detection tools without relying on simulated reads.

Gene fusions are an important class of transcriptional variants that can influence cancer development and can be predicted from RNA sequencing (RNA-seq) data by multiple existing tools. However, the real-world performance of these tools is unclear due to the lack of known positive and negative events, especially with regard to fusion genes in individual samples. Often simulated reads are used, but these cannot account for all technical biases in RNA-seq data generated from real samples. Here, we present ArtiFuse, a novel approach that simulates fusion genes by sequence modification to the genomic reference, and therefore, can be applied to any RNA-seq dataset without the need for any simulated reads. We demonstrate our approach on eight RNA-seq datasets for three fusion gene prediction tools: average recall values peak for all three tools between 0.4 and 0.56 for high-quality and high-coverage datasets. As ArtiFuse affords total control over involved genes and breakpoint position, we also assessed performance with regard to gene-related properties, showing a drop-in recall value for low-expressed genes in high-coverage samples and genes with co-expressed paralogues. Overall tool performance assessed from ArtiFusions is lower compared to previously reported estimates on simulated reads. Due to the use of real RNA-seq datasets, we believe that ArtiFuse provides a more realistic benchmark that can be used to develop more accurate fusion gene prediction tools for application in clinical settings. ArtiFuse is implemented in Python. The source code and documentation are available at https://github.com/TRON-Bioinformatics/ArtiFusion. Supplementary data are available at Bioinformatics online.

Optimizing graph-based patterns to extract biomedical events from the literature.

In BioNLP-ST 2013We participated in the BioNLP 2013 shared tasks on event extraction. Our extraction method is based on the search for an approximate subgraph isomorphism between key context dependencies of events and graphs of input sentences. Our system was able to address both the GENIA (GE) task focusing on 13 molecular biology related event types and the Cancer Genetics (CG) task targeting a challenging group of 40 cancer biology related event types with varying arguments concerning 18 kinds of biological entities. In addition to adapting our system to the two tasks, we also attempted to integrate semantics into the graph matching scheme using a distributional similarity model for more events, and evaluated the event extraction impact of using paths of all possible lengths as key context dependencies beyond using only the shortest paths in our system. We achieved a 46.38% F-score in the CG task (ranking 3rd) and a 48.93% F-score in the GE task (ranking 4th).After BioNLP-ST 2013We explored three ways to further extend our event extraction system in our previously published work: (1) We allow non-essential nodes to be skipped, and incorporated a node skipping penalty into the subgraph distance function of our approximate subgraph matching algorithm. (2) Instead of assigning a unified subgraph distance threshold to all patterns of an event type, we learned a customized threshold for each pattern. (3) We implemented the well-known Empirical Risk Minimization (ERM) principle to optimize the event pattern set by balancing prediction errors on training data against regularization. When evaluated on the official GE task test data, these extensions help to improve the extraction precision from 62% to 65%. However, the overall F-score stays equivalent to the previous performance due to a 1% drop in recall.

The Lingering Effects of Tobacco Control Advertising

In commercial market research, “pulsing” advertising is found to be more cost-effective than placing advertisements continuously. However, the benefits of pulsing policy in social marketing have rarely been tested. This study explores the lingering effect of tobacco control advertising. If the effects of tobacco control advertisements linger beyond the time they are on-air, this would provide a theoretical basis for using pulsing policy in tobacco control campaigns. This study analyzed responses from two independent samples of smokers and recent quitters, where respondents reported whether they were exposed to any tobacco control advertising in the last week. Sample 1 was cross-sectional ( n = 3,106), and Sample 2 followed a self-refreshed panel methodology where 846 participants were interviewed for up to 6 times (3,120 interviews). Both studies used the same survey instrument and were in field concurrently. Eighty percent of Sample 1 reported recent exposure to advertising. This was strikingly similar to the response from Sample 2 recorded at their first interview (81%). However, panel members’ self-reported exposure to campaigns decreased at each subsequent interview (approximately a fortnight apart) and eventually reached 56% at the sixth interview. Building on the premises of priming theory, the first response from each Sample 2 respondent represented his or her intuitive thought—a result of the lingering effect of advertising. The drop in recall over time could be due to respondents self-correcting their responses after being primed in the previous surveys. Our findings suggest the effects of tobacco control campaigns linger and support the use of pulsing policy in tobacco control advertising.

Discrimination of pop‐up conditions according to PC usage situation for dissemination of business information

AbstractRecently, the use of digital documents in companies has been increasing rapidly in number and must be assured. Pop‐up of business information according to the PC usage situation can achieve this goal. Two functions are required for it: One is to correctly discriminate the pop‐up conditions from the content of business‐use applications, and the other is to select suppression keywords from incorrect pop‐up documents. The suppression keywords should improve the true negative rate without a drop in recall. Based on observation of typical phrases and increases of the frequencies of characteristic phrases in the content of business‐use applications, we select suppression keywords according to their frequency in positive and negative cases and their occurrence in feedback documents. Our evaluation using three kinds of business information shows that the proposed method can achieve both a high true negative rate and a high recall, and can correctly discriminate pop‐up conditions. © 2012 Wiley Periodicals, Inc. Electron Comm Jpn, 95(10): 17–26, 2012; Published online in Wiley Online Library (wileyonlinelibrary.com). DOI 10.1002/ecj.11401

Changes in retrograde memory following temporal lobectomy

Extensive retrograde memory deficits have been detected in patients with temporal lobe epilepsy (TLE) and in patients who have undergone temporal lobectomy (TL). To date, no group study has used a prospective longitudinal design. We assessed memory for public and autobiographical facts and events in patients ( n = 15) with focal left- or right-sided TLE before and after TL using a battery of tests that covered the life span. Patients who underwent TL demonstrated material-specific memory decline in retrograde memory. Left TL was associated with a significant drop in recall of famous people’s names. Patients who underwent right TL exhibited a significant decline in the recall of autobiographical events from across the life span. Our study indicates for the first time that TL is associated with a risk of mild, material-specific retrograde memory decline. Verifications with a larger sample size and other measures of remote memory are needed.

The effect of changing from one to two views at incident (subsequent) screens in the NHS breast screening programme in England: impact on cancer detection and recall rates

To assess the effect on cancer detection and recall rates of changing from one to two views for incident (subsequent) screens. Controlled, comparative, observational study of programmes in NHS breast screening programme in England. women aged 50-64 years were screened by the NHSBSP between 1 April 2001 and 31 March 2003. The effect of changing to two-view mammography was a 20% increase in overall incident screen cancer detection rate, with the biggest effect seen for small (<15 mm) invasive cancers. This increased detection rate was achieved with an 11% drop-in recall rate. The introduction of two-view mammography for incident screens has resulted in considerable improvements in overall NHS breast screening performance.

The "memory cliff" beyond span in immediate recall.

Immediate recall of sub-span, span length, and supra-span memoranda was tested in 27 normal nursing students. Random series of digits and consonant letters ranging from 5 to 20 items were presented at the rate of 1 per second. Maximum recall occurred with memoranda of approximately span length, with a sharp drop of recall as span was exceeded. Absolute recall of supra-span memoranda remained at a plateau after the initial drop. The use of a special scoring method and analysis of data by relating the results to each individual's span permitted an accurate representation of the drop in recall as span was exceeded and the constancy in recall with supra-span memoranda. It is concluded that for immediate ordered recall maximum information gain occurs for each individual at his span rather than at an arbitrary memorandum length This study indirectly supports the separation of immediate memory and storage mechanisms.