Biomedical Citations Research Articles

Dynamics of technology emergence in innovation networks.

To create the next innovative product, participants in science need to understand which existing technologies can be combined, what new science must be discovered, and what new technologies must be invented. Knowledge of these often arrives by means of expert consensus or popularity metrics, masking key information on how intellectual efforts accumulate into technological progress. To address this shortcoming, we first present a method to establish a mathematical link between technological evolution and complex networks: a path of events that narrates innovation bottlenecks. Next, we quantify the position and proximity of documents to these innovation paths. The result is an innovation network that more exhaustively captures deterministic knowledge flows with respect to a marketed innovative product. Our dataset, containing over three million biomedical citations, demonstrates the possibility of quantifying the accumulation, speed, and division of labour in innovation over a sixty-year time horizon. The significance of this study includes the (i) use of a purpose-generated dataset showing causal paths from research to development to product; (ii) analysis of the innovation process as a directed acyclic graph; (iii) comparison between calendar time and network time; (iv) ordering of science funders along technology lifecycles; (v) quantification of innovative activities' importance to an innovative outcome; and (vi) integration of publication, patent, clinical trial, regulatory data to study innovation holistically.

Predicting substantive biomedical citations without full text

Insights from biomedical citation networks can be used to identify promising avenues for accelerating research and its downstream bench-to-bedside translation. Citation analysis generally assumes that each citation documents substantive knowledge transfer that informed the conception, design, or execution of the main experiments. Citations may exist for other reasons. In this paper, we take advantage of late-stage citations added during peer review because these are less likely to represent substantive knowledge flow. Using a large, comprehensive feature set of open access data, we train a predictive model to identify late-stage citations. The model relies only on the title, abstract, and citations to previous articles but not the full-text or future citations patterns, making it suitable for publications as soon as they are released, or those behind a paywall (the vast majority). We find that high prediction scores identify late-stage citations that were likely added during the peer review process as well as those more likely to be rhetorical, such as journal self-citations added during review. Our model conversely gives low prediction scores to early-stage citations and citation classes that are known to represent substantive knowledge transfer. Using this model, we find that US federally funded biomedical research publications represent 30% of the predicted early-stage (and more likely to be substantive) knowledge transfer from basic studies to clinical research, even though these comprise only 10% of the literature. This is a threefold overrepresentation in this important type of knowledge flow.

Improved biomedical word embeddings in the transformer era

Background:Recent natural language processing (NLP) research is dominated by neural network methods that employ word embeddings as basic building blocks. Pre-training with neural methods that capture local and global distributional properties (e.g., skip-gram, GLoVE) using free text corpora is often used to embed both words and concepts. Pre-trained embeddings are typically leveraged in downstream tasks using various neural architectures that are designed to optimize task-specific objectives that might further tune such embeddings. Objective:Despite advances in contextualized language model based embeddings, static word embeddings still form an essential starting point in BioNLP research and applications. They are useful in low resource settings and in lexical semantics studies. Our main goal is to build improved biomedical word embeddings and make them publicly available for downstream applications. Methods:We jointly learn word and concept embeddings by first using the skip-gram method and further fine-tuning them with correlational information manifesting in co-occurring Medical Subject Heading (MeSH) concepts in biomedical citations. This fine-tuning is accomplished with the transformer-based BERT architecture in the two-sentence input mode with a classification objective that captures MeSH pair co-occurrence. We conduct evaluations of these tuned static embeddings using multiple datasets for word relatedness developed by previous efforts. Results:Both in qualitative and quantitative evaluations we demonstrate that our methods produce improved biomedical embeddings in comparison with other static embedding efforts. Without selectively culling concepts and terms (as was pursued by previous efforts), we believe we offer the most exhaustive evaluation of biomedical embeddings to date with clear performance improvements across the board. Conclusion:We repurposed a transformer architecture (typically used to generate dynamic embeddings) to improve static biomedical word embeddings using concept correlations. We provide our code and embeddings for public use for downstream applications and research endeavors: https://github.com/bionlproc/BERT-CRel-Embeddings.

Thesaurus-based word embeddings for automated biomedical literature classification.

The special nature, volume and broadness of biomedical literature pose barriers for automated classification methods. On the other hand, manually indexing is time-consuming, costly and error prone. We argue that current word embedding algorithms can be efficiently used to support the task of biomedical text classification even in a multilabel setting, with many distinct labels. The ontology representation of Medical Subject Headings provides machine-readable labels and specifies the dimensionality of the problem space. Both deep- and shallow network approaches are implemented. Predictions are determined by the similarity between extracted features from contextualized representations of abstracts and headings. The addition of a separate classifier for transfer learning is also proposed and evaluated. Large datasets of biomedical citations are harvested for their metadata and used for training and testing. These automated approaches are still far from entirely substituting human experts, yet they can be useful as a mechanism for validation and recommendation. Dataset balancing, distributed processing and training parallelization in GPUs, all play an important part regarding the effectiveness and performance of proposed methods.

IPDS: A semantic mediator‐based system using Spark for the integration of heterogeneous proteomics data sources

SummaryWith the constant rise of data volumes in many disciplines, various new Big data management systems have emerged to provide scalable tools for efficient data integration, processing, and analysis. In this article, we provide an overview of biomedical data integration systems focusing on ontology‐based semantic systems and Big data technologies based systems such as Apache Spark. We also propose a new semantic data integration system, called Integrated Proteomics Data System (IPDS), which uses a mediator approach. IPDS provides users a unified interface for query processing and data exploration. This system takes advantage of the Apache Spark framework to perform the query transformation and execution needed to question the integrated data sources. We develop a domain ontology that allows the user to formulate its queries in terms defined in the ontology. IPDS is a case study of semantic proteomics data integration linking four data sources UniProt (protein annotation), String (protein‐protein interaction), PDB (protein structure), and Pubmed (biomedical citation).

Biomedical semantic indexing by deep neural network with multi-task learning

BackgroundBiomedical semantic indexing is important for information retrieval and many other research fields in bioinformatics. It annotates biomedical citations with Medical Subject Headings. In face of unbalanced category distribution in the training data, sampling methods are difficult to apply for semantic indexing task.ResultsIn this paper, we present a novel deep serial multi-task learning model. The primary task treats the biomedical semantic indexing as a multi-label text classification issue that considers the relations of the labels. The auxiliary task is a regression task that predicts the MeSH number of the citation and provides hints for the network to make it converge faster. The experimental results on the BioASQ-Task5A open dataset show that our model outperforms the state-of-the-art solution “MTI”, proposed by the US National Library of Medicine. Further, it not only achieves the highest precision among all the solutions in BioASQ-Task5A but also has faster convergence speed compared with some naive deep learning methods.ConclusionsRather than parallel in an ordinary multi-task structure, the tasks in our model are serial and tightly coupled. It can achieve satisfied performance without any handcrafted feature.

Development of an Eastern Mediterranean Region search strategy for biomedical citations indexed in PubMed.

PubMed, a 'barrier-free' bibliographic database covering biomedical and health disciplines, has been successfully used to identify a multitude of indicators that assist in analyzing global trends for biomedical research productivity. The current study represents an original attempt to develop and optimize an Eastern Mediterranean Region (EMR) search strategy in PubMed to pave the way for subsequent descriptive analyses. The refined EMR search strategy contains elaborate syntaxes which facilitate controlling the search process and maintaining a proper balance between sensitivity and precision of the obtained results. Country-specific citation data were manually scanned for false positive publications. Our results indicate that publication productivity increased nearly five-fold in the EMR from 2004 to 2013. Five countries (Islamic Republic of Iran, Egypt, Saudi Arabia, Tunisia and Pakistan; in order of total publications) contributed to 80% of all EMR publications during this period. Each of the remaining 17 EMR countries contributed less than 4%. We believe that the methodology presented in this study can be used in conjunction with other metrics to extract invaluable indicators to describe EMR health research systems.

PubMed’s Native Interface Remains the Best Tool for Systematic Searching of its Biomedical Citations

A Review of:
 Wildgaard, L. E., & Lund, H. (2016). Advancing PubMed? A comparison of third-party PubMed/Medline tools. Library Hi Tech, 34 (4), 669-684. http://dx.doi.org/doi: 10.1108/LHT-06-2016-0066
 
 Abstract
 
 Objective – To compare the functionality of third-party PubMed tools for searching biomedical citations in PubMed, in the specific context of systematic searching.
 
 Design – Comparative analysis of software functionality. 
 
 Setting – Online, freely accessible search software.
 
 Subjects – Sixteen third-party tools for searching and managing the full range of PubMed citations (tools which focused on specific disciplines were not included).
 
 Methods – Tools for analysis were identified in two ways; those discussed in two published articles were used, and a supplementary PubMed search was performed. The initial list of 76 possibilities was assessed for study inclusion on 4 criteria: covering the entire range of PubMed content; being freely available; not limiting to a particular bio-medical discipline; and incorporating online PubMed/MEDLINE content. After assessment, 16 tools were chosen for further analysis (the authors provide a list and description of the tools in their Table I). Each was examined in relation to 11 crucial operational aspects. Result sets were tested against a control (a literature search result set on a particular clinical question which was determined by physicians to yield relevant results, details of which are provided by the authors in an online appendix). 
 
 Main Results – The 11 identified aspects related to tool functionality were examined for each tool selected, with results grouped into three sets of factors: 1) supporting the search (field codes, filters, limits and Boolean operators); 2) managing the search (output, related articles, links to articles, number of results, exporting); and 3) documenting the search (saving the search and search history). In some cases, the tests had to be adjusted to accommodate the tool's specifications. In Table II the authors present a grid with the results of the testing, on each of the 11 aspects, for each tool.
 
 The authors found that with many tools it was not straightforward, if even possible, to filter and limit in order to get more specific result sets. Few tools were effective at suggesting related articles within the tool itself, instead linking the user out to PubMed, and only two tools provided the same number of citation results as the comparison PubMed search. In addition, the display of results often made it difficult to assess result sets; and only two tools provided the option to save searches and see search history. Furthermore, due to unexpected tool limitations, it was not possible to assess the relevance of citation result sets delivered by the third-party tools, as compared with the control PubMed search.
 
 Conclusion – Close analysis of the tools studied indicated that they were not created in order to support systematic searches. They lack support for filtering/limiting, saving or exporting searches, which are central functionalities to the work of performing such searches. While some of the tools studied may still be in the early phases of development, and while several of them, in enhancing PubMed searches in particular ways, may suggest additional profitable strategies for performing a systematic search, not one of them can replace the functionalities of the native PubMed interface. It remains the best tool for searching and managing the full range of PubMed citations, for the purposes of performing systematic searches.

Comparative analysis of the factors associated with citation and media coverage of clinical research

Our objective was to study the relationship between the design and content of randomized clinical trials (RCTs) and the subsequent number of citations in the medical literature and attention in online news and social media. We studied RCTs published during 2014 in five highly cited medical journals. This was a retrospective review focusing on characteristics of the individual trials and measures of citation and lay media attention. Primary outcome measures included citation count and Altmetric® scores (a composite score measuring attention in news, blogs, Twitter®, and Facebook®). Two hundred and forty two RCTs were included in the final analysis. Trial characteristics that were positive predictors of citation count included investigation of Hepatitis C treatment (r = 0.35, p < 0.001), private funding (r = 0.24, p < 0.001), mortality-related endpoint (r = 0.22, p < 0.001), and research setting within the United States (r = 0.13, p < 0.001). The trial characteristic that positively predicted Altmetric score was the population size potentially affected (r = 0.39, p < 0.001). The only negative predictor of citation count was the size of the population potentially affected (r = −0.21, p < 0.001). Negative predictors of the Altmetric score included investigation of Hepatitis C treatment (r = −0.21, p < 0.001) and private funding (r = −0.13, p < 0.001). While correlation magnitudes were weak, the predictors of biomedical literature citation and non-academic media coverage were different. These predictors may affect editorial decisions and, given the rising influence of health journalism, further study is warranted.

Gene Set to Diseases (GS2D): disease enrichment analysis on human gene sets with literature data

Large sets of candidate genes derived from high-throughput biological experiments can be characterized by functional enrichment analysis. The analysis consists of comparing the functions of one gene set against that of a background gene set. Then, functions related to a significant number of genes in the gene set are expected to be relevant. Web tools offering disease enrichment analysis on gene sets are often based on gene-disease associations from manually curated or experimental data that is accurate but does not cover all diseases discussed in the literature. Using associations automatically derived from literature data could be a cost effective method to improve the coverage of diseases for enrichment analysis at comparable levels of accuracy.&#x0D; We have implemented a method named Gene set to Diseases, GS2D, as a web tool performing disease enrichment analysis on human protein coding gene sets. It uses an automatically built dataset of more than 63 thousand gene-disease associations defined as statistically significant co-occurrences of genes and diseases in annotations of biomedical citations from PubMed. The dataset covers more diseases for enrichment analysis than the largest comparable curated database, Comparative Toxicogenomics Database, and its performance compared favourably to similar approaches based on manually curated or experimental data. Graphical and programmatic interfaces are available at http://cbdm.uni-mainz.de/geneset2diseases.

Combining Crowd and Expert Labels Using Decision Theoretic Active Learning

We consider a finite-pool data categorization scenario which requires exhaustively classifying a given set of examples with a limited budget. We adopt a hybrid human-machine approach which blends automatic machine learning with human labeling across a tiered workforce composed of domain experts and crowd workers. To effectively achieve high-accuracy labels over the instances in the pool at minimal cost, we develop a novel approach based on decision-theoretic active learning. On the important task of biomedical citation screening for systematic reviews, results on real data show that our method achieves consistent improvements over baseline strategies. To foster further research by others, we have made our data available online.

Alkemio: association of chemicals with biomedical topics by text and data mining.

The PubMed® database of biomedical citations allows the retrieval of scientific articles studying the function of chemicals in biology and medicine. Mining millions of available citations to search reported associations between chemicals and topics of interest would require substantial human time. We have implemented the Alkemio text mining web tool and SOAP web service to help in this task. The tool uses biomedical articles discussing chemicals (including drugs), predicts their relatedness to the query topic with a naïve Bayesian classifier and ranks all chemicals by P-values computed from random simulations. Benchmarks on seven human pathways showed good retrieval performance (areas under the receiver operating characteristic curves ranged from 73.6 to 94.5%). Comparison with existing tools to retrieve chemicals associated to eight diseases showed the higher precision and recall of Alkemio when considering the top 10 candidate chemicals. Alkemio is a high performing web tool ranking chemicals for any biomedical topics and it is free to non-commercial users. Availability: http://cbdm.mdc-berlin.de/∼medlineranker/cms/alkemio.

Does academic output correlate with better mortality rates in NHS trusts in England?

To establish whether there is an association between academic output and mortality rates for National Health Service (NHS) trusts. Hospital standardized mortality ratios were obtained from Dr Foster hospital report cards. The Medline database of biomedical citations was queried to establish the number of citations credited to each NHS trust and constituent hospitals from 2006 to 2010. Admissions totals for NHS trusts for 2009-2010 were obtained from Hospital Episode Statistics Online. The number of citations per admission was calculated and used as an indicator of academic output as this reflects the workload of the trust. Spearman's rank analysis was performed to identify any correlation between citations per admission and the inverse of four types of mortality rate: high-risk conditions, r = 0.20 (P = 0.01); low-risk conditions, r = -0.06 (P = 0.46); deaths after surgery, r = 0.193 (P = 0.019); and overall mortality, r = 0.291 (P < 0.01). The results of this preliminary study demonstrate a significant correlation between academic output and mortality rates. The correlation coefficients are small, but the findings of this study encourage further debate.

Screening nonrandomized studies for medical systematic reviews: a comparative study of classifiers.

To investigate whether (1) machine learning classifiers can help identify nonrandomized studies eligible for full-text screening by systematic reviewers; (2) classifier performance varies with optimization; and (3) the number of citations to screen can be reduced. We used an open-source, data-mining suite to process and classify biomedical citations that point to mostly nonrandomized studies from 2 systematic reviews. We built training and test sets for citation portions and compared classifier performance by considering the value of indexing, various feature sets, and optimization. We conducted our experiments in 2 phases. The design of phase I with no optimization was: 4 classifiers × 3 feature sets × 3 citation portions. Classifiers included k-nearest neighbor, naïve Bayes, complement naïve Bayes, and evolutionary support vector machine. Feature sets included bag of words, and 2- and 3-term n-grams. Citation portions included titles, titles and abstracts, and full citations with metadata. Phase II with optimization involved a subset of the classifiers, as well as features extracted from full citations, and full citations with overweighted titles. We optimized features and classifier parameters by manually setting information gain thresholds outside of a process for iterative grid optimization with 10-fold cross-validations. We independently tested models on data reserved for that purpose and statistically compared classifier performance on 2 types of feature sets. We estimated the number of citations needed to screen by reviewers during a second pass through a reduced set of citations. In phase I, the evolutionary support vector machine returned the best recall for bag of words extracted from full citations; the best classifier with respect to overall performance was k-nearest neighbor. No classifier attained good enough recall for this task without optimization. In phase II, we boosted performance with optimization for evolutionary support vector machine and complement naïve Bayes classifiers. Generalization performance was better for the latter in the independent tests. For evolutionary support vector machine and complement naïve Bayes classifiers, the initial retrieval set was reduced by 46% and 35%, respectively. Machine learning classifiers can help identify nonrandomized studies eligible for full-text screening by systematic reviewers. Optimization can markedly improve performance of classifiers. However, generalizability varies with the classifier. The number of citations to screen during a second independent pass through the citations can be substantially reduced.

Podiatric Health Needs of Homeless Populations as a Public Health Concern

Foot and ankle health among the homeless is an important public health concern. There are limited studies done thus far on foot and ankle conditions and the podiatric medical needs of homeless populations. A literature review was undertaken to evaluate any studies published about the lower-extremity health needs among the homeless. We did a literature search through PubMed, the US National Library of Medicine's database of biomedical citations and abstracts for relevant publications from 1988 through 2008. We also searched the references cited in the articles found for any studies relevant to podiatric needs for homeless populations. We found three relevant articles that addressed the needs of podiatric care for the homeless. The articles highlighted the community health importance of foot care for homeless populations, especially in helping prevent potentially limb-threatening pathologies. The small number of studies published so far all emphasize the major public health need for podiatric care among homeless populations. More studies are needed to help address this important public health concern.

Effective Navigation of Query Results Based on Concept Hierarchies

Search queries on biomedical databases, such as PubMed, often return a large number of results, only a small subset of which is relevant to the user. Ranking and categorization, which can also be combined, have been proposed to alleviate this information overload problem. Results categorization for biomedical databases is the focus of this work. A natural way to organize biomedical citations is according to their MeSH annotations. MeSH is a comprehensive concept hierarchy used by PubMed. In this paper, we present the BioNav system, a novel search interface that enables the user to navigate large number of query results by organizing them using the MeSH concept hierarchy. First, the query results are organized into a navigation tree. At each node expansion step, BioNav reveals only a small subset of the concept nodes, selected such that the expected user navigation cost is minimized. In contrast, previous works expand the hierarchy in a predefined static manner, without navigation cost modeling. We show that the problem of selecting the best concepts to reveal at each node expansion is NP-complete and propose an efficient heuristic as well as a feasible optimal algorithm for relatively small trees. We show experimentally that BioNav outperforms state-of-the-art categorization systems by up to an order of magnitude, with respect to the user navigation cost. BioNav for the MEDLINE database is available at http://db.cse.buffalo.edu/bionav.

EpiphaNet: An Interactive Tool to Support Biomedical Discoveries

Background. EpiphaNet is an interactive knowledge discovery system which enables researchers to explore visually sets of relations extracted from MEDLINE using a combination of language processing techniques. In this paper, we discuss the theoretical and methodological foundations of the system, and evaluate the utility of the models that underlie it for literature-based discovery. In addition, we present a summary of results drawn from a qualitative analysis of over six hours of interaction with the system by basic medical scientists. The system is able to simulate open and closed discovery, and is shown to generate associations that are both surprising and interesting within the area of expertise of the researchers concerned. EpiphaNet provides an interactive visual representation of associations between concepts, which is derived from distributional statistics drawn from across the spectrum of biomedical citations in MEDLINE. This tool is available online, providing biomedical scientists with the opportunity to identify and explore associations of interest to them.

Semi-automated screening of biomedical citations for systematic reviews

BackgroundSystematic reviews address a specific clinical question by unbiasedly assessing and analyzing the pertinent literature. Citation screening is a time-consuming and critical step in systematic reviews. Typically, reviewers must evaluate thousands of citations to identify articles eligible for a given review. We explore the application of machine learning techniques to semi-automate citation screening, thereby reducing the reviewers' workload.ResultsWe present a novel online classification strategy for citation screening to automatically discriminate "relevant" from "irrelevant" citations. We use an ensemble of Support Vector Machines (SVMs) built over different feature-spaces (e.g., abstract and title text), and trained interactively by the reviewer(s).Semi-automating the citation screening process is difficult because any such strategy must identify all citations eligible for the systematic review. This requirement is made harder still due to class imbalance; there are far fewer "relevant" than "irrelevant" citations for any given systematic review. To address these challenges we employ a custom active-learning strategy developed specifically for imbalanced datasets. Further, we introduce a novel undersampling technique. We provide experimental results over three real-world systematic review datasets, and demonstrate that our algorithm is able to reduce the number of citations that must be screened manually by nearly half in two of these, and by around 40% in the third, without excluding any of the citations eligible for the systematic review.ConclusionsWe have developed a semi-automated citation screening algorithm for systematic reviews that has the potential to substantially reduce the number of citations reviewers have to manually screen, without compromising the quality and comprehensiveness of the review.

A recent advance in the automatic indexing of the biomedical literature

The volume of biomedical literature has experienced explosive growth in recent years. This is reflected in the corresponding increase in the size of MEDLINE ®, the largest bibliographic database of biomedical citations. Indexers at the US National Library of Medicine (NLM) need efficient tools to help them accommodate the ensuing workload. After reviewing issues in the automatic assignment of Medical Subject Headings (MeSH ® terms) to biomedical text, we focus more specifically on the new subheading attachment feature for NLM’s Medical Text Indexer (MTI). Natural Language Processing, statistical, and machine learning methods of producing automatic MeSH main heading/subheading pair recommendations were assessed independently and combined. The best combination achieves 48% precision and 30% recall. After validation by NLM indexers, a suitable combination of the methods presented in this paper was integrated into MTI as a subheading attachment feature producing MeSH indexing recommendations compliant with current state-of-the-art indexing practice.

Reliability and Validity of Current Physical Examination Techniques of the Foot and Ankle

This literature review was undertaken to evaluate the reliability and validity of the orthopedic, neurologic, and vascular examination of the foot and ankle. We searched PubMed-the US National Library of Medicine's database of biomedical citations-and abstracts for relevant publications from 1966 to 2006. We also searched the bibliographies of the retrieved articles. We identified 35 articles to review. For discussion purposes, we used reliability interpretation guidelines proposed by others. For the kappa statistic that calculates reliability for dichotomous (eg, yes or no) measures, reliability was defined as moderate (0.4-0.6), substantial (0.6-0.8), and outstanding (> 0.8). For the intraclass correlation coefficient that calculates reliability for continuous (eg, degrees of motion) measures, reliability was defined as good (> 0.75), moderate (0.5-0.75), and poor (< 0.5). Intraclass correlations, based on the various examinations performed, varied widely. The range was from 0.08 to 0.98, depending on the examination performed. Concurrent and predictive validity ranged from poor to good. Although hundreds of articles exist describing various methods of lower-extremity assessment, few rigorously assess the measurement properties. This information can be used both by the discerning clinician in the art of clinical examination and by the scientist in the measurement properties of reproducibility and validity.