Term Frequency-inverse Document Frequency Vectors Research Articles

322 Distributed Vector Representations of Neurosurgical Admission Notes Predict Discharge Disposition

INTRODUCTION: Natural language processing (NLP) aims to extract information from unstructured language input for use in machine learning, without the need for laborious manual review of text to retrieve data of potential interest. An NLP-based model utilizing term frequency-inverse document frequency (TF-IDF) vectoral representations of pre-operative clinical notes has previously been shown to predict discharge disposition following meningioma resection. The TF-IDF vectorization scheme is based on word counts across documents but does not account for word order or semantic information. An alternative algorithm, paragraph vectorization, utilizes an unsupervised neural network to learn document vector representations. METHODS: We analyzed 7,122 admission notes. We trained a neural network to derive paragraph vector representations of notes and trained a classifier to predict home vs non-home discharge disposition based on these representations. Out-of-sample performance was assessed on a held-out testing set. We repeated the analysis using TF-IDF document representations to compare the two approaches. Performance was quantified using area under the receiver operating characteristic curve (ROC-AUC). RESULTS: Model performance increased with vector dimensionality. A 200-dimensional distributed document vectorization was highly predictive of nonhome discharge with an ROC-AUC of 0.82 ± 0.01 (mean ± standard error over 5-fold nested cross-validation). When the dimensionality of the embedding vector space was relatively low (e.g. <100), learned paragraph vectors predicted discharge disposition better than TF-IDF vectors. CONCLUSIONS: Natural language processing of admission notes via neural network-learned distributed vectoral representations is a viable strategy of information extraction from neurosurgical clinical documentation. This embedding scheme overcomes some of the limitations of TF-IDF and encodes more information in lower-dimensional embedding vector spaces. It holds promise as a method of using free text to train machine learning algorithms for prediction of clinically relevant outcomes.

A classification-based approach for integrated service matching and composition in cloud manufacturing

Cloud manufacturing has been acknowledged as a transformative manufacturing paradigm aiming towards producing highly customized products via sharing distributed manufacturing resources and capabilities. One of the pivotal challenges regarding the practical realization of this idea is the process of matching manufacturing resources with personalized service demands. This problem contains two main steps: (1) retrieval of functionally similar services through assessing semantic similarity between resources’ and subtasks’ descriptions and (2) optimal composition of subtasks according to non-functional quality of service (QoS) indexes. However, almost all the research work in the field so far has focused on tackling each of these dimensions individually which barely represents actual conditions of the cloud manufacturing paradigm. To this end, this paper aims towards a novel integrated approach that first, successfully retrieves candidate sets for each corresponding subtask via implementing five classification algorithms and using TF-IDF (term frequency–inverse document frequency) vectors extracted from the manufacturing capability data. Then, optimal composite services are obtained for each scenario by using two well-known metaheuristic algorithms. Results obtained from the experiments have proven the advantages of this method resulting in a more comprehensive and realistic way for dealing with the service composition problems.

How Machine Learning Is Improving U.S. Navy Customer Support

The U.S. Navy is successfully using natural language processing (NLP) and common machine-learning (ML) algorithms to categorize and automatically route plain text support requests at a Navy fleet support center. The algorithms enhance routine IT support tasks with automation and reduce the workload of service desk agents. The ML pipeline works in a five-step process. First, an archive of documents is created from various sources, including standard operating procedure (SOP) memos, frequently asked questions (FAQs), knowledge articles, Wikipedia articles, encyclopedia articles, previously closed support requests, and other relevant documents. Next, a library of words and phrases is generated from the archive. Then, this library is used to vectorize an incoming support request, producing a term frequency inverse document frequency (TF-IDF) vector. Following, the TF-IDF vector is used to compute similarity scores between the support request and the documents in the previously-created archive. Finally, the similarity scores are processed by support vector machine (SVM) classifiers to categorize and route the incoming support request to the correct support provider. This algorithm was deployed at a U.S. Navy customer support center as part of a pilot study, where it decreased the amount of time agents spend on tickets by 35%; the amount of time required to assign tickets by 74%; and the amount of time to close tickets by 60%. Our internal tests show that, with an error rate of 2%, a 35% reduction in ticket volume could be achieved by fully deploying these algorithms.

Comparison of 2 Natural Language Processing Methods for Identification of Bleeding Among Critically Ill Patients

To improve patient safety, health care systems need reliable methods to detect adverse events in large patient populations. Events are often described in clinical notes, rather than structured data, which make them difficult to identify on a large scale. To develop and compare 2 natural language processing methods, a rules-based approach and a machine learning (ML) approach, for identifying bleeding events in clinical notes. This diagnostic study used deidentified notes from the Medical Information Mart for Intensive Care, which spans 2001 to 2012. A training set of 990 notes and a test set of 660 notes were randomly selected. Physicians classified each note as present or absent for a clinically relevant bleeding event during the hospitalization. A bleeding dictionary was developed for the rules-based approach; bleeding mentions were then aggregated to arrive at a classification for each note. Three ML models (support vector machine, extra trees, and convolutional neural network) were developed and trained using the 990-note training set. Another instance of each ML model was also trained on a sample of 450 notes, with equal numbers of bleeding-present and bleeding-absent notes. The notes were represented using term frequency-inverse document frequency vectors and global vectors for word representation. The main outcomes were accuracy, sensitivity, specificity, positive predictive value, and negative predictive value for each model. Following training, the models were tested on the test set and sensitivities were compared using a McNemar test. The 990-note training set represented 769 patients (296 [38.5%] female; mean [SD] age, 67.42 [14.7] years). The 660-note test set represented 527 patients (211 [40.0%] female; mean [SD] age, 67.86 [14.7] years). Bleeding was present in 146 notes (22.1%). The extra trees down-sampled model and rules-based approaches were similarly sensitive (93.8% vs 91.1%; difference, 2.7%; 95% CI, -3.8% to 7.9%; P = .44). The positive predictive value for the extra trees model, however, was 48.6%. The rules-based model had the best performance overall, with 84.6% specificity, 62.7% positive predictive value, and 97.1% negative predictive value. Bleeding is a common complication in health care, and these results demonstrate an automated and scalable detection method. The rules-based natural language processing approach, compared with ML, had the best performance in identifying bleeding, with high sensitivity and negative predictive value.

Biomedical Text Mining for Disease Gene Discovery

Biomedical Text Mining for Disease Gene Discovery

Clustering More than Two Million Biomedical Publications: Comparing the Accuracies of Nine Text-Based Similarity Approaches

BackgroundWe investigate the accuracy of different similarity approaches for clustering over two million biomedical documents. Clustering large sets of text documents is important for a variety of information needs and applications such as collection management and navigation, summary and analysis. The few comparisons of clustering results from different similarity approaches have focused on small literature sets and have given conflicting results. Our study was designed to seek a robust answer to the question of which similarity approach would generate the most coherent clusters of a biomedical literature set of over two million documents.MethodologyWe used a corpus of 2.15 million recent (2004-2008) records from MEDLINE, and generated nine different document-document similarity matrices from information extracted from their bibliographic records, including titles, abstracts and subject headings. The nine approaches were comprised of five different analytical techniques with two data sources. The five analytical techniques are cosine similarity using term frequency-inverse document frequency vectors (tf-idf cosine), latent semantic analysis (LSA), topic modeling, and two Poisson-based language models – BM25 and PMRA (PubMed Related Articles). The two data sources were a) MeSH subject headings, and b) words from titles and abstracts. Each similarity matrix was filtered to keep the top-n highest similarities per document and then clustered using a combination of graph layout and average-link clustering. Cluster results from the nine similarity approaches were compared using (1) within-cluster textual coherence based on the Jensen-Shannon divergence, and (2) two concentration measures based on grant-to-article linkages indexed in MEDLINE.ConclusionsPubMed's own related article approach (PMRA) generated the most coherent and most concentrated cluster solution of the nine text-based similarity approaches tested, followed closely by the BM25 approach using titles and abstracts. Approaches using only MeSH subject headings were not competitive with those based on titles and abstracts.