Concept Extraction Research Articles

Embedding-based Detection and Extraction of Research Topics from Academic Documents Using Deep Clustering

Abstract Purpose Detection of research fields or topics and understanding the dynamics help the scientific community in their decisions regarding the establishment of scientific fields. This also helps in having a better collaboration with governments and businesses. This study aims to investigate the development of research fields over time, translating it into a topic detection problem. Design/methodology/approach To achieve the objectives, we propose a modified deep clustering method to detect research trends from the abstracts and titles of academic documents. Document embedding approaches are utilized to transform documents into vector-based representations. The proposed method is evaluated by comparing it with a combination of different embedding and clustering approaches and the classical topic modeling algorithms (i.e. LDA) against a benchmark dataset. A case study is also conducted exploring the evolution of Artificial Intelligence (AI) detecting the research topics or sub-fields in related AI publications. Findings Evaluating the performance of the proposed method using clustering performance indicators reflects that our proposed method outperforms similar approaches against the benchmark dataset. Using the proposed method, we also show how the topics have evolved in the period of the recent 30 years, taking advantage of a keyword extraction method for cluster tagging and labeling, demonstrating the context of the topics. Research limitations We noticed that it is not possible to generalize one solution for all downstream tasks. Hence, it is required to fine-tune or optimize the solutions for each task and even datasets. In addition, interpretation of cluster labels can be subjective and vary based on the readers’ opinions. It is also very difficult to evaluate the labeling techniques, rendering the explanation of the clusters further limited. Practical implications As demonstrated in the case study, we show that in a real-world example, how the proposed method would enable the researchers and reviewers of the academic research to detect, summarize, analyze, and visualize research topics from decades of academic documents. This helps the scientific community and all related organizations in fast and effective analysis of the fields, by establishing and explaining the topics. Originality/value In this study, we introduce a modified and tuned deep embedding clustering coupled with Doc2Vec representations for topic extraction. We also use a concept extraction method as a labeling approach in this study. The effectiveness of the method has been evaluated in a case study of AI publications, where we analyze the AI topics during the past three decades.

Fuzzy Semantic Classification of Multi-Domain E-Learning Concept

Scholarly articles are a great source of knowledge. Learning from them like E-learning requires automatic approaches to build concept-maps, learning paths, etc., as these sources are monotonically increasing and are big too. These sources have multi-domain, variety, huge volumes, which are, in fact, Big Data’s characteristics. Thus data from different domains have to be handled together, especially in the E-learning systems. This paper presents a new approach for concept extraction and semantically clustering and classification of these e-learning concepts using fuzzy membership values. Scholarly articles from different domains are taken for our experimental work, and we tested on BBC datasets with 100 documents and 650 documents. Since the number of domains is known, and all concepts are stored, we have done both clustering and classification for testing our fuzzy-based semantic system. We have used logistics regression, Support Vector Machine (SVM) with Linear kernel, Polynomial kernel, Radial Basis Function (RBF) Kernel, Sigmoid kernel to obtain maximum accuracy up to 94% to 96% for all data sets. In clustering, using K-Means, we got precision up to 93%. The system can be used to generate adaptive learning paths, concept map extraction; Big-Data based E-Learning portals.

Harnessing Current Knowledge of DNA N6-Methyladenosine From Model Plants for Non-model Crops.

Epigenetic modifications alter the gene activity and function by causing change in the chromosomal architecture through DNA methylation/demethylation, or histone modifications without causing any change in DNA sequence. In plants, DNA cytosine methylation (5mC) is vital for various pathways such as, gene regulation, transposon suppression, DNA repair, replication, transcription, and recombination. Thanks to recent advances in high throughput sequencing (HTS) technologies for epigenomic “Big Data” generation, accumulated studies have revealed the occurrence of another novel DNA methylation mark, N6-methyladenosine (6mA), which is highly present on gene bodies mainly activates gene expression in model plants such as eudicot Arabidopsis (Arabidopsis thaliana) and monocot rice (Oryza sativa). However, in non-model crops, the occurrence and importance of 6mA remains largely less known, with only limited reports in few species, such as Rosaceae (wild strawberry), and soybean (Glycine max). Given the aforementioned vital roles of 6mA in plants, hereinafter, we summarize the latest advances of DNA 6mA modification, and investigate the historical, known and vital functions of 6mA in plants. We also consider advanced artificial-intelligence biotechnologies that improve extraction and prediction of 6mA concepts. In this Review, we discuss the potential challenges that may hinder exploitation of 6mA, and give future goals of 6mA from model plants to non-model crops.

A Framework for Ontology Based Semantic Search System in Ayurvedic Medicine

India is known for its traditional medicinal system such as Ayurveda, Yoga, Unani, Siddha and Homeopathy. Ayurveda plays a significant role in curing the diseases without any side effects. Medicinal plants or herbs are considered as a major resource in meeting the need of people health care. Information about this medicinal knowledge must be preserved and digitized. There have been a massive number of publications and large number of articles on ayurvedic research in the form of unstructured textual data. Text mining approach is used to provide the solution to handle such voluminous of unstructured data. With the exponential growth of text based data, navigating the relevant information needed is the challenging task. Semantic understanding of document content forms the vital requirement for ensuring the quality of content retrieval. However, the current approaches are finding variation in textual classification in bringing the classification accuracy which may fail to understand the data during classification. Hence, an efficient model is required to search, classify and retrieve the most relevant data. The main objective of this research is to develop an effective and efficient framework and algorithm to search and retrieve the most relevant facts by including the application of ontology-based text mining approach. The current status of research is analyzed and reviewed in the area of semantic web retrieval, ontology-based approaches and various classification technique for building the framework. Text mining with the special emphasis on understanding the semantic meaning of content is achieved by using domain ontology called medicinal plant ontology construction. The challenges in finding the semantically related content for the given query are achieved through semantic web and ontology which enriched the data on web for structured representation thereby providing the strong semantics in knowledge representation. The methodology of information extraction is implemented by using medicinal plant ontology with semantic knowledge representation, an algorithm called OCEC (Ontology based Concept Extraction and Classification) was developed where each term is described semantically by mapping the terms and its related terms in the medicinal plant ontology. The web language called Web Ontology Language (OWL) is used for knowledge representation and is considered as richer semantic description language for describing unstructured and semi-structured content on the web thereby extracting the exact and relevant data and to offer a strong semantic search. To evaluate the performance of the proposed method, less relevant and most relevant documents were collected from online sources and digital libraries. Comparative study has been performed with various classification techniques. The experimental results show that the proposed method out performed. To further prove the efficiency of the model, experiments were conducted by giving different queries and the results are compared with other existing methods. The results show that the content retrieved by the proposed model improves precision and recall results. Keyword : Traditional Medicine, Ayurveda, Ontology, Semantic Web, Web Ontology Language

Enhanced concept-level sentiment analysis system with expanded ontological relations for efficient classification of user reviews

Background/introductionConcept-level sentiment analysis deals with the extraction and classification of concepts and features from user reviews expressed online about products and other entities like political leaders, government policies, and others. The prior studies on concept-level sentiment analysis have used a limited set of linguistic rules for extracting concepts and their associated features. Furthermore, the ontological relations used in the early works for performing concept-level sentiment analysis need enhancement in terms of the extended set of features concepts and ontological relations. MethodsThis work aims at addressing the aforementioned issues and tries to bridge the literature gap by proposing an extended set of linguistic rules for concept-feature pair extraction along with enhanced set ontological relations. Additionally, a supervised a machine learning technique is implemented for performing concept-level sentiment analysis. Results and conclusionsExperimental results depict the effectiveness of the proposed system in terms of improved efficiency (P: 88%, R: 88%, F-score: 88%, and A: 87.5%).

Research on the Practical Design Process of Lady Bags Through Big Data

Based on survey and data analysis, this paper established a separate and hierarchical design process for lady bags. The design process is divided into four parts: survey, concept extraction, separation and hierarchy design and detail improvement. In light of the influence value grade of lady bag design elements, the data of key elements of lady bags were analyzed, and elements related to new product design were extracted to form conceptual elements, and integrated into design experiments at different levels. Then, their usable proportion in design was measured and applied in design to complete the design of new products. Through the experiment, this design process can provide designers with a new design perspective and improves the timeliness and practicability of fashion design.

Intelligent Energy - Cognitive AI to Augment the Human Knowledge

<div class="section abstract"><div class="htmlview paragraph">Andrew Grove (founder CEO Intel) defines strategic inflectionpoints as what happens to a business when a major event alters itsfundamentals. The Covid-19 pandemic is one such historic event thatis changing fundamental business assumptions in the Oil industry.Companies with a hunter-gatherer mindset will ride this wave withthe help of technologies that make their operations lean andefficient. Current developments in AI, specifically aroundCognitive Sciences is one such area that will empower the earlyadopters to a many-fold improvement in engineering and researchproductivity. This paper explores 'how to augment the humanintelligence with insights from engineering literature, leveragingCognitive AI techniques?'. The key challenge of acquiringknowledge from engineering literature (patents, books, journals,articles, papers etc.) is the sheer volume at which it growsannually (100s of millions existing and new papers growing at 40%year-on-year as per IDC). 6 million+ patents are filed every year,of which a mere 20-40% are relevant to any technology domain. It ishumanely impossible to scour, review and take the right decisionsby reading them all. This paper describes key cognitive techniqueslike concept extraction, cause-effect formulation, sentimentanalysis, document summarization, topic modeling, automaticmetadata extraction etc., that conforms to the analyticalmethodology used by E&P researchers worldwide. Such techniquescan also be applied to legacy internal data of energy companies,empowering digitalization and knowledge management initiatives. Cognitive techniques have been around for a while now (firstgeneric patents from IBM in early 1990s) but have never beenapplied to the engineering world, until the 2010s when IHS Markit(under entity Invention Machine Corp.) patented and productizedthis as an engineering research, problem solving and innovationplatform. This paper also explores the efficiency gains and valuesaccrued by the researcher community, while using IHS Markit'sCognitive AI platform.</div></div>

Health-adjusted life expectancy (HALE) in Chongqing, China, 2017: An artificial intelligence and big data method estimating the burden of disease at city level

BackgroundA universally applicable approach that provides standard HALE measurements for different regions has yet to be developed because of the difficulties of health information collection. In this study, we developed a natural language processing (NLP) based HALE estimation approach by using individual-level electronic medical records (EMRs), which made it possible to calculate HALE timely in different temporal or spatial granularities. MethodsWe performed diagnostic concept extraction and normalisation on 13•99 million EMRs with NLP to estimate the prevalence of 254 diseases in WHO Global Burden of Disease Study (GBD). Then, we calculated HALE in Chongqing, 2017, by using the life table technique and Sullivan's method, and analysed the contribution of diseases to the expected years “lost” due to disability (DLE). FindingsOur method identified a life expectancy at birth (LE0) of 77•9 years and health-adjusted life expectancy at birth (HALE0) of 71•7 years for the general Chongqing population of 2017. In particular, the male LE0 and HALE0 were 76•3 years and 68•9 years, respectively, while the female LE0 and HALE0 were 80•0 years and 74•4 years, respectively. Cerebrovascular diseases, cancers, and injuries were the top three deterioration factors, which reduced HALE by 2•67, 2•15, and 1•19 years, respectively. InterpretationThe results demonstrated the feasibility and effectiveness of EMRs-based HALE estimation. Moreover, the method allowed for a potentially transferable framework that facilitated a more convenient comparison of cross-sectional and longitudinal studies on HALE between regions. In summary, this study provided insightful solutions to the global ageing and health problems that the world is facing. FundingNational Key R and D Program of China (2018YFC2000400).

An open computing language‐based parallel Brute Force algorithm for formal concept analysis on heterogeneous architectures

AbstractAlgorithms for the extraction of formal concepts are widely studied in several areas of knowledge, such as finance, health, and statistics. However, these algorithms require high‐performance processing due to their combinatorial characteristics. In this work, an Open computing language (OpenCL)‐based Brute Force algorithm is proposed and evaluated for formal concept extraction on heterogeneous architectures (CPU+GPU and CPU+FPGA). The CPU+GPU architecture presents higher performance and scalability than other architectures when our Brute Force algorithm processes high dimensional contexts with many objects and attributes. Our parallel approach shows performance results up to 18× better than a smarter sequential algorithm called Data‐Peeler. Moreover, our Brute Force algorithm running on CPU+GPU architecture has greater energy efficiency, reaching at least 1.79× more operations per energy consumption than other algorithms on different architectures explored in this work.

An Approach Based on Patterns for Synonymy Relations Detection

Thesaurus, taxonomies, and ontologies are created as a way to represent knowledge. In recent years, the efforts to create them are focused on automatic knowledge extraction. For this purpose, research in the extraction of concepts and semantic relations is needed. Concepts or entities extracted from unstructured data has a became a challenge for the research communities. Texts, specifically, scientific papers contain knowledge where experts express their contributions to the science, including: techniques, methods and claims. In order to describe them, experts use synonyms, which are relevant to consider whereas it is desirable to extract such type of contribution form scientific text. This paper describes a method based on patterns for semantic relations extraction from scientific texts. The semantic relations correspond to synonymy. We describe pattern acquisition and relations extraction as phases to data evaluation. The evaluation shows promising results with 75% precision, 58% recall and 65% F-measure.

Semi‐supervised, knowledge‐integrated pattern learning approach for fact extraction from judicial text

AbstractTremendous growth in the availability of judicial documents has demanded the rise of information extraction (IE) techniques that support the automatic extraction of relevant concepts or data from judicial texts. Among various approaches available for IE, ontology‐based IE has proven to be the most appropriate for extracting domain‐specific information from natural language text. Through this article, we propose a knowledge‐driven, semi‐supervised pattern‐based learning (bootstrapping) approach to extract domain‐specific facts from judicial text, starting with a small set of seed facts. In the semantic analysis of legal text, fact extraction is the next step to entity identification, which involves the identification of roles played by each entity in the judicial text. The proposed methodology learns extraction patterns for 12 classes of facts from the judicial text through the integration of the domain ontology called judicial case ontology (JCO). The experimental results were evaluated by human judges and found to be quite promising. One main feature of the proposed methodology is its portability across various domains (such as medical, banking, insurance, etc.) which in turn helps build expert systems in various sectors.

Learning Non-Taxonomic Relations of Ontologies

Ontologies, as semantic knowledge representation, have a crucial role in various information systems. The main pitfall of manually building ontologies is effort and time-consuming. Ontology learning is a key solution. Learning Non-Taxonomic Relationships of Ontologies (LNTRO) is the process of automatic/semi-automatic extraction of all possible relationships between concepts in a specific domain, except the hierarchal relations. Most of the research works focused on the extraction of concepts and taxonomic relations in the ontology learning process. This article presents the results of a systematic review of the state-of-the-art approaches for LNTRO. Sixteen approaches have been described and qualitatively analyzed. The solutions they provide are discussed along with their respective positive and negative aspects. The goal is to provide researchers in this area a comprehensive understanding of the drawbacks of the existing work, thereby encouraging further improvement of the research work in this area. Furthermore, this article proposes a set of recommendations for future research.

Machine Learning-based Flu Forecasting Study Using the Official Data from the Centers for Disease Control and Prevention and Twitter Data

Aim/Purpose: In the United States, the Centers for Disease Control and Prevention (CDC) tracks the disease activity using data collected from medical practice's on a weekly basis. Collection of data by CDC from medical practices on a weekly basis leads to a lag time of approximately 2 weeks before any viable action can be planned. The 2-week delay problem was addressed in the study by creating machine learning models to predict flu outbreak. Background: The 2-week delay problem was addressed in the study by correlation of the flu trends identified from Twitter data and official flu data from the Centers for Disease Control and Prevention (CDC) in combination with creating a machine learning model using both data sources to predict flu outbreak. Methodology: A quantitative correlational study was performed using a quasi-experimental design. Flu trends from the CDC portal and tweets with mention of flu and influenza from the state of Georgia were used over a period of 22 weeks from December 29, 2019 to May 30, 2020 for this study. Contribution: This research contributed to the body of knowledge by using a simple bag-of-word method for sentiment analysis followed by the combination of CDC and Twitter data to generate a flu prediction model with higher accuracy than using CDC data only. Findings: The study found that (a) there is no correlation between official flu data from CDC and tweets with mention of flu and (b) there is an improvement in the performance of a flu forecasting model based on a machine learning algorithm using both official flu data from CDC and tweets with mention of flu. Recommendations for Practitioners: In this study, it was found that there was no correlation between the official flu data from the CDC and the count of tweets with mention of flu, which is why tweets alone should be used with caution to predict a flu out-break. Based on the findings of this study, social media data can be used as an additional variable to improve the accuracy of flu prediction models. It is also found that fourth order polynomial and support vector regression models offered the best accuracy of flu prediction models. Recommendations for Researchers: Open-source data, such as Twitter feed, can be mined for useful intelligence benefiting society. Machine learning-based prediction models can be improved by adding open-source data to the primary data set. Impact on Society: Key implication of this study for practitioners in the field were to use social media postings to identify neighborhoods and geographic locations affected by seasonal outbreak, such as influenza, which would help reduce the spread of the disease and ultimately lead to containment. Based on the findings of this study, social media data will help health authorities in detecting seasonal outbreaks earlier than just using official CDC channels of disease and illness reporting from physicians and labs thus, empowering health officials to plan their responses swiftly and allocate their resources optimally for the most affected areas. Future Research: A future researcher could use more complex deep learning algorithms, such as Artificial Neural Networks and Recurrent Neural Networks, to evaluate the accuracy of flu outbreak prediction models as compared to the regression models used in this study. A future researcher could apply other sentiment analysis techniques, such as natural language processing and deep learning techniques, to identify context-sensitive emotion, concept extraction, and sarcasm detection for the identification of self-reporting flu tweets. A future researcher could expand the scope by continuously collecting tweets on a public cloud and applying big data applications, such as Hadoop and MapReduce, to perform predictions using several months of historical data or even years for a larger geographical area.

Social and Behavioral Determinants of Health in the Era of Artificial Intelligence with Electronic Health Records: A Scoping Review.

Background. There is growing evidence that social and behavioral determinants of health (SBDH) play a substantial effect in a wide range of health outcomes. Electronic health records (EHRs) have been widely employed to conduct observational studies in the age of artificial intelligence (AI). However, there has been limited review into how to make the most of SBDH information from EHRs using AI approaches.Methods. A systematic search was conducted in six databases to find relevant peer-reviewed publications that had recently been published. Relevance was determined by screening and evaluating the articles. Based on selected relevant studies, a methodological analysis of AI algorithms leveraging SBDH information in EHR data was provided.Results. Our synthesis was driven by an analysis of SBDH categories, the relationship between SBDH and healthcare-related statuses, natural language processing (NLP) approaches for extracting SBDH from clinical notes, and predictive models using SBDH for health outcomes.Discussion. The associations between SBDH and health outcomes are complicated and diverse; several pathways may be involved. Using NLP technology to support the extraction of SBDH and other clinical ideas simplifies the identification and extraction of essential concepts from clinical data, efficiently unlocks unstructured data, and aids in the resolution of unstructured data-related issues.Conclusion. Despite known associations between SBDH and diseases, SBDH factors are rarely investigated as interventions to improve patient outcomes. Gaining knowledge about SBDH and how SBDH data can be collected from EHRs using NLP approaches and predictive models improves the chances of influencing health policy change for patient wellness, ultimately promoting health and health equity.

Automatic extraction of 12 cardiovascular concepts from German discharge letters using pre-trained language models.

ObjectiveA vast amount of medical data is still stored in unstructured text documents. We present an automated method of information extraction from German unstructured clinical routine data from the cardiology domain enabling their usage in state-of-the-art data-driven deep learning projects.MethodsWe evaluated pre-trained language models to extract a set of 12 cardiovascular concepts in German discharge letters. We compared three bidirectional encoder representations from transformers pre-trained on different corpora and fine-tuned them on the task of cardiovascular concept extraction using 204 discharge letters manually annotated by cardiologists at the University Hospital Heidelberg. We compared our results with traditional machine learning methods based on a long short-term memory network and a conditional random field.ResultsOur best performing model, based on publicly available German pre-trained bidirectional encoder representations from the transformer model, achieved a token-wise micro-average F1-score of 86% and outperformed the baseline by at least 6%. Moreover, this approach achieved the best trade-off between precision (positive predictive value) and recall (sensitivity).ConclusionOur results show the applicability of state-of-the-art deep learning methods using pre-trained language models for the task of cardiovascular concept extraction using limited training data. This minimizes annotation efforts, which are currently the bottleneck of any application of data-driven deep learning projects in the clinical domain for German and many other European languages.

A Semantic Conceptualization Using Tagged Bag-of-Concepts for Sentiment Analysis

Sentiment could be expressed implicitly or explicitly in the text. Hence, it is the main challenge for current sentiment analysis (SA) approaches to identify hidden sentiments, other common challenges include false classification of opinion words, ignoring context information, and bad handling of a short text that arise from the bad interpretation of the text and lack of enough data required for analysis tasks. In this study, a semantic conceptualization method using tagged bag-of-concepts for SA is proposed to detect the correct sentiment towards the actual target entity that considers all affective and conceptual information conveyed in the text with a special focus on the short text. Tagged bag-of-concepts (TBoC) is a novel approach to analyze and decompose text to uncover latent sentiments while preserving all relations and vital information to boost the accuracy of SA. This study answers questions: Does the information provided via TBoC enhance sentiment classification results on different analysis levels? Is building a structure of concepts increases the accuracy of overall sentiment towards specific opinion target? Does TBoC approach enhance SA results for short text messages? The proposed solution has been applied on two datasets from the restaurant domain, sentiment analysis is performed using the TBoCs structure on multiple levels including document, aspect, aspect-category, and topic levels. TBoC method with domain-specific sentiment lexicon showed exceptional performance and outperformed other state-of-the-art NB, SVM, and NN methods, especially for aspect-level SA. The use of TBoC within the semantic conceptualization model that leverages NLP tasks, Ontology, and semantic methods proved its high capabilities for concept extraction while preserving the information about the context, interrelations, and latent feelings.

Extracting Family History of Patients From Clinical Narratives: Exploring an End-to-End Solution With Deep Learning Models.

BackgroundPatients’ family history (FH) is a critical risk factor associated with numerous diseases. However, FH information is not well captured in the structured database but often documented in clinical narratives. Natural language processing (NLP) is the key technology to extract patients’ FH from clinical narratives. In 2019, the National NLP Clinical Challenge (n2c2) organized shared tasks to solicit NLP methods for FH information extraction.ObjectiveThis study presents our end-to-end FH extraction system developed during the 2019 n2c2 open shared task as well as the new transformer-based models that we developed after the challenge. We seek to develop a machine learning–based solution for FH information extraction without task-specific rules created by hand.MethodsWe developed deep learning–based systems for FH concept extraction and relation identification. We explored deep learning models including long short-term memory-conditional random fields and bidirectional encoder representations from transformers (BERT) as well as developed ensemble models using a majority voting strategy. To further optimize performance, we systematically compared 3 different strategies to use BERT output representations for relation identification.ResultsOur system was among the top-ranked systems (3 out of 21) in the challenge. Our best system achieved micro-averaged F1 scores of 0.7944 and 0.6544 for concept extraction and relation identification, respectively. After challenge, we further explored new transformer-based models and improved the performances of both subtasks to 0.8249 and 0.6775, respectively. For relation identification, our system achieved a performance comparable to the best system (0.6810) reported in the challenge.ConclusionsThis study demonstrated the feasibility of utilizing deep learning methods to extract FH information from clinical narratives.

Natural language processing (NLP) tools in extracting biomedical concepts from research articles: a case study on autism spectrum disorder

BackgroundNatural language processing (NLP) tools can facilitate the extraction of biomedical concepts from unstructured free texts, such as research articles or clinical notes. The NLP software tools CLAMP, cTAKES, and MetaMap are among the most widely used tools to extract biomedical concept entities. However, their performance in extracting disease-specific terminology from literature has not been compared extensively, especially for complex neuropsychiatric disorders with a diverse set of phenotypic and clinical manifestations.MethodsWe comparatively evaluated these NLP tools using autism spectrum disorder (ASD) as a case study. We collected 827 ASD-related terms based on previous literature as the benchmark list for performance evaluation. Then, we applied CLAMP, cTAKES, and MetaMap on 544 full-text articles and 20,408 abstracts from PubMed to extract ASD-related terms. We evaluated the predictive performance using precision, recall, and F1 score.ResultsWe found that CLAMP has the best performance in terms of F1 score followed by cTAKES and then MetaMap. Our results show that CLAMP has much higher precision than cTAKES and MetaMap, while cTAKES and MetaMap have higher recall than CLAMP.ConclusionThe analysis protocols used in this study can be applied to other neuropsychiatric or neurodevelopmental disorders that lack well-defined terminology sets to describe their phenotypic presentations.

Clinical concept extraction using transformers.

The goal of this study is to explore transformer-based models (eg, Bidirectional Encoder Representations from Transformers [BERT]) for clinical concept extraction and develop an open-source package with pretrained clinical models to facilitate concept extraction and other downstream natural language processing (NLP) tasks in the medical domain. We systematically explored 4 widely used transformer-based architectures, including BERT, RoBERTa, ALBERT, and ELECTRA, for extracting various types of clinical concepts using 3 public datasets from the 2010 and 2012 i2b2 challenges and the 2018 n2c2 challenge. We examined general transformer models pretrained using general English corpora as well as clinical transformer models pretrained using a clinical corpus and compared them with a long short-term memory conditional random fields (LSTM-CRFs) mode as a baseline. Furthermore, we integrated the 4 clinical transformer-based models into an open-source package. The RoBERTa-MIMIC model achieved state-of-the-art performance on 3 public clinical concept extraction datasets with F1-scores of 0.8994, 0.8053, and 0.8907, respectively. Compared to the baseline LSTM-CRFs model, RoBERTa-MIMIC remarkably improved the F1-score by approximately 4% and 6% on the 2010 and 2012 i2b2 datasets. This study demonstrated the efficiency of transformer-based models for clinical concept extraction. Our methods and systems can be applied to other clinical tasks. The clinical transformer package with 4 pretrained clinical models is publicly available at https://github.com/uf-hobi-informatics-lab/ClinicalTransformerNER. We believe this package will improve current practice on clinical concept extraction and other tasks in the medical domain.

A Concept-based Sentiment Analysis Approach for Arabic

Concept-Based Sentiment Analysis (CBSA) methods are considered to be more advanced and more accurate when it compared to ordinary Sentiment Analysis methods, because it has the ability of detecting the emotions that conveyed by multi-word expressions concepts in language. This paper presented a CBSA system for Arabic language which utilizes both of machine learning approaches and concept-based sentiment lexicon. For extracting concepts from Arabic, a rule-based concept extraction algorithm called semantic parser is proposed. Different types of feature extraction and representation techniques are experimented among the building prosses of the sentiment analysis model for the presented Arabic CBSA system. A comprehensive and comparative experiments using different types of classification methods and classifier fusion models, together with different combinations of our proposed feature sets, are used to evaluate and test the presented CBSA system. The experiment results showed that the best performance for the sentiment analysis model is achieved by combined Support Vector Machine-Logistic Regression (SVM-LR) model where it obtained a F-score value of 93.23% using the Concept-Based-Features+Lexicon-Based-Features+Word2vec-Features (CBF+LEX+W2V) features combinations