Deep Natural Language Processing Research Articles

Advanced Misinformation Detection: A Bi-LSTM Model Optimized by Genetic Algorithms

The proliferation of misinformation, as insidious and pervasive as water, presents an unprecedented challenge to public discourse and comprehension. Often propagated to further specific ideologies or political objectives, misinformation not only misleads the populace but also fuels online advertising revenue generation. As such, the urgent need to pinpoint and eliminate misinformation from digital platforms has never been more critical. In response to this dilemma, this paper proposes a solution built on the backbone of massive data generation in today’s digital landscape. By leveraging advanced technologies, such as AI-driven systems with deep learning models and natural language processing capabilities, we can monitor and analyze an extensive scope of social media data. This, in turn, facilitates the identification of misinformation across multiple platforms and alerts users to potential propaganda. Central to our study is the development of misinformation classifiers based on a deep bi-directional long short-term memory (Bi-LSTM) model. This model is further enhanced by employing a genetic algorithm (GA), which automates the search for an optimal neural architecture, thereby significantly impacting the training behavior of the deep learning algorithm and the performance of the model being trained. To validate our approach, we compared the efficacy of our proposed model with nine traditional machine learning algorithms and a deep learning model rooted in long short-term memory (LSTM). The results affirmed the superiority of our GA-tuned Bi-LSTM model, which outperformed all other models in detecting misinformation with remarkable accuracy. Our intention with this paper is not to present our model as a comprehensive solution to misinformation but rather as a technological tool that can aid in the process, supplementing and bolstering the existing methodologies in the field of misinformation detection.

Natural language processing to convert unstructured COVID-19 chest-CT reports into structured reports

BackgroundStructured reporting has been demonstrated to increase report completeness and to reduce error rate, also enabling data mining of radiological reports. Still, structured reporting is perceived by radiologists as a fragmented reporting style, limiting their freedom of expression. PurposeA deep learning-based natural language processing method was developed to automatically convert unstructured COVID-19 chest CT reports into structured reports. MethodsTwo hundred-two COVID-19 chest CT were retrospectively reviewed by two experienced radiologists, who wrote for each exam a free-form text radiological report and coherently filled the template provided by the Italian Society of Medical and Interventional Radiology, used as ground-truth. A semi-supervised convolutional neural network was implemented to extract 62 categorical variables from the report. Two iterations were carried-out, the first without fine-tuning, the second one performing a fine-tuning. The performance was measured using the mean accuracy and the F1 mean score. An error analysis was performed to identify errors entirely attributable to incorrect processing of the model. ResultsThe algorithm achieved a mean accuracy of 93.7% and an F1 score 93.8% in the first iteration. Most of the errors were exclusively attributable to wrong inference (46%). In the second iteration the model achieved for both parameters 95,8% and percentage of errors attributable to wrong inference decreased to 26%. ConclusionsThe convolutional neural network achieved an optimal performance in the automated conversion of free-form text into structured radiological reports, overcoming all the limitation attributed to structured reporting and finally paving the way for data mining of radiological report.

Exploring the effect of digital CSR communication on firm performance: A deep learning approach

This study proposes a novel research framework to examine the effect of digital CSR communications on financial performance while incorporating deep learning techniques to identify firms' CSR communications on social media. Particularly, this research aims to quantify firms' efforts in digital CSR communications by employing cutting-edge deep learning-based natural language processing (NLP) models to detect CSR-related tweets on social media. Utilizing a unique dataset of 65 Chinese public companies in the manufacturing sector between 2015 and 2019, we detected 64,769 long-form tweets posted on WeChat to acquire both digital CSR communications and stakeholder engagement data. Combining financial and secondary data of sample firms, this research reveals the positive but time-lagged influence of digital CSR communications on firms' financial performance, primarily through the lens of agenda-setting theory. We also find that stakeholder engagement plays an essential bridging role in the relationship above, while CSR ratings surprisingly hamper such a positive effect.

Leveraging Fine-Tuned Large Language Models in Bioinformatics: A Research Perspective

Bioinformatics synergizes biology, computer science, and statistics and is further propelled by the integration of deep learning and natural language processing (NLP). This analysis extensively explores the applications of fine-tuned language models within bioinformatics, providing empirical evidence and unique perspectives on the impact, challenges, and limitations in this field. The broad scope includes biomedical literature analysis, drug discovery, clinical decision support, protein structure prediction, and pharmacovigilance, among others. This analysis underscores the need to overcome hurdles such as data availability, domain-specific knowledge, bias, interpretability, resource efficiency, ethical implications, and validation for a reliable application of these models. Collaborative efforts between computational and experimental biologists, ethicists, and regulatory bodies are vital to establish ethical guidelines and best practices for their use.

Leveraging Fine-Tuned Language Models in Bioinformatics: A Research Perspective

Bioinformatics, an interdisciplinary field combining biology, computer science, and statistics, has advanced with deep learning and natural language processing techniques. This perspective explores the applications of fine-tuned language models in bioinformatics, highlighting their potential in various domains while discussing challenges and limitations. Fine-tuned language models benefit biomedical literature analysis, extracting information from scientific papers to synthesize knowledge and generate synthetic sequences for DNA, RNA, and protein research. In drug discovery, these models can identify novel drug targets, accelerate virtual screening, and aid drug repurposing by finding new therapeutic indications for existing drugs. For clinical decision support, fine-tuned language models can analyse patient data, medical literature, and guidelines to provide personalized recommendations and alerts to healthcare professionals. They can also aid accurate protein structure prediction for drug design and target identification. In pharmacovigilance, these models can analyse unstructured data sources to detect adverse events from social media, patient forums, and health records, enabling early intervention and improving patient safety. However, challenges like data availability, domain-specific knowledge, bias, interpretability, resource efficiency, ethics, and validation must be addressed for reliable application. Addressing these challenges will unlock the full potential of fine-tuned language models in bioinformatics, driving advancements and benefiting human health. Collaboration between computational and experimental biologists, ethicists, and regulatory bodies is crucial to establish ethical guidelines and best practices for their use.

Knowledge Enhanced Semantic Communication Receiver

In recent years, with the rapid development of deep learning and natural language processing technologies, semantic communication has become a topic of great interest in the field of communication. Although existing deep learning-based semantic communication approaches have shown many advantages, they still do not make sufficient use of prior knowledge. Moreover, most existing semantic communication methods focus on the semantic encoding at the transmitter side, while we believe that the semantic decoding capability of the receiver should also be concerned. In this paper, we propose a knowledge enhanced semantic communication framework in which the receiver can more actively utilize the facts in the knowledge base for semantic reasoning and decoding, on the basis of only affecting the parameters rather than the structure of the neural networks at the transmitter side. Specifically, we design a transformer-based knowledge extractor to find relevant factual triples for the received noisy signal. Extensive simulation results on the WebNLG dataset demonstrate that the proposed receiver yields superior performance on top of the knowledge graph enhanced decoding.

Syntactic Versus Semantic Similarity of Artificial and Real Faults in Mutation Testing Studies

Fault seeding is typically used in empirical studies to evaluate and compare test techniques. Central to these techniques lies the hypothesis that artificially seeded faults involve some form of realistic properties and thus provide realistic experimental results. In an attempt to strengthen realism, a recent line of research uses machine learning techniques, such as deep learning and Natural Language Processing, to seed faults that look like (syntactically) real ones, implying that fault realism is related to syntactic similarity. This raises the question of whether seeding syntactically similar faults indeed results in semantically similar faults and, more generally whether syntactically dissimilar faults are far away (semantically) from the real ones. We answer this question by employing 4 state-of-the-art fault-seeding techniques (PiTest - a popular mutation testing tool, IBIR - a tool with manually crafted fault patterns, DeepMutation - a learning-based fault seeded framework and μBERT - a mutation testing tool based on the pre-trained language model CodeBERT) that operate in a fundamentally different way, and demonstrate that syntactic similarity does not reflect semantic similarity. We also show that 65.11%, 76.44%, 61.39% and 9.76% of the real faults of Defects4J V2 are semantically resembled by PiTest, IBIR, μBERT and DeepMutation faults, respectively.

Image Captioning Using Deep Learning

Abstract: This paper focuses on developing an image captioning system using deep learning techniques. The paper aims to generate descriptive textual captions for images, enabling machines to understand and communicate the content of visual data. The methodology involves leveraging convolutional neural networks (CNNs) for image feature extraction and recurrent neural networks (RNNs) for sequential language generation. The paper includes steps such as dataset collection, data preprocessing, CNN feature extraction, RNN-based captioning model implementation, model evaluation using metrics like BLEU score and METEOR, and presenting the results obtained. The expected deliverables include a functional image captioning system, comprehensive documentation, and a well-documented codebase. Through this paper, students gain practical experience in deep learning, computer vision, and natural language processing, contributing to advancements in image understanding and humanmachine interaction with visual data

BERT‐TriF: An inductive short text classification model for power equipment defect records

AbstractThe descriptions of power equipment defect records are often characterized by colloquial short texts. Standardized classification of a large number of colloquial defect descriptions has laid a solid foundation for building a power equipment knowledge graph and improving the level of intelligence in the field of power inspection. Using deep learning and natural language processing technology, this paper proposes a text classification model for power equipment defect records named Bidirectional Encoder Representations from Transformers with Family Feature Fusion (BERT‐TriF). The model firstly leverages BERT to semantically represent the input text. To extract family history as well as text implicit information, we creatively propose a family feature fusion algorithm for training. An improved multi‐head attention mechanism is developed subsequently to enhance text semantic category features and strengthen the learning ability of the model. By comparing BERT‐TriF and baseline models such as TextCNN, TextRNN, and fastText on the specified and generic text dataset, the experimental results demonstrate that it has better performance, robustness, and universality for short text classification.

Detecting Ground Glass Opacity Features in Patients With Lung Cancer: Automated Extraction and Longitudinal Analysis via Deep Learning-Based Natural Language Processing.

Ground-glass opacities (GGOs) appearing in computed tomography (CT) scans may indicate potential lung malignancy. Proper management of GGOs based on their features can prevent the development of lung cancer. Electronic health records are rich sources of information on GGO nodules and their granular features, but most of the valuable information is embedded in unstructured clinical notes. We aimed to develop, test, and validate a deep learning-based natural language processing (NLP) tool that automatically extracts GGO features to inform the longitudinal trajectory of GGO status from large-scale radiology notes. We developed a bidirectional long short-term memory with a conditional random field-based deep-learning NLP pipeline to extract GGO and granular features of GGO retrospectively from radiology notes of 13,216 lung cancer patients. We evaluated the pipeline with quality assessments and analyzed cohort characterization of the distribution of nodule features longitudinally to assess changes in size and solidity over time. Our NLP pipeline built on the GGO ontology we developed achieved between 95% and 100% precision, 89% and 100% recall, and 92% and 100% F1-scores on different GGO features. We deployed this GGO NLP model to extract and structure comprehensive characteristics of GGOs from 29,496 radiology notes of 4521 lung cancer patients. Longitudinal analysis revealed that size increased in 16.8% (240/1424) of patients, decreased in 14.6% (208/1424), and remained unchanged in 68.5% (976/1424) in their last note compared to the first note. Among 1127 patients who had longitudinal radiology notes of GGO status, 815 (72.3%) were reported to have stable status, and 259 (23%) had increased/progressed status in the subsequent notes. Our deep learning-based NLP pipeline can automatically extract granular GGO features at scale from electronic health records when this information is documented in radiology notes and help inform the natural history of GGO. This will open the way for a new paradigm in lung cancer prevention and early detection.

Movie information AI question answering system based on knowledge graph

Knowledge is essential for understanding. The ongoing explosion of information highlights the need for electronic texts that allow machines to better understand human language. The explosion of information in various fields has greatly changed our lives. Among them, the information explosion of the film is particularly prominent. Nowadays, people's requirements for high quality of spiritual life are increasing, and people's demand for movies is also increasing. With the extensive coverage of the Internet, the amount of movie-related information is also increasing, so how to design and build a question-and-answer system about movies is more and more urgent and important. At the same time, the original intention of the knowledge graph is to express various entities, concepts that exist objectively in the real world, and various relationships between them. It can better understand the abstract components in the language, and better to confirm the information through the interaction of the ambiguous parts of the language. In addition, the upgrade of deep learning natural language processing technology allows us to better understand and analyse language. In view of the above advantages, the movie question answering system in this paper is constructed based on the knowledge graph and uses natural language processing technology.

An Effective Language Convention Model Based on Deep Structured Learning and Natural Language Processing for Higher Education

Language translation is highly needed in the increasingly networked world. The writing of English essays is an essential competency for higher education students. For non-English students in China, this is especially important. They often represent large numbers of Chinese English as second language students. Therefore this paper introduced Deep Structured Learning with Natural language processing (DSL-NLP) has been proposed to convert the Chinese language to the English language for student learning. An automatic input tool for writing assistance will be beneficial in building the proposed DSL-NLP method. The relationship between textual attributes and input from human teachers and provide characteristics feedback system. This study explored natural language processing (NLP) to determine the interpretative actions required for a successful understanding of the literary text. NLP attempts to assist in achieving consistency in applied linguistics, improving classroom communication, optimizing learner attitudes and motivation, raising personality, facilitating personal development, and even changing students' perspectives toward life. Thus the experimental results show the DSL-NLP improves student learning in the English language, enhances student performance in education, and effectively increases the student career

Skin Disease Detection using Image Processing and CNN

Busy lifestyles these days have led people to forget to drink water regularly which results in inadequate hydration and oily skin, oily skin has become one of the main factors for Acne vulgaris. Acne vulgaris, particularly on the face, greatly affects a person’s social, mental wellbeing and personal satisfaction for teens. Besides the fact that acne is well known as an inflammatory disorder, it was reported to have caused serious long-term consequences such as depression, scarring, mental illness, including pain and suicide. In this research work, a smartphone-based expert system namely “Cureto” is implemented using a hybrid approach i.e. using deep convolutional neural network (CNN) and natural language processing (NLP). The proposed work is designed, implemented and tested to classify Acne density, skin sensitivity and to identify the specific acne subtypes namely whiteheads, blackheads, papules, pustules, nodules and cysts. The proposed work not only classifies Acne Vulgaris but also recommends appropriate treatments based on their classification, severity and other demographic factors such as age, gender, etc. The results obtained show that for Acne type classification the accuracy ranges from 90%-95% and for Skin Sensitivity and Acne density the accuracy ranges from 93%–96%..

Readiness for Artificial Intelligence in Biobanking

Readiness for Artificial Intelligence in Biobanking

Deep learning-assisted prediction of protein-protein interactions in Arabidopsis thaliana.

Currently, the experimentally identified interactome of Arabidopsis (Arabidopsis thaliana) is still far from complete, suggesting that computational prediction methods can complement experimental techniques. Motivated by the prosperity and success of deep learning algorithms and natural language processing techniques, we introduce an integrative deep learning framework, DeepAraPPI, allowing us to predict protein-protein interactions (PPIs) of Arabidopsis utilizing sequence, domain and Gene Ontology (GO) information. Our current DeepAraPPI comprises (i) a word2vec encoding-based Siamese recurrent convolutional neural network (RCNN) model, (ii) a Domain2vec encoding-based multiple layer perceptron (MLP) model, and (iii) a GO2vec encoding-based MLP model. Finally, DeepAraPPI combines prediction results of the three individual predictors through a logistic regression model. Compiling high-quality positive and negative training and test samples by applying strict filtering strategies, DeepAraPPI shows superior performance compared to existing state-of-the-art Arabidopsis PPI prediction methods. DeepAraPPI also provides better cross-species predictive ability in rice (Oryza sativa) than traditional machine learning methods, although the overall performance in cross-species prediction remains to be improved. DeepAraPPI is freely accessible at http://zzdlab.com/deeparappi/. In the meantime, we have also made the source code and datasets of DeepAraPPI available at https://github.com/zjy1125/DeepAraPPI.

Identifying Reasons for Statin Nonuse in Patients With Diabetes Using Deep Learning of Electronic Health Records.

Background Statins are guideline-recommended medications that reduce cardiovascular events in patients with diabetes. Yet, statin use is concerningly low in this high-risk population. Identifying reasons for statin nonuse, which are typically described in unstructured electronic health record data, can inform targeted system interventions to improve statin use. We aimed to leverage a deep learning approach to identify reasons for statin nonuse in patients with diabetes. Methods and Results Adults with diabetes and no statin prescriptions were identified from a multiethnic, multisite Northern California electronic health record cohort from 2014 to 2020. We used a benchmark deep learning natural language processing approach (Clinical Bidirectional Encoder Representations from Transformers) to identify statin nonuse and reasons for statin nonuse from unstructured electronic health record data. Performance was evaluated against expert clinician review from manual annotation of clinical notes and compared with other natural language processing approaches. Of 33 461 patients with diabetes (mean age 59±15 years, 49% women, 36% White patients, 24% Asian patients, and 15% Hispanic patients), 47% (15 580) had no statin prescriptions. From unstructured data, Clinical Bidirectional Encoder Representations from Transformers accurately identified statin nonuse (area under receiver operating characteristic curve [AUC] 0.99 [0.98-1.0]) and key patient (eg, side effects/contraindications), clinician (eg, guideline-discordant practice), and system reasons (eg, clinical inertia) for statin nonuse (AUC 0.90 [0.86-0.93]) and outperformed other natural language processing approaches. Reasons for nonuse varied by clinical and demographic characteristics, including race and ethnicity. Conclusions A deep learning algorithm identified statin nonuse and actionable reasons for statin nonuse in patients with diabetes. Findings may enable targeted interventions to improve guideline-directed statin use and be scaled to other evidence-based therapies.

Ethical requirements in job advertisements: A deep learning approach

AbstractThis study examines ethical requirements in job advertisements as a proxy to understand how ethical qualities are required in the job market. We apply deep learning and natural language processing to analyze the ethical requirements in 196,272 job postings from 13,703 companies across a variety of industries and professions. This large sample allows us to generate a holistic review of ethical requirements in job postings by industry and profession. Specifically, our results show that less than half of the job postings in our sample include requirements for ethics, despite increased attention to ethical conduct. The IT industry had the fewest job postings with ethical requirements compared to other industries. Of the topics within ethical requirements coded in this study, obligations to professional standards occurred most frequently whereas information confidentiality and privacy are less prevalent. Our results have important implications for the ethical practices of businesses and academic ethical education.

Hybridizing qualitative coding with natural language processing and deep learning to assess public comments: A case study of the clean power plan

Public comments are a rich source of data on attitudes toward public policy, but the scale poses major challenges for qualitative analyses. Supervised deep learning and natural language processing potentially enable classification of these data but require high quality labeled inputs. This study investigates whether hybridizing rigorous qualitative coding methods with machine learning approaches can classify large amounts of policy-oriented text data (public comments) while keeping manual effort tractable. Using a convolutional neural network, we evaluate spatiotemporal variation in themes and expressed public attitudes toward a specific US climate policy, the Clean Power Plan (CPP), which was proposed in 2014 and repealed in 2019 without ever taking effect. Public comments were solicited for both the proposal and proposed repeal across eight cities, representing a large and highly targeted dataset on dynamic public attitudes toward the CPP. Rigorous manual coding and data augmentation techniques enabled good model performance (F1 scores of 0.71 and 0.81, respectively, for sentiment and topic classification), even with a very small training set. We find that most speakers supported the CPP despite its eventual repeal and uncover notable rhetorical trends like competing narratives of justice for those affected by climate change versus fossil fuel host communities.

Information extraction from German radiological reports for general clinical text and language understanding

Recent advances in deep learning and natural language processing (NLP) have opened many new opportunities for automatic text understanding and text processing in the medical field. This is of great benefit as many clinical downstream tasks rely on information from unstructured clinical documents. However, for low-resource languages like German, the use of modern text processing applications that require a large amount of training data proves to be difficult, as only few data sets are available mainly due to legal restrictions. In this study, we present an information extraction framework that was initially pre-trained on real-world computed tomographic (CT) reports of head examinations, followed by domain adaptive fine-tuning on reports from different imaging examinations. We show that in the pre-training phase, the semantic and contextual meaning of one clinical reporting domain can be captured and effectively transferred to foreign clinical imaging examinations. Moreover, we introduce an active learning approach with an intrinsic strategic sampling method to generate highly informative training data with low human annotation cost. We see that the model performance can be significantly improved by an appropriate selection of the data to be annotated, without the need to train the model on a specific downstream task. With a general annotation scheme that can be used not only in the radiology field but also in a broader clinical setting, we contribute to a more consistent labeling and annotation process that also facilitates the verification and evaluation of language models in the German clinical setting.

Deep learning approach to detection of colonoscopic information from unstructured reports

BackgroundColorectal cancer is a leading cause of cancer deaths. Several screening tests, such as colonoscopy, can be used to find polyps or colorectal cancer. Colonoscopy reports are often written in unstructured narrative text. The information embedded in the reports can be used for various purposes, including colorectal cancer risk prediction, follow-up recommendation, and quality measurement. However, the availability and accessibility of unstructured text data are still insufficient despite the large amounts of accumulated data. We aimed to develop and apply deep learning-based natural language processing (NLP) methods to detect colonoscopic information.MethodsThis study applied several deep learning-based NLP models to colonoscopy reports. Approximately 280,668 colonoscopy reports were extracted from the clinical data warehouse of Samsung Medical Center. For 5,000 reports, procedural information and colonoscopic findings were manually annotated with 17 labels. We compared the long short-term memory (LSTM) and BioBERT model to select the one with the best performance for colonoscopy reports, which was the bidirectional LSTM with conditional random fields. Then, we applied pre-trained word embedding using large unlabeled data (280,668 reports) to the selected model.ResultsThe NLP model with pre-trained word embedding performed better for most labels than the model with one-hot encoding. The F1 scores for colonoscopic findings were: 0.9564 for lesions, 0.9722 for locations, 0.9809 for shapes, 0.9720 for colors, 0.9862 for sizes, and 0.9717 for numbers.ConclusionsThis study applied deep learning-based clinical NLP models to extract meaningful information from colonoscopy reports. The method in this study achieved promising results that demonstrate it can be applied to various practical purposes.