Natural Language Processing Methodologies Research Articles

Simplifying synthesis of the expanding glioblastoma literature: a topic modeling approach.

Our study aims to discover the leading topics within glioblastoma (GB) research, and to examine if these topics have "hot" or "cold" trends. Additionally, we aim to showcase the potential of natural language processing (NLP) in facilitating research syntheses, offering an efficient strategy to dissect the landscape of academic literature in the realm of GB research. The Scopus database was queried using "glioblastoma" as the search term, in the "TITLE" and "KEY" fields. BERTopic, an NLP-based topic modeling (TM) method, was used for probabilistic TM. We specified a minimum topic size of 300 documents and 5% probability cutoff for outlier detection. We labeled topics based on keywords and representative documents and visualized them with word clouds. Linear regression models were utilized to identify "hot" and "cold" topic trends per decade. Our TM analysis categorized 43,329 articles into 15 distinct topics. The most common topics were Genomics, Survival, Drug Delivery, and Imaging, while the least common topics were Surgical Resection, MGMT Methylation, and Exosomes. The hottest topics over the 2020s were Viruses and Oncolytic Therapy, Anticancer Compounds, and Exosomes, while the cold topics were Surgical Resection, Angiogenesis, and Tumor Metabolism. Our NLP methodology provided an extensive analysis of GB literature, revealing valuable insights about historical and contemporary patterns difficult to discern with traditional techniques. The outcomes offer guidance for research directions, policy, and identifying emerging trends. Our approach could be applied across research disciplines to summarize and examine scholarly literature, guiding future exploration.

Natural Language Processing (NLP) in Recommendation Systems

This paper delves into the application of Natural Language Processing (NLP) techniques in recommendation systems, specifically focusing on novel approaches to enhance recommendation accuracy and user satisfaction. The utilization of NLP algorithms has revolutionized how content is recommended to users, leveraging linguistic analysis and machine learning to understand user preferences and provide tailored suggestions. Our research explores various NLP methodologies, including sentiment analysis, topic modelling, and semantic analysis, to extract meaningful insights from textual data. Furthermore, we investigate the integration of deep learning models, such as neural networks and transformer architectures, to capture complex patterns and improve recommendation precision. A key highlight of our study is the introduction of a novel recommendation method termed "Knowledge Graph Embedding for Contextual Recommendation." This innovative approach combines knowledge graph representation with contextual understanding, allowing for more nuanced and personalized recommendations based on user interactions, historical data, and contextual relevance. We delve into the intricacies of this technique, detailing its implementation, training process, and evaluation metrics.

Mapping the Degenerative Cervical Myelopathy Research Landscape: Topic Modeling of the Literature.

Topic modeling of literature. Our study has 2 goals: (i) to clarify key themes in degenerative cervical myelopathy (DCM) research, and (ii) to evaluate the current trends in the popularity or decline of these topics. Additionally, we aim to highlight the potential of natural language processing (NLP) in facilitating research syntheses. Documents were retrieved from Scopus, preprocessed, and modeled using BERTopic, an NLP-based topic modeling method. We specified a minimum topic size of 25 documents and 50 words per topic. After the models were trained, they generated a list of topics and corresponding representative documents. We utilized linear regression models to examine trends within the identified topics. In this context, topics exhibiting increasing linear slopes were categorized as "hot topics," while those with decreasing slopes were categorized as "cold topics". Our analysis retrieved 3510 documents that were classified into 21 different topics. The 3 most frequently occurring topics were "OPLL" (ossification of the posterior longitudinal ligament), "Anterior Fusion," and "Surgical Outcomes." Trend analysis revealed the hottest topics of the decade to be "Animal Models," "DCM in the Elderly," and "Posterior Decompression" while "Morphometric Analyses," "Questionnaires," and "MEP and SSEP" were identified as being the coldest topics. Our NLP methodology conducted a thorough and detailed analysis of DCM research, uncovering valuable insights into research trends that were otherwise difficult to discern using traditional techniques. The results provide valuable guidance for future research directions, policy considerations, and identification of emerging trends.

DNA of learning behaviors: A novel approach of learning performance prediction by NLP

In recent years, the field of learning analytics has gained significant attention as educators and researchers seek to understand and optimize the learning process in online learning systems. This paper presents a novel methodology for predicting learning performance in online learning systems by leveraging natural language processing (NLP) and embedding techniques. The study focuses on two online learning systems, namely BookRoll and Viscode, and aims to analyze the learning behaviors of students using system logs extracted from the databases. The logs are converted into semester and daily learning description documents to capture the daily activities and progress of the learners. To transform the natural language data into numerical representations, the transformer-based BERT model, Google Gemini, and OpenAI large text embedding methodology are employed to generate embeddings for the learning descriptions. Subsequently, k-means clustering is applied to identify distinct learning behaviors exhibited by students. These clusters are labeled with numbers, and the daily learning descriptions are combined into a sequence, referred to as the DNA of learning behaviors. By utilizing this DNA representation, the learning status of students is effectively captured, and a machine learning model is trained to predict learning performance. The experimental results demonstrate the efficacy of the proposed methodology in achieving highly convincing predictions. The contributions of this research lie in the adoption of a unique approach, integrating NLP methodologies and embeddings techniques, to enable accurate learning performance prediction in online learning systems.

SALSA: Semantically-Aware Latent Space Autoencoder

In deep learning for drug discovery, molecular representations are often based on sequences, known as SMILES, which allow for straightforward implementation of natural language processing methodologies, one being the sequence-to-sequence autoencoder. However, we observe that training an autoencoder solely on SMILES is insufficient to learn molecular representations that are semantically meaningful, where semantics are specified by the structural (graph-to-graph) similarities between molecules. We demonstrate by example that SMILES-based autoencoders may map structurally similar molecules to distant codes, resulting in an incoherent latent space that does not necessarily respect the semantic similarities between molecules. To address this shortcoming we propose Semantically-Aware Latent Space Autoencoder (SALSA) for molecular representations: a SMILES-based transformer autoencoder modified with a contrastive task aimed at learning graph-to-graph similarities between molecules. To accomplish this, we develop a novel dataset comprised of sets of structurally similar molecules and opt for a supervised contrastive loss that is able to incorporate full sets of positive samples. We evaluate semantic awareness of SALSA representations by comparing to its ablated counterparts, and show empirically that SALSA learns representations that maintain 1) structural awareness, 2) physicochemical awareness, 3) biological awareness, and 4) semantic continuity.

Improving Automated Labeling for ATT&CK Tactics in Malware Threat Reports

Once novel malware is detected, threat reports are written by security companies that discover it. The reports often vary in the terminology describing the behavior of the malware making comparisons of reports of the same malware from different companies difficult. To aid in the automated discovery of novel malware, it was recently proposed that novel malware could be detected by identifying behaviors. This assumes that a core set of behaviors are present in most, if not all, malware variants. However, there is a lack of malware datasets that are labeled with behaviors. Motivated by a need to label malware with a common set of behaviors, this work examines automating the process of labeling malware with behaviors identified in malware threat reports despite the variability of terminology. To do so, we examine several techniques from the natural language processing (NLP) domain. We find that most state-of-the-art word embedding NLP methods require large amounts of data and are trained on generic corpora of text data—missing the nuances related to information security. To address this, we use simple feature selection techniques. We find that simple feature selection techniques generally outperform word embedding methods and achieve an increase of 6% in the F .5 -score over prior work when used to predict MITRE ATT&CK tactics in threat reports. Our work indicates that feature selection, which has commonly been overlooked by sophisticated methods in NLP tasks, is beneficial for information security related tasks, where more sophisticated NLP methodologies are not able to pick out relevant information security terms.

Innovative agricultural ontology construction using NLP methodologies and graph neural network

Advancements in technology brought various innovations to agricultural practices. As a part of the development, establishing an agricultural ontology would unleash the growth of cross-domain agriculture and Natural Language Processing (NLP). For constructing such domain-based ontology, semantic and syntactic understanding of the domain data is needed. In agriculture, the availability of pre-determined domain-based data is not sufficient hence, a standard methodology with syntactic and general semantic features are required for processing the data. In this research work, Agricultural Domain based Ontology Construction (ADOC) is proposed and the overall framework has three approaches for establishing the agriculture domain based ontologies. The input text documents undergo anaphora resolution phase utilizing the semantic-based method. In the first method of ADOC the ontology is developed using the terms and relationships that are extracted from the NLP techniques. The second method of ADOC uses pretrained BERT model and Hearst patterns while the third model of ADOC is based on pretrained BERT with regular expressions and unsupervised Graph Neural Network (GNN) for creating the agricultural ontology. The efficacy of the proposed ADOC utilizing BERT with regular expressions and GNN method shows an outstanding result when compared to other proposed and prevailing systems, with a precision and recall of 96.67% and 98.31%.

An Explainable Prediction for Dietary-Related Diseases via Language Models.

Our study harnesses the power of natural language processing (NLP) to explore the relationship between dietary patterns and metabolic health outcomes among Korean adults using data from the Seventh Korea National Health and Nutrition Examination Survey (KNHANES VII). Using Latent Dirichlet Allocation (LDA) analysis, we identified three distinct dietary patterns: "Traditional and Staple", "Communal and Festive", and "Westernized and Convenience-Oriented". These patterns reflect the diversity of dietary preferences in Korea and reveal the cultural and social dimensions influencing eating habits and their potential implications for public health, particularly concerning obesity and metabolic disorders. Integrating NLP-based indices, including sentiment scores and the identified dietary patterns, into our predictive models significantly enhanced the accuracy of obesity and dyslipidemia predictions. This improvement was consistent across various machine learning techniques-XGBoost, LightGBM, and CatBoost-demonstrating the efficacy of NLP methodologies in refining disease prediction models. Our findings underscore the critical role of dietary patterns as indicators of metabolic diseases. The successful application of NLP techniques offers a novel approach to public health and nutritional epidemiology, providing a deeper understanding of the diet-disease nexus. This study contributes to the evolving field of personalized nutrition and emphasizes the potential of leveraging advanced computational tools to inform targeted nutritional interventions and public health strategies aimed at mitigating the prevalence of metabolic disorders in the Korean population.

Tracing topics and trends in drug-resistant epilepsy research using a natural language processing-based topic modeling approach.

Epilepsy is a common neurological disorder affecting over 70 million people worldwide. Although many patients achieve seizure control with anti-epileptic drugs (AEDs), 30%-40% develop drug-resistant epilepsy (DRE), where seizures persist despite adequate trials of AEDs. DRE is associated with reduced quality of life, increased mortality and morbidity, and greater socioeconomic challenges. The continued intractability of DRE has fueled exponential growth in research that aims to understand and treat this serious condition. However, synthesizing this vast and continuously expanding DRE literature to derive insights poses considerable difficulties for investigators and clinicians. Conventional review methods are often prolonged, hampering the timely application of findings. More-efficient approaches to analyze the voluminous research are needed. In this study, we utilize a natural language processing (NLP)-based topic modeling approach to examine the DRE publication landscape, uncovering key topics and trends. Documents were retrieved from Scopus, preprocessed, and modeled using BERTopic. This technique employs transformer models like BERT (Bidirectional Encoder Representations from Transformers) for contextual understanding, thereby enabling accurate topic categorization. Analysis revealed 18 distinct topics spanning various DRE research areas. The 10 most common topics, including "AEDs," "Neuromodulation Therapy," and "Genomics," were examined further. "Cannabidiol," "Functional Brain Mapping," and "Autoimmune Encephalitis" emerged as the hottest topics of the current decade, and were examined further. This NLP methodology provided valuable insights into the evolving DRE research landscape, revealing shifting priorities and declining interests. Moreover, we demonstrate an efficient approach to synthesizing and visualizing patterns within extensive literature that could be applied to other research fields.

The importance of online customer reviews characteristics on remanufactured product sales: Evidence from the mobile phone market on Amazon.com

Remanufactured products have emerged as an important option for achieving sustainable development, which is crucial to the success of the circular economy. Consequently, it is necessary to investigate the influence of online customer reviews on remanufactured product sales, an aspect that has not been well understood in current research. In this study, we employed natural language processing methodologies, analyzing over 50,000 reviews from 2020 to 2021 on Amazon.com to explore the impacts of online customer reviews on remanufactured product sales. The findings indicate that the formal characteristics of online customer reviews, that is, review length and review valence have positive influences on remanufactured product sales. Furthermore, the content characteristics of online customer reviews, including the volume and valence of product usability, product service quality, and product cost performance, positively promote remanufactured product sales. Additionally, this study reveals three emotional characteristics of online customer reviews that significantly influence remanufactured product sales. Specifically, contentment and surprise positively affect remanufactured product sales, while anger negatively affects remanufactured product sales. The findings of this study provide useful insights for remanufactured product sellers to manage online customer reviews and develop marketing strategies.

Incivility in COVID-19 Vaccine Mandate Discourse and Moral Foundations: Natural Language Processing Approach.

Vaccine hesitancy poses a substantial threat to efforts to mitigate the harmful effects of the COVID-19 pandemic. To combat vaccine hesitancy, officials in the United States issued vaccine mandates, which were met with strong antivaccine discourse on social media platforms such as Reddit. The politicized and polarized nature of COVID-19 on social media has fueled uncivil discourse related to vaccine mandates, which is known to decrease confidence in COVID-19 vaccines. This study examines the moral foundations underlying uncivil COVID-19 vaccine discourse. Moral foundations theory poses that individuals make decisions to express approval or disapproval (ie, uncivil discourse) based on innate moral values. We examine whether moral foundations are associated with dimensions of incivility. Further, we explore whether there are any differences in the presence of incivility between the r/coronaviruscirclejerk and r/lockdownskepticism subreddits. Natural language processing methodologies were leveraged to analyze the moral foundations underlying uncivil discourse in 2 prominent antivaccine subreddits, r/coronaviruscirclejerk and r/lockdownskepticism. All posts and comments from both of the subreddits were collected since their inception in March 2022. This was followed by filtering the data set for key terms associated with the COVID-19 vaccine (eg, "vaccinate" and "Pfizer") and mandates (eg, "forced" and "mandating"). These key terms were selected based on a review of existing literature and because of their salience in both of the subreddits. A 10% sample of the filtered key terms was used for the final analysis. Findings suggested that moral foundations play a role in the psychological processes underlying uncivil vaccine mandate discourse. Specifically, we found substantial associations between all moral foundations (ie, care and harm, fairness and cheating, loyalty and betrayal, authority and subversion, and sanctity and degradation) and dimensions of incivility (ie, toxicity, insults, profanity, threat, and identity attack) except for the authority foundation. We also found statistically significant differences between r/coronaviruscirclejerk and r/lockdownskepticism for the presence of the dimensions of incivility. Specifically, the mean of identity attack, insult, toxicity, profanity, and threat in the r/lockdownskepticism subreddit was significantly lower than that in the r/coronaviruscirclejerk subreddit (P<.001). This study shows that moral foundations may play a substantial role in the presence of incivility in vaccine discourse. On the basis of the findings of the study, public health practitioners should tailor messaging by addressing the moral values underlying the concerns people may have about vaccines, which could manifest as uncivil discourse. Another way to tailor public health messaging could be to direct it to parts of social media platforms with increased uncivil discourse. By integrating moral foundations, public health messaging may increase compliance and promote civil discourse surrounding COVID-19.

Utilizing an Attention-Based LSTM Model for Detecting Sarcasm and Irony in Social Media

Sarcasm and irony represent intricate linguistic forms in social media communication, demanding nuanced comprehension of context and tone. In this study, we propose an advanced natural language processing methodology utilizing long short-term memory with an attention mechanism (LSTM-AM) to achieve an impressive accuracy of 99.86% in detecting and interpreting sarcasm and irony within social media text. Our approach involves innovating novel deep learning models adept at capturing subtle cues, contextual dependencies, and sentiment shifts inherent in sarcastic or ironic statements. Furthermore, we explore the potential of transfer learning from extensive language models and integrating multimodal information, such as emojis and images, to heighten the precision of sarcasm and irony detection. Rigorous evaluation against benchmark datasets and real-world social media content showcases the efficacy of our proposed models. The outcomes of this research hold paramount significance, offering a substantial advancement in comprehending intricate language nuances in digital communication. These findings carry profound implications for sentiment analysis, opinion mining, and an enhanced understanding of social media dynamics.

Applying Natural Language Processing to ClinicalTrials.gov: mRNA cancer vaccine case study.

Recently, biotechnology and pharmaceutical industries have made strides to adopt and implement Natural Language Processing (NLP) to address challenges faced when extracting and synthesizing high volumes of information found in unstructured and semistructured text. Here we present, and provide a summary of the findings from, a use case where NLP and text mining methodologies were used to extract clinical trial data from ClinicalTrials.gov for mRNA cancer vaccines.

Investigating online activity in UK adolescent mental health patients: a feasibility study using a natural language processing approach for electronic health records

ObjectivesTo assess the feasibility of using a natural language processing (NLP) application for extraction of free-text online activity mentions in adolescent mental health patient electronic health records (EHRs).SettingThe Clinical Records...

Daily Posture Behavior Patterns Derived From Multitime-Scale Topic Models Using Wearable Triaxial Acceleration for Assessment of Concern About Falling.

Concern about falling is prevalent in older population. This condition would cause a series of adverse physical and psychological consequences for older adults' health. Traditional assessment of concern about falling is relied on self-reported questionnaires and thus is too subjective. Therefore, we proposed a novel multi-time-scale topic modelling approach to quantitatively evaluate concern about falling by analyzing triaxial acceleration signals collected from a wearable pendent sensor. Different posture segments were firstly recognized to extract their corresponding feature subsets. Then, each selected feature related to concern about falling was clustered into discrete levels as feature letters of artificial words in different time scales. As a result, all older participants' signal recordings were converted to a collection of artificial documents, which can be processed by natural language processing methodologies. The topic modelling technique was used to discover daily posture behavior patterns from these documents as discriminants between older adults with different levels of concern about falling. The results indicated that there were significant differences in distributions of posture topics between groups of older adults with different levels of concern about falling. Additionally, the transitions of posture topics over daytime and nighttime revealed temporal regularities of posture behavior patterns of older adult's active and inactive status, which were substantially different for older adults with different levels of concern about falling. Finally, the level of concern about falling was accurately determined with accuracy of 71.2% based on the distributions of posture topics combined with the mobility performance metrics of walking behaviors and demographic information.

Information consistency as response to pre-launch advertising communications: The case of YouTube trailers

IntroductionPre-launch advertising communications are critical for the early adoption of experiential products. Often, companies release a variety of advertising messages for the same product, which results in a lack of information consistency. Research on the effect of advertising communications with different message content is scarce. Further, most studies on information consistency rely on experimental methods, leaving the actual effect of consumer response on product adoption unknown.MethodsTreating online comments to movie trailers as consumer response to advertising communication, we propose a natural language processing methodology to measure information consistency. We validate our measurement through an online experiment and test it on 1.3 million YouTube comments.ResultsOur empirical results provide evidence that information consistency driven by trailer-viewing is a key driver of opening box office success.DiscussionInsights deriving from this study are important to marketing communications research, especially in contexts where early product adoption is critical.

Text Mining Oral Histories in Historical Archaeology.

Advances in text mining and natural language processing methodologies have the potential to productively inform historical archaeology and oral history research. However, text mining methods are largely developed in the context of contemporary big data and publicly available texts, limiting the applicability of these tools in the context of historical and archaeological interpretation. Given the ability of text analysis to efficiently process and analyze large volumes of data, the potential for such tools to meaningfully inform historical archaeological research is significant, particularly for working with digitized data repositories or lengthy texts. Using oral histories recorded about a half-century ago from the anthracite coal mining region of Pennsylvania, USA, we discuss recent methodological developments in text analysis methodologies. We suggest future pathways to bridge the gap between generalized text mining methods and the particular needs of working with historical and place-based texts.

Connect with Student-Educational Consumer Brand Management Engagement: A Social Media Management Perspective

Social media empowers individuals to express their views regarding the events that take place in their day-to-day life. These events also incorporate the sentiments of students as well as educators towards their educational institutions. Thus, the study aims to analyze the sentiments of the students regarding their educational institutions to hike consumer brand management. However, Understanding the student at all levels of his being (physical, mental, emotional spiritual) is crucial to generate valuable learning and bringing forth a sense of belongingness. The study proposes to use social media platforms to connect, understand and engage the students at the professional as well as personal levels. Design/methodology/approach: Sentiment Analysis using vectorization and polarity has been applied to data collected from tweets of several leading universities of the world with their social media appearances to acquire significant insights into the objective of the study. During the analysis, it has been observed that symbol such as #hashtags and @ has played an intensified role in arousing the emotions of the individuals which eventually contributed to the branding of a particular designated university. Findings: Sentiment and emotional identification play a significant role in increasing brand value and nurturing the relationship between the student and the educator. It has been observed from the study that a sense of intensified community building is promoted which encourages new individuals to become a part of the reputed universities. Research Limitation/Implication: Since the field of education plays a great important role in the policies and economy of every country, students are the driving factors of this sector. Hence it is important to identify the emotion and sentiments of an individual regarding each small role that he goes through while studying. The university plays a very important role thus the study focuses on how each activity in the university affects the sentiments of a student and at the same time, it targets the nurturing of the student-educator relationship. The study focuses on how distinctive activities at the university level, their implication, and engagement at social media platforms affect the student's sentiments. The study is limited to the student-educator relationship. The present study contributes to the increase in consumer brand participation through emotion analysis and natural language processing methodology. Practical Implication: It refers to the fulfillment of certain conditions that if gets fulfilled will change reality to some extent. In this study, we have incorporated different tweets collected from several top universities across the world, since the data is in the form of tweets it has been processed to be subjected to analytical purposes. The practical implication for the study lies in the collection of datasets as the core of the study is the tweets that form the dataset. Social Implication: The social implication of the study is the representation of the sentiment of an individual regarding his or her university activities (Quality of Education, Research activities, etc.). After the analysis, there is a sense of pride has been observed among the student regarding the achievements of their educators and university. This not only inculcates a sense of belongingness among a group but also leads to the engagement of the student-educator relationship which further nurtures the policies of the educational universities to increase the brand value and consumer engagement. Originality/value: Engaging students on social media platforms, not only urge them to pursue it as a crucial medium of information but also bring a sense of pride to be part of their university. The study's findings not only contribute to the current understanding of sentiment analysis in brand engagement but are also important to develop a sense of belongingness in the students towards their educational institution which further nurtures the student-educator relationship management.

Insights into Search Engine Optimization using Natural Language Processing and Machine Learning

Among the potential tools in digital marketing, Search Engine Optimization (SEO) facilitates the use of appropriate data by providing appropriate results according to the search priority of the user. Various research-based approaches have been developed to improve the optimization performance of search engines over the past decade; however, it is still unclear what the strengths and weaknesses of these methods are. As a result of the increased proliferation of Machine Learning (ML) and Natural Language Processing (NLP) in complex content management, there is potential to achieve successful SEO results. Therefore, the purpose of this paper is to contribute towards performing an exhaustive study on the respective NLP and ML methodologies to explore their strengths and weaknesses. Additionally, the paper highlights distinct learning outcomes and a specific research gap intended to assist future research work with a guideline necessary for optimizing search engine performance.

Automatic Classification of Tumor Response From Radiology Reports With Rule-Based Natural Language Processing Integrated Into the Clinical Oncology Workflow.

Imaging reports in oncology provide critical information about the disease evolution that should be timely shared to tailor the clinical decision making and care coordination of patients with advanced cancer. However, tumor response stays unstructured in free-text and underexploited. Natural language processing (NLP) methods can help provide this critical information into the electronic health records (EHR) in real time to assist health care workers. A rule-based algorithm was developed using SAS tools to automatically extract and categorize tumor response within progression or no progression categories. 2,970 magnetic resonance imaging, computed tomography scan, and positron emission tomography French reports were extracted from the EHR of a large comprehensive cancer center to build a 2,637-document training set and a 603-document validation set. The model was also tested on 189 imaging reports from 46 different radiology centers. A tumor dashboard was created in the EHR using the Timeline tool of the vis.js javascript library. An NLP methodology was applied to create an ontology of radiographic terms defining tumor response, mapping text to five main concepts, and application decision rules on the basis of clinical practice RECIST guidelines. The model achieved an overall accuracy of 0.88 (ranging from 0.87 to 0.94), with similar performance on both progression and no progression classification. The overall accuracy was 0.82 on reports from different radiology centers. Data were visualized and organized in a dynamic tumor response timeline. This tool was deployed successfully at our institution both retrospectively and prospectively as part of an automatic pipeline to screen reports and classify tumor response in real time for all metastatic patients. Our approach provides an NLP-based framework to structure and classify tumor response from the EHR and integrate tumor response classification into the clinical oncology workflow.