Unstructured Text Data Research Articles

Deriving comprehensive literature trends on multi-omics analysis studies in autism spectrum disorder using literature mining pipeline

Autism spectrum disorder (ASD) is characterized by highly heterogenous abnormalities in functional brain connectivity affecting social behavior. There is a significant progress in understanding the molecular and genetic basis of ASD in the last decade using multi-omics approach. Mining this large volume of biomedical literature for insights requires considerable amount of manual intervention for curation. Machine learning and artificial intelligence fields are advancing toward simplifying data mining from unstructured text data. Here, we demonstrate our literature mining pipeline to accelerate data to insights. Using topic modeling and generative AI techniques, we present a pipeline that can classify scientific literature into thematic clusters and can help in a wide array of applications such as knowledgebase creation, conversational virtual assistant, and summarization. Employing our pipeline, we explored the ASD literature, specifically around multi-omics studies to understand the molecular interplay underlying autism brain.

Leveraging LLMs for Unstructured Direct Elicitation of Decision Rules

Unstructured Direct Elicitation (UDE) offers a flexible method to capture consumer preferences and decision rules in an unstructured format such as writing an email. However, it relies on subjective human coding and indicative consideration set sizes to make accurate predictions on consideration decisions. This research leverages large language models (LLMs) to replace human judges and make predictions without the need for additional information like indicative consideration set sizes. Empirical analyses show that fine-tuned LLMs effectively interpret decision rules and handle sophisticated considerations in a complex product scenario (automotive study), outperforming the best UDE models by capturing over 25% more information, while their performance in a moderate-scale study on mobile phones is comparable to the best UDE models. The use of LLMs enhances scalability, cost efficiency, and consistency in comprehending unstructured text data and making predictions, offering a promising alternative to human judges and enabling large-scale, real-time implementation of UDE in marketing research and practice. Together with their ability to interact with users, LLMs fine-tuned with representative datasets may serve as a valuable knowledgebase to summarize consumer preferences and decision rules and supply insights for the creation and simulation of marketing strategies.

An Extended Pattern Based Comprehensive Stemmer for the Urdu Language

The Urdu language is used by approximately 200 million people for spoken and written communications on a daily basis. There is a substantial amount of unstructured Urdu textual data that is available worldwide. Data mining techniques can be used to extract meaningful knowledge from such a large, potentially informative source of data. There are many text processing systems available to process unstructured textual data. However, these systems are mostly language specific and developed for a variety of languages such as English, Spanish, Chinese, etc. Unfortunately, there are not as many language processing resources available for Urdu. Stemming is one of the most important preprocessing steps in the text mining process and its goal is to reduce grammatical words form, e.g., parts of speech, gender, tense, and so on, to their root form. In this work, we have extended the stemming capabilities of our existing pattern-based comprehensive stemming system for Urdu text. In addition to the existing stemming rules in previous work, we introduce novel stemming rules for prefix, and infix stemming. We also optimize the existing suffix removal rules and extend the add character lists for word normalization. These stemming rules are generic and have the ability to generate the stem of Urdu words as well as loan words (words belonging to other languages i.e. Arabic, Persian, Turkish, etc). In the experimental evaluation, we have observed a significant improvement in the overall stemming accuracy of our proposed pattern-based Urud stemmer, which demonstrates the adoptability of the proposed stemming approach for a variety of text-processing applications.

Exploring Natural Language Processing through an Exemplar Using YouTube.

There has been a growing emphasis on data across various health-related fields, not just in nursing research, due to the increasing volume of unstructured data in electronic health records (EHRs). Natural Language Processing (NLP) provides a solution by transforming this unstructured data into structured formats, thereby facilitating valuable insights. This methodology paper explores the application of NLP in nursing, using an exemplar case study that analyzes YouTube data to investigate social phenomena among adults living alone. The methodology involves five steps: accessing data through YouTube's API, data cleaning, preprocessing (tokenization, sentence segmentation, linguistic normalization), sentiment analysis using Python, and topic modeling. This study serves as a comprehensive guide for integrating NLP into nursing research, supplemented with digital content demonstrating each step. For successful implementation, nursing researchers must grasp the fundamental concepts and processes of NLP. The potential of NLP in nursing is significant, particularly in utilizing unstructured textual data from nursing documentation and social media. Its benefits include streamlining nursing documentation, enhancing patient communication, and improving data analysis.

Understanding critical service factors in neobanks: crafting strategies through text mining

Purpose This study aims to analyse and understand customer sentiments and perceptions from neobanking mobile applications by using advanced machine learning and text mining techniques. Design/methodology/approach This study explores a substantial large data set of 330,399 user reviews available in the form of unstructured textual data from neobanking mobile applications. This study is aimed to extract meaningful patterns, topics, sentiments and themes from the data. Findings The results show that the success of neobanking mobile applications depends on user experience, security features, personalised services and technological innovation. Research limitations/implications This study is limited to textual resources available in the public domain, and hence may not present the entire range of user experiences. Further studies should incorporate a wider range of data sources and investigate the impact of regional disparities on user preferences. Practical implications This study provides actionable ideas for neobanking service providers, enabling them to improve service quality and mobile application user experience by integrating customer input and the latest trends. These results can offer important inputs to the process of user interaction design, implementation of new features and customer support services. Originality/value This study uses text mining approaches to analyse neobanking mobile applications, which further contribute to the growing literature on digital banking and FinTech. This study offers a unique view of consumer behaviour and preferences in the realm of digital banking, which will add to the literature on the quality of service concerning mobile applications.

Comparing public health-related material in print and web page versions of legacy media.

The objectives of this study were to create a database of public health content from a sample of legacy media, and to compare the prevalence of public health themes in print and web-based versions over time. A database was created from eleven nationally published magazines as a sample of legacy media content. Relevant material was extracted and coded by the title of the article, periodical, print or web edition, month of publication, item type, and 1-3 public health theme codes. Theme codes emerged as the documents were reviewed based on the primary discussion in each piece. A total of 2558 unique documents were extracted from print issues and 6440 from web-based issues. Seventeen public health themes were identified. Individual coded documents were saved with file names identical to the code string, thus creating a searchable database. Legacy media are those that existed before the internet and social media. Publishers target readership groups defined by age, gender, race, sexual orientation, and other commonalities. Although legacy media have been identified as trusted sources of health information, they have not been examined as sources of public health communication. Because both print and web-based versions exist as unstructured textual data, these are rarely examined with informatics methods. The process described can serve as a model for application of informatics approaches to similar data and assist development of targeted public health communications. Having a better understanding of what types of health content is distributed through legacy media can help to target health messages to specific demographic and interest groups in ways that are understandable and appealing to them.

Development of an Automated Label Data Entry System from Herbarium Specimen Images at Hyogo Herbarium (HYO)

We would like to introduce our recently developed systems for taking images of herbarium specimens and for the automatic extraction of data from specimen labels at the Herbarium of the Museum of Nature and Human Activities, Hyogo, Japan (HYO). Firstly, we designed a low-cost, but high-quality specimen imaging system for non-professional photographers to obtain images rapidly (Takano et al. 2019). Our system uses a mass-produced, mirrorless single-lens reflex (SLR) camera (SONY ILCE6300) with a zoom lens (Samyang Optics SYIO35AF-E35 mm F/2.8). We made a photo stand by ourselves to reduce costs. In addition, we have adopted an LED (light-emitting diode) lighting system with high color rendering. This imaging system has been introduced, with some improvements or adjustments for available space, to various herbaria in Japan (e.g., University of Tokyo (TI), Kyoto University (KYO)), contributing to the digitization of herbarium specimens across Japan. Next, we developed a system to extract label information from specimen images. The specimen image was uploaded to Google OCR and data were extracted in the form of text. Uploading the whole specimen image decreased the reading accuracy of the software because the plant images behaved as OCR (Optical Character Reader) noise. Therefore, the label part was cut out from the whole specimen image by using D-Lib*1 and uploaded to tesseract OCR*2 for OCR extraction of the label information (Aoki 2019, Takano et al. 2020). When installing this system for HYO, we designed it as an application accessible externally via the internet, which proved very useful during the coronavirus pandemic: part-time workers checked and conducted label data input from home. Finally, we decided to develop a system that would automatically label the text data extracted by OCR and input them into the appropriate cells of the database. Even though the text data could be extracted from specimen images, it needed a human to input them into the database. Therefore, we adopted Named Entity Recognition (NER), a system that extracts named entities such as place names, identifying proper nouns from unstructured text data. It enables information recorded in herbarium specimens to be tagged as named entities. We tried text matching at first, but the result was not satisfactory, so we started to use machine learning instead. We compared three natural language libraries for Japanese: BERT (Bidirectional Encoder Representations from Transformers), Albert (A Lite version of BERT), and SpaCy. Despite BERT and SpaCy returning similarly high f-scores (indicating good performance), we decided to use SpaCy because it runs better on ordinary PCs or servers. With sufficient machine learning after the creation of a text corpus (a specialised dataset) specific to labels on herbarium specimens, we successfully developed the application. The project files are available on GitHub*3 (Takano et al. 2024). We then examined whether this system could be applied to non-plant specimen images, i.e., fishes or birds, and found that it could efficiently extract data. Therefore, we decided to publicize this system on the cloud server and share it with other natural history museums in Japan*4. Curators can obtain a unique ID and password and upload specimen images from their collection to extract label data. The digitization of natural history collections in Japan has been long behind other countries, and this system will help to accelerate it. The system mentioned above is specialized for the natural history collections of Japan, but we believe it is possible to build similar programs in other countries, and we hope our experience will contribute to the mobilization of the world’s natural history collections.

Predicting ICU Readmission from Electronic Health Records via BERTopic with Long Short Term Memory Network Approach.

Background: The increasing rate of intensive care unit (ICU) readmissions poses significant challenges in healthcare, impacting both costs and patient outcomes. Predicting patient readmission after discharge is crucial for improving medical quality and reducing expenses. Traditional analyses of electronic health record (EHR) data have primarily focused on numerical data, often neglecting valuable text data. Methods: This study employs a hybrid model combining BERTopic and Long Short-Term Memory (LSTM) networks to predict ICU readmissions. Leveraging the MIMIC-III database, we utilize both quantitative and text data to enhance predictive capabilities. Our approach integrates the strengths of unsupervised topic modeling with supervised deep learning, extracting potential topics from patient records and transforming discharge summaries into topic vectors for more interpretable and personalized predictions. Results: Utilizing a comprehensive dataset of 36,232 ICU patient records, our model achieved an AUROC score of 0.80, thereby surpassing the performance of traditional machine learning models. The implementation of BERTopic facilitated effective utilization of unstructured data, generating themes that effectively guide the selection of relevant predictive factors for patient readmission prognosis. This significantly enhanced the model's interpretative accuracy and predictive capability. Additionally, the integration of importance ranking methods into our machine learning framework allowed for an in-depth analysis of the significance of various variables. This approach provided crucial insights into how different input variables interact and impact predictions of patient readmission across various clinical contexts. Conclusions: The practical application of BERTopic technology in our hybrid model contributes to more efficient patient management and serves as a valuable tool for developing tailored treatment strategies and resource optimization. This study highlights the significance of integrating unstructured text data with traditional quantitative data to develop more accurate and interpretable predictive models in healthcare, emphasizing the importance of individualized care and cost-effective healthcare paradigms.

Real-Time Extraction of News Events Based on BERT Model

Abstract For the large number of news reports generated every day, in order to obtain effective information from these unstructured news text data more efficiently. In this paper, we study the real-time crawling of news data from news websites through crawling techniques and propose a BERT model-based approach to extract events from news long text. In this study, NetEase news website is selected as an example for real-time extraction to crawl the news data of this website. BERT model as a pre-trained model based on two-way encoded representation of transformer performs well on natural language understanding and natural language generation tasks. In this study, we will fine-tune the training based on BERT model on news corpus related dataset and perform sequence annotation through CRF layer to finally complete the event extraction task. In this paper, the DUEE dataset is chosen to train the model, and the experiments show that the overall performance of the BERT model is better than other network models. Finally, the model of this paper is further optimised, using the ALBERT and RoBERTa models improved on the basis of the BERT model, experiments were conducted, the results show that both models are improved compared to the BERT model, the ALBERT model algorithm performs the best, the model algorithm's F1 value is 1% higher than that of BERT. The results show that the performance is optimised.

A Review of Sentiment Analysis in Social Media Perspectives

The widespread use of the Internet and social media platforms has led to an increase in the number of individuals who declare their feelings publicly. Therefore, sentiment analysis systems have proceeded because of their crucial role in determining the personal opinions of users. This is can greatly influence the decision-making process in various fields. To create a robust and reliable sentiment analysis system, it was necessary to apply techniques capable of dealing with these scattered opinions. Natural language processing techniques are commonly used to extract information from unstructured text data published by humans. The comments and posts in social media platforms are often ignore the grammar rules and sentence structure. This is resulting in many ambiguities in lexical, syntactic, and semantic aspects. As a result, researchers have developed different methods for text mining and defining real information. This survey aims to study the different methods used in sentiment analysis filed. We discussed two common models of classification, including the vocabulary-based model and the supervision-based approach.

Visualizing Nursing Narratives: An Evaluation of Latent Dirichlet Allocation Topic Modeling for Care Reports.

The increasing volume of unstructured textual data in healthcare, particularly in nursing care reports, presents both challenges and opportunities for enhancing patient care and operational efficiency. This study explores the application of Latent Dirichlet Allocation (LDA) topic modeling to analyze free-text nursing narratives from inpatient stays in three different clinics, aiming to uncover the latent thematic structures within. Utilizing the R programming environment and the visualization tool LDAvis, we identified three main themes: "Patient Well-being," "Patient Mobility and Care Activities," and "Treatment and Pain Management," the latter combining two closely related but initially distinct topics due to their overlapping content. Our findings demonstrate the potential of LDA topic modeling in extracting meaningful insights from nursing narratives, which could inform patient care strategies and healthcare practices. However, the study also highlights significant challenges associated with the method, including the sensitivity to parameter settings, the lack of updates for key software packages, and concerns about reproducibility. These issues highlight the need for meticulous parameter validation and the exploration of alternative text analysis methodologies for future research. By addressing these methodological challenges and emphasizing the importance of comparative method analysis, this study contributes to the advancement of text analytics in healthcare. It opens avenues for further research aimed at developing more robust, efficient, and accessible tools for analyzing free-text data, thereby enhancing the ability of healthcare professionals to use unstructured data to improve decision making and patient outcomes.

Manufacturing time estimation for offer pricing: A machine learning application in a French metallurgy industry

- In today's market where many companies are competing for the same opportunities, a quick, accurate, and reliable estimation of the offers' prices is essential for the suppliers. This allows them to answer to more solicitations and to remain competitive. Many suppliers use the manufacturing time as key parameter to define their offers' prices. However, in Make-To-Order (MTO) and Engineer-To-Order (ETO) contexts, with the increasing number of demands, the huge diversity of the products and the manufacturing process complexity, estimating the manufacturing time is challenging. To face this issue, this article proposes a new approach for the manufacturing time estimation using machine learning. Its originality relies on the exploitation of structured tabular data, unstructured textual data and domain knowledge. It also proposes two strategies that allow to choose the best estimation model for each product family, while minimizing the number of estimation models and considering product families with fewer available data. All the proposals have been applied to a real industrial case in a French small metallurgy industry, and the results have shown their applicability and effectiveness. Among the plethora of machine learning techniques implemented, CatBoost, which is a member of the family of Gradient Boosted Decision Trees (GBDT) ensemble techniques, provided the best results.

Multiclass Text Classification of Climate Change Reports: Insights into Natural Disasters, Impacts, Locations, and Time Periods

In recent years, the quantity of unstructured information sources, such as documents that include complex information about climate change has grown exponentially. Machine learning techniques may be used to extract semantic information from these unstructured sources to enable semantic organization and classification, as well as a deeper comprehension of the inherent knowledge. Multinomial logistic regression is an effective method for classifying texts into multiple categories simultaneously. Its ability to manage several dependent variables enables precise prediction and categorization, making it ideal for complex climate change data. The paper focuses on the implementation of multinomial logistic regression for text classification to predict the class of a document based on its semantic content. Four custom classifiers were implemented for climate change reports that are tailored to identify and categorize natural disasters and their impacts, locations, and time periods respectively. The ability of the custom classifiers to effectively classify climate change texts was evaluated using an extended report from the Intergovernmental Panel on Climate Change. The resulting predictions from the classifiers highlighted various aspects of the reports regarding these four dimensions and the potential to reveal immanent patterns and relations from unstructured text data.

MedNER: A Service-Oriented Framework for Chinese Medical Named-Entity Recognition with Real-World Application

Named-entity recognition (NER) is a crucial task in natural language processing, especially for extracting meaningful information from unstructured text data. In the healthcare domain, accurate NER can significantly enhance patient care by enabling efficient extraction and analysis of clinical information. This paper presents MedNER, a novel service-oriented framework designed specifically for medical NER in Chinese medical texts. MedNER leverages advanced deep learning techniques and domain-specific linguistic resources to achieve good performance in identifying diabetes-related entities such as symptoms, tests, and drugs. The framework integrates seamlessly with real-world healthcare systems, offering scalable and efficient solutions for processing large volumes of clinical data. This paper provides an in-depth discussion on the architecture and implementation of MedNER, featuring the concept of Deep Learning as a Service (DLaaS). A prototype has encapsulated BiLSTM-CRF and BERT-BiLSTM-CRF models into the core service, demonstrating its flexibility, usability, and effectiveness in addressing the unique challenges of Chinese medical text processing.

Large language models overcome the challenges of unstructured text data in ecology

The vast volume of currently available unstructured text data, such as research papers, news, and technical report data, shows great potential for ecological research. However, manual processing of such data is labour-intensive, posing a significant challenge. In this study, we aimed to assess the application of three state-of-the-art prompt-based large language models (LLMs), GPT-3.5, GPT-4, and LLaMA-2-70B, to automate the identification, interpretation, extraction, and structuring of relevant ecological information from unstructured textual sources. We focused on species distribution data from two sources: news outlets and research papers. We assessed the LLMs for four key tasks: classification of documents with species distribution data, identification of regions where species are recorded, generation of geographical coordinates for these regions, and supply of results in a structured format. GPT-4 consistently outperformed the other models, demonstrating a high capacity to interpret textual data and extract relevant information, with the percentage of correct outputs often exceeding 90% (average accuracy across tasks: 87–100%). Its performance also depended on the data source type and task, with better results achieved with news reports, in the identification of regions with species reports and presentation of structured output. Its predecessor, GPT-3.5, exhibited slightly lower accuracy across all tasks and data sources (average accuracy across tasks: 81–97%), whereas LLaMA-2-70B showed the worst performance (37–73%). These results demonstrate the potential benefit of integrating prompt-based LLMs into ecological data assimilation workflows as essential tools to efficiently process large volumes of textual data.

ADVANCED TECHNIQUES IN MULTI-LABEL TEXT CLASSIFICATION: INTEGRATION OF BETA ANT COLONY AND DEEP LEARNING APPROACHES

The rapid growth of unstructured textual data in various domains has necessitated the development of sophisticated techniques for multi-label text classification. Traditional methods often struggle with handling the complexity and interdependence of multiple labels, leading to suboptimal performance. This paper presents an advanced approach that integrates the Beta Ant Colony Optimization (BACO) algorithm with deep learning techniques for multi-label text classification. The BACO algorithm effectively explores the feature space and selects relevant features by leveraging pheromone trails, while the deep learning model captures intricate patterns and relationships within the textual data. The integration of these two methodologies aims to enhance the efficiency and accuracy of multi-label classification tasks, particularly in domains where label dependency is prominent. Empirical evaluations on benchmark datasets demonstrate that the proposed hybrid approach outperforms existing state-of-the-art techniques in terms of precision, recall, F1-score, and computational efficiency. The findings suggest that combining heuristic optimization algorithms with deep learning can significantly improve multi-label text classification performance, providing a robust solution for real-world applications.

인천 관광지의 속성별 평가를 위한 딥러닝 기반 감성분석

This study applied the IPA(importance-performance analysis) methodology to unstructured text data to evaluate tourists' satisfaction levels with Incheon attractions. Focusing on attractions within the maintenance and improvement-needed areas identified by IPA, it extracted key positive and negative keywords and sentences directly from tourists' opinions. By applying a text-based approach to evolving IPA research, this exploratory study assessed the concrete satisfaction and dissatisfaction elements felt by tourists. The study limited its scope to 15 Incheon attractions and analyzed 114,007 sentences from Naver blogs written by tourists. Satisfaction levels were derived through deep learning sentiment analysis of the text, while the importance of each attraction was determined by the TF-IDF(term frequency-inverse document frequency) ratio of the attraction names in the text categorized by attributes such as attraction, transportation, activities, and food/drink. Key positive and negative keywords and sentences were extracted using the TextRank algorithm, focusing on attractions in the maintenance and improvementneeded areas. The findings reveal that experiential elements of attractions act as differentiating factors, while their absence can provoke negative reactions from tourists. Moreover, content reflecting local culture and history, the development of family-friendly experiential attractions, parking space expansion, and food service improvements emerged as crucial elements for attraction development.

Multi-Source Information Graph Embedding with Ensemble Learning for Link Prediction

Link prediction is a key technique for connecting entities and relationships in a graph reasoning field. It leverages known information about the graph structure data to predict missing factual information. Previous studies have either focused on the semantic representation of a single triplet or on the graph structure data built on triples. The former ignores the association between different triples, and the latter ignores the true meaning of the node itself. Furthermore, common graph-structured datasets inherently face challenges, such as missing information and incompleteness. In light of this challenge, we present a novel model called Multi-source Information Graph Embedding with Ensemble Learning for Link Prediction (EMGE), which can effectively improve the reasoning of link prediction. Ensemble learning is systematically applied throughout the model training process. At the data level, this approach enhances entity embeddings by integrating structured graph information and unstructured textual data as multi-source information inputs. The fusion of these inputs is effectively addressed by introducing an attention mechanism. During the training phase, the principle of ensemble learning is employed to extract semantic features from multiple neural network models, facilitating the interaction of enriched information. To ensure effective model learning, a novel loss function based on contrastive learning is devised, effectively minimizing the discrepancy between predicted values and the ground truth. Moreover, to enhance the semantic representation of graph nodes in link prediction, two rules are introduced during the aggregation of graph structure information. These rules incorporate the concept of spreading activation, enabling a more comprehensive understanding of the relationships between nodes and edges in the graph. During the testing phase, the EMGE model is validated on three datasets, including WN18RR, FB15k-237, and a private Chinese financial dataset. The experimental results demonstrate a reduction in the mean rank (MR) by 0.2 times, an improvement in the mean reciprocal rank (MRR) by 5.9%, and an increase in the Hit@1 by 12.9% compared to the baseline model.

Crafting clarity: Leveraging large language models to decode consumer reviews

Large Language Models (LLMs) have emerged as powerful tools for understanding consumer perceptions and extracting insights from unstructured textual data. This study investigates the effectiveness of LLMs in comprehending consumer opinions, particularly in service industries. We fine-tuned four prominent LLMs—Falcon-7B, MPT-7B, GPT-2, and BERT—using 1,031,478 consumer reviews and assessed their ability to identify topics, emotions, and sentiments. Our results indicate that Falcon-7B excels in accuracy and reliability for complex tasks. This study is the first, to our knowledge, to fine-tune Large Language Models (LLMs) specifically for consumer data, showcasing the efficacy of attention mechanisms in extracting valuable insights. Our findings offer strategic decision-making insights for the service industry and underscore the transformative potential of LLMs in business intelligence from consumer feedback.

Advancing multimodal diagnostics: Integrating industrial textual data and domain knowledge with large language models

The rapid advancement and application of large language models (LLMs) in various domains prompt an investigation into their potential in the field of prognostics and health management (PHM), particularly for enhancing data-driven model capabilities. This study explores the integration of domain knowledge accumulated in unstructured text data such as technical documents and maintenance logs into diagnostics models using LLMs. The study demonstrates the new possibilities to exploit data that are traditionally underutilized due to their complexity and the presence of domain-specific jargon. By leveraging LLMs for contextual understanding and information extraction from such texts, this study proposes a novel approach that combines textual data with existing condition monitoring systems to improve the accuracy of diagnostics models. A case study on hydrogenerators illustrates the feasibility and value of integrating LLMs into PHM systems. The findings suggest that the incorporation of LLMs can lead to more informed, accurate diagnostics, ultimately enhancing operational efficiency and safety within industrial environments.