Large Volumes Of Text Data Research Articles

Comprehensive Survey of Abstractive Text Summarization Techniques

Text summarization using pre-trained encoders has become a crucial technique for efficiently managing large volumes of text data. The rise of automatic summarization systems addresses the need to process ever-increasing data while meeting user-specific requirements. Recent scientific research highlights significant advancements in abstractive summarization, with a particular focus on neural network-based methods. A detailed review of various neural network models for abstractive summarization identifies five key components essential to their design: encoder-decoder architecture, mechanisms, training strategies and optimization algorithms, dataset selection, and evaluation metrics. Each of these elements is pivotal in enhancing the summarization process. This study aims to provide a thorough understanding of the latest developments in neural network-based abstractive summarization models, offering insights into the evolving field and underscoring the associated challenges. Qualitative analysis using a concept matrix reveals common design trends in contemporary neural abstractive summarization systems. Notably, BERT-based encoder-decoder models have emerged as leading innovations, representing the most recent progress in the field. Based on the insights from this review, the study recommends integrating pre-trained language models with neural network techniques to achieve optimal performance in abstractive summarization tasks. As the volume of online information continues to surge, the field of automatic text summarization has garnered significant attention within the Natural Language Processing (NLP) community. Spanning over five decades, researchers have approached this problem from diverse angles, exploring various domains and employing a multitude of paradigms. This survey aims to delve into some of the most pertinent methodologies, focusing on both single-document and multiple-document summarization techniques, with a particular emphasis on empirical methods and extractive approaches. Additionally, the survey explores promising strategies that target specific intricacies of the summarization task. Notably, considerable attention is dedicated to the automatic evaluation of summarization systems, recognizing its pivotal role in guiding future research endeavors.

Towards digital-twin-enabled facility management: the natural language processing model for managing facilities in buildings

ABSTRACT Facility management is an essential part of the building which encompasses multiple service areas and processes to ensuring optimum functionality of the facility. Digital twin (DT) has been increasingly introduced to the FM domain to facilitate FM tasks such as decision-making. Despite the advancements in DT-enabled FM, there still exists a problem in the disconnections between DT academics and FM practitioners. The emergence of knowledge management can act as a bridge between academics and FM practitioners. To mine knowledge in the DT-enabled FM industry for a better communication between different stakeholders and timely decisions in the case of emergency, this study proposes a natural language processing (NLP)-based approach. A detailed classification of FM-related entities is first presented, which divides the FM into five categories. The NLP model is used to extract entities and knowledge from a large volume of text data. The results indicate that an NLP-based approach can accurately extract FM-related knowledge. The findings of this study not only provide a basis for automatic DT-enabled FM-related knowledge extraction but also support the downstream tasks such as knowledge graph construction and FM-related emergency decision-making.

Analysis of the Present and Future of the Korea Online Platform Regulation Using Latent Dirichlet Allocation

Purpose: This study was conducted to provide a comprehensive understanding of online platform regulation in South Korea. It aims to illuminate key issues related to online platform regulation from 2017 to 2021, exploring the current state and future prospects. The primary goal is to foster social discourse on online platform regulation and to offer valuable insights for policymakers and academics. Design/methodology/approach: The methodology involved collecting articles from a large news database using keywords associated with online platform regulation. This data was analyzed using Latent Dirichlet Allocation (LDA) topic modeling. This technique is instrumental in identifying themes and patterns within large volumes of text data, facilitating the identification of significant themes and trends related to online platform regulation. Findings: The study unearthed a variety of themes. Among the most significant are the impact of the COVID-19 pandemic and its ramifications, political aspects, the roles of major Korean corporations (e.g., Naver and Kakao), the influence of global tech giants, the impact of news and online communities, technology and innovation, and social issues. These themes indicate that online platform regulation in Korea is influenced by a multitude of factors. Research limitations/implications: While this study offers diverse perspectives on online platform regulation within Korea, it has limitations. Reliance on data from a single news source may restrict the study's diversity and depth. Future research should employ a variety of news sources to provide a broader perspective on online platform regulation. Originality/value: This research makes a significant contribution to the academic study and policy formulation in the field by analyzing the key themes and trends related to online platform regulation over time. By exploring the complex aspects of online platform regulation, the study provides new insights for policymakers, researchers, and the general public.

A Survey of the Applications of Text Mining for the Food Domain

In the food domain, text mining techniques are extensively employed to derive valuable insights from large volumes of text data, facilitating applications such as aiding food recalls, offering personalized recipes, and reinforcing food safety regulation. To provide researchers and practitioners with a comprehensive understanding of the latest technology and application scenarios of text mining in the food domain, the pertinent literature is reviewed and analyzed. Initially, the fundamental concepts, principles, and primary tasks of text mining, encompassing text categorization, sentiment analysis, and entity recognition, are elucidated. Subsequently, an analysis of diverse types of data sources within the food domain and the characteristics of text data mining is conducted, spanning social media, reviews, recipe websites, and food safety reports. Furthermore, the applications of text mining in the food domain are scrutinized from the perspective of various scenarios, including leveraging consumer food reviews and feedback to enhance product quality, providing personalized recipe recommendations based on user preferences and dietary requirements, and employing text mining for food safety and fraud monitoring. Lastly, the opportunities and challenges associated with the adoption of text mining techniques in the food domain are summarized and evaluated. In conclusion, text mining holds considerable potential for application in the food domain, thereby propelling the advancement of the food industry and upholding food safety standards.

Harnessing Generative Pre-Trained Transformers for Construction Accident Prediction with Saliency Visualization

Leveraging natural language processing models using a large volume of text data in the construction safety domain offers a unique opportunity to improve understanding of safety accidents and the ability to learn from them. However, little effort has been made to date in regard to utilizing large language models for the prediction of accident types that can help to prevent and manage potential accidents. This research aims to develop a model for predicting the six types of accidents (caught-in-between, cuts, falls, struck-by, trips, and others) by employing transfer learning with a fine-tuned generative pre-trained transformer (GPT). Additionally, to enhance the interpretability of the fine-tuned GPT model, a method for saliency visualization of input text was developed to identify words that significantly impact prediction results. The models were evaluated using a comprehensive dataset comprising 15,000 actual accident records. The results indicate that the suggested model for detecting the six accident types achieves 82% accuracy. Furthermore, it was observed that the proposed saliency visualization method can identify accident precursors from unstructured free-text data of construction accident reports. These results highlight the advancement of the generalization performance of large language processing-based accident prediction models, thereby proactively preventing construction accidents.

K-means Clustering of Tweet Emotions: A 2D PCA Visualization Approach

With the growing prominence of social media as a platform for expressing opinions and emotions, understanding the emotional undercurrents in large volumes of text data has become increasingly crucial. Tweets, often reflecting public sentiment, contain a rich tapestry of emotions that can be harnessed for diverse applications ranging from market analysis to mental health monitoring. The dataset comprises 40,000 tweet records, each tagged with one of thirteen distinct emotions, making it a challenging task to perform multiclass emotion classification due to the sheer volume of data and the nuanced spectrum of emotional expressions. Traditional classification models often struggle with such high-dimensional, multi-category data, leading to the need for a more sophisticated approach that ensures both accuracy and computational efficiency. To address the complexity of multiclass emotion classification, we propose a novel approach that combines text preprocessing, advanced feature extraction using TF-IDF, and dimensionality reduction via 2D Principal Component Analysis (PCA). We then apply K-means clustering to the reduced feature set to identify inherent groupings within the emotional content of the tweets. This method not only reduces computational demands but also logically consolidates the emotions into fewer categories, potentially enhancing the performance of subsequent classification models. The implementation of our method yielded distinct clusters that suggest a logical grouping of the emotions within the tweets. The 2D PCA visualization revealed clear separations among clusters, indicating that our approach successfully captured meaningful patterns in the dataset. The ability to effectively cluster complex emotional data opens the door to creating more nuanced and efficient multiclass classification models. By reducing the number of categories and focusing on clustered groups, we can streamline the classification process and enhance the interpretability of results. This has significant implications for real-world applications, including targeted marketing campaigns, public policy development, and mental health assessment from social media content.

არგუმენტაციული სტრუქტურების ანალიზი: არგუმენტაციის სქემები ქართულ პოლიტიკურ ზეპირმეტყველებაში

Our daily verbal communication – official, formal, informal, about general, professional, or private topics – represents our subjective opinions that mostly include reasoning and arguments which we use in order to prove our point. The linguistic realisation (verbalisation) of our personal position always requires the use of extraordinary syntactic-pragmatic structural elements, such as arguments that add credibility to the statement. The study of argumentative structures allows us to identify the linguistic cues that turn a regular expression into an argumentative statement. Understanding those cues in a particular language helps the speakers improve their communication skills.Researching argumentative structures in the context of natural language processing has become a very active process over the past few years. One of the main focuses is on the computational processing of argumentative texts, which significantly increases the feasibility of automatic processing and the semantic analysis of large volumes of text data. This paper aims to describe and reveal argumentation structures in the spoken Georgian language. The research data comes from a sub-corpus of the Georgian National Corpus, the Georgian Political Text Corpus. The database is thematically diverse and contains politicians’ speeches and TV debates, specifically:● public speeches of the Presidents of Georgia and other politicians (Zviad Gamsaxurdia, Eduard Shevardnadze, Mikheil Saakashvili, Noe Jordania, Giorgi Kvirikashvili, Shalva Natelashvili)● recordings of Parliament and TV debates (Parliament sessions, political TV shows).The corpus of political texts not only contains thematically diverse materials but is also quite representative from a quantitative point of view: This article analyses the constituent parts of an argumentative text such as premises and reasoning and their connections, with the aim to generally characterise the argumentative structures in Georgian on the basis of Janier and Saint-Disier 2019.1) Arguments based on analogy:Premise 1: Case C1 is similar to case C2.Premise 2: A is true (false) in case C1.Conclusion: A is true (false) in case C2. 2) Arguments based on negative consequences:Premise 1: If an action leads to bad consequences, everything else being equal, it should not be exerted.Premise 2: If action A is exerted, bad consequences will occur.Conclusion: A should not be exerted.3) Arguments based on expert opinions:Premise 1: E is a reliable authority in the domain S.Premise 2: A is a proposition contained in S.Premise 3: E asserts that A.Conclusion: A is true.4) Arguments based on examples:Premise 1: Example 1 is an example that supports claim P.Premise 2: Example n is an example that supports claim P.Conclusion: Claim P is true.5) Arguments based on positions to know:Premise 1: Source A is in a position to know something about domain S which contains proposition P.Premise 2: A asserts that P (in domain S) is true (false),Conclusion: P is true (false)6) Arguments based on popular opinions:Premise 1: Everybody does X.Premise 2: X is a good thing to do.Conclusion: X must be the right thing to do.7) Arguments based on causes and effects:Premise 1: Doing X will cause Y to occur, or if X occurs then Y will occur.Premise 2: X is done, or X occurs.Conclusion: Y will occur

Argumentation Mining System for Corpus-based Discourse Analysis based on Structured Arguments

An Argumentation mining system can analyze a large volume of text data through a variety of sources. Nowadays it is highly useful in the areas of business, economics, and finance with digital marketing being the most promising field along with social media. It is the study of corpus-based discourse analysis that involves the automatic identification of argumentative structure in text. Initially, AM talks about extracting structured arguments from natural text, often unstructured or noisy text. Theoretical approaches of AM and pragmatic schemes that satisfy the needs of social media generated data, recognizing the need for adapting more flexible and expandable schemes, capable of adjusting to argumentation conditions that exist in social media. In this scenario it is a very challenging argumentation scheme able to identify the distinct sub-task and capture the needs of social media text, revealing the need for adopting a more flexible and extensible framework. Corpus-based Machine Learning of linguistic annotations has enabled researchers to identify repetitive linguistic patterns of language use and to uncover hidden meaning in all areas of Natural Language Processing.

Chatgpt: A New Era In Research Writing Assistance.

The integration of artificial intelligence in everyday tasks is becoming prevalent, and its application has extended to academic and scientific writing. The utilization of large language models, such as ChatGPT, has gained significant traction among researchers and scientific writers in recent times. According to a recent article published in Nature, ChatGPT is capable of generating text that sounds both realistic and intelligent in response to user prompts (1). A recent study published in December 2022 evaluated the ability of ChatGPT, to execute complex cognitive tasks and generate text that is indistinguishable from text produced by humans. (2), suggesting that it can be difficult for researchers to distinguish between abstracts created by AI and original ones (1). Another study published in December 2022 showed that ChatGPT could generate precise and believable scientific abstracts (3).Likewise An additional study published in 2022 postulated that based on the analysis of ChatGPT's applications, capabilities, and limitations, it can significantly enhance academic research across a broad range of fields. (4). A recently published literature review using ChatGPT has been cited as an example of ChatGPT application in the field of scientific writing and research (5). Moreover, ChatGPT has also been used to write and publish editorials and research papers (1, 6, 7). Despite the benefits, some researchers are still evaluating the implications and effects of using these models in the long term. (1). This letter aims to present the advantages of utilizing ChatGPT, a large language model developed by OpenAI, in the field of scientific writing. ChatGPT can help researchers in a various ways, including generating text, summarizing text, completing the text, editing text, and creating references, bibliography and citations. The most significant application of ChatGPT in scientific writing is text generation, as the model can be used to generate outlines, abstracts, and introductions to research papers. However, it is imperative to point out that ChatGPT cannot provide text with proper citations to research papers and sources. ChatGPT can also support researchers in the text summarization process, enabling them to extract meaningful information from vast amounts of text data like research papers or articles. Additionally, some researchers use ChatGPT to analyze large volumes of text data to gain deeper insights and knowledge, which is particularly useful in qualitative research and in writing literature reviews. ---Continue

Leveraging Text-Mining Techniques On Electronic Medical Records to Analyze National Drug-insured Medication Use

Processing electronic medical record (EMR) data has become a common practice among scientists for extracting valuable insights and studying diseases. Given the large volumes of text data in EMRs, efficient computerized text-mining techniques are necessary. As academics, we recognize that drug-used analysis from EMR data in Indonesia is currently limited. This study focuses on obtaining meaningful insights from EMR data to make positive recommendations for hospitals. The proposed method uses pattern-based Regular Expressions (regex) to extract drug names and a Levenshtein distance algorithm to check their compatibility. We developed the pattern based on analyzing Indonesia EMR data. The extracted drug names were compared to a list of selected drugs (National Drug-Insured/Fornas) that are required and must be provided at healthcare facilities in Indonesia. The Levenshtein distance threshold was set to two to decide whether the extracted drug names belonged to nationally drug-insured or not. Only about 11.09 – 16.11% of medications given by doctors are listed in the Fornas drug list. Between 2019 and 2021, there was an inaccuracy in the writing of prescriptions for Fornas drugs, with as many as 57.53% to 63.21% of drug names being written incorrectly. The results of this study indicate that the Levenshtein distance algorithm has promising potential for implementation in the Ministry of Health of Indonesia, with a precision rate of 97.07%.

K-Means and J48 Algorithms to Categorize Student Research Abstracts

Text mining is a rapidly growing field in computer science that is used to extract meaningful information from text data. This information can be used for various applications, such as categorizing research abstracts based on their content. This study focuses on the use of text mining techniques. The goal was to determine which algorithm was more accurate in categorizing the research abstracts. The results of the study indicated that the J48 algorithm outperformed the K-Means algorithm in terms of accuracy. This suggests that the J48 algorithm is a more effective method for categorizing research abstracts based on their content. Additionally, the findings provide insight into the use of text mining techniques for categorizing research abstracts in specific fields, such as computer science. Overall, the study demonstrates the potential of text mining techniques for analyzing and categorizing large volumes of text data. As the field of text mining continues to grow, it is likely that more applications will emerge, making it easier to extract valuable information from unstructured text data. The findings of this study can be used to improve the efficiency and accuracy of text mining techniques, particularly for categorizing research abstracts in specific fields.

Applications of Text Mining in the Transportation Infrastructure Sector: A Review

Transportation infrastructure is vital to the well-functioning of economic activities in a region. Due to the digitalization of data storage, ease of access to large databases, and advancement of social media, large volumes of text data that relate to different aspects of transportation infrastructure are generated. Text mining techniques can explore any large amount of textual data within a limited time and with limited resource allocation for generating easy-to-understand knowledge. This study aims to provide a comprehensive review of the various applications of text mining techniques in transportation infrastructure research. The scope of this research ranges across all forms of transportation infrastructure-related problems or issues that were investigated by different text mining techniques. These transportation infrastructure-related problems or issues may involve issues such as crashes or accidents investigation, driving behavior analysis, and construction activities. A Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA)-based structured methodology was used to identify relevant studies that implemented different text mining techniques across different transportation infrastructure-related problems or issues. A total of 59 studies from both the U.S. and other parts of the world (e.g., China, and Bangladesh) were ultimately selected for review after a rigorous quality check. The results show that apart from simple text mining techniques for data pre-processing, the majority of the studies used topic modeling techniques for a detailed evaluation of the text data. Other techniques such as classification algorithms were also later used to predict and/or project future scenarios/states based on the identified topics. The findings from this study will hopefully provide researchers and practitioners with a better understanding of the potential of text mining techniques under different circumstances to solve different types of transportation infrastructure-related problems. They will also provide a blueprint to better understand the ever-evolving area of transportation engineering and infrastructure-focused studies.

Sentimental analysis on user’s reviews using BERT

In the current E-Commerce and M−Commerce evolved Internet world, people are expressing their sentiments on the product or service in the form of reviews and ratings. Sentiment analysis finds a contextual inference in the user sentiment which helps the E-Commerce sector and other end-users to realize the opinion of the product or service. People give their opinion in a variety of formats like reviews, blogs, news, or comments. Sentiment Analysis is used quick gaining insights using large volumes of text data which can be helpful in improving the quality of service and can make companies to earn huge profits. In sentimental analysis, modeling of sentimental relations such as word negation and word intensification are great challenge. In the proposed work, Bidirectional Encoder Representations from Transformers (BERT) is constructed for classifying user sentiments on IMDB Movie Review and Amazon Fine Food Review datasets. Word negations and intensifications are also considered for deriving the sentiment of reviews. The experimental results shows that the BERT can capture the sentimental relations effectively than basic ML, DL models and other recent literatures.

Cross-Societal Analysis of Climate Change Awareness and Its Relation to SDG 13: A Knowledge Synthesis from Text Mining

The awareness and the engagement of various stakeholders play a crucial role in the successful implementation of climate policy and Sustainable Development Goals (SDGs). SDG 13, which refers to climate action, has three targets for combating climate change and its impact. Among the three targets, SDG 13.3 aims to “improve education, awareness-raising and human and institutional capacity on climate change mitigation, adaptation, impact reduction, and early warning”. This target should be implemented based on the understanding of climate change awareness among various groups of societies. Furthermore, the indicator related to awareness-raising is absent in SDG 13.3. Hence, this study aims to explore the differences in climate change awareness among various social groups within a country from a text mining technique. By collecting and analyzing a large volume of text data from various sources, climate change awareness was investigated from a multilateral perspective. Two text analyses were utilized for this purpose: Latent Dirichlet Allocation (LDA) topic modeling and term co-occurrence network analysis. In order to integrate and comparatively analyze the awareness differences among diverse groups, extracted topics were compared by classifying them into four indicators derived from the detailed targets in SDG 13.3: mitigation, adaptation, impact reduction, and early warning. The results show that the Korean public exhibited a relatively high awareness of early warning compared to the other four groups, and the media dealt with climate change issues with the widest perspective. The Korean government and academia notably had a high awareness of both climate change mitigation and adaptation. In addition, corporations based in Korea were observed to have substantially focused awareness on climate change mitigation for greenhouse gas reduction. This research successfully explored the disproportion and lack of climate change awareness formed in different societies of public, social, government, industry, and academic groups. Consequently, these results could be utilized as a decision criterion for society-tailored policy formulation and promoting climate action. Our results suggest that this methodology could be utilized as a new SDG indicator and to measure the differences in awareness.

Somun: entity-centric summarization incorporating pre-trained language models

Text summarization resolves the issue of capturing essential information from a large volume of text data. Existing methods either depend on the end-to-end models or hand-crafted preprocessing steps. In this study, we propose an entity-centric summarization method which extracts named entities and produces a small graph with a dependency parser. To extract entities, we employ well-known pre-trained language models. After generating the graph, we perform the summarization by ranking entities using the harmonic centrality algorithm. Experiments illustrate that we outperform the state-of-the-art unsupervised learning baselines by improving the performance more than 10% for ROUGE-1 and more than 50% for ROUGE-2 scores. Moreover, we achieve comparable results to recent end-to-end models.

Innovation hotspots in food waste treatment, biogas, and anaerobic digestion technology: A natural language processing approach

The objective of this study is to apply natural language processing to identifying innovative technology trends related to food waste treatment, biogas, and anaerobic digestion. The methodology used involved analyzing large volumes of text data mined from 3186 patents related to these three fields. Latent Dirichlet Allocation and the perplexity method were used to identify the main topics which the patent corpora were comprised of and which technological concepts were most associated with each topic. In addition, term frequency-inverse document frequency (TF-IDF) was used to gauge the “emergingness” of certain technical concepts across the patent corpora in various years. The key results were as follows: (1) perplexity computations showed that a 20 topic models were feasible for these patent corpora; (2) topics were identified, providing an accurate picture of the patenting landscape in the analyzed fields; (3) TF-IDF analysis on unigrams, bigrams, and trigrams, supplemented with network graph analysis, revealed emerging technology trends in each year. This study has important implications for governments who need to decide where to invest resources in anaerobic food waste treatment.

Automatic event detection in microblogs using incremental machine learning

The global popularity of microblogs has led to an increasing accumulation of large volumes of text data on microblogging platforms such as Twitter. These corpora are untapped resources to understand social expressions on diverse subjects. Microblog analysis aims to unlock the value of such expressions by discovering insights and events of significance hidden among swathes of text. Besides velocity; diversity of content, brevity, absence of structure and time‐sensitivity are key challenges in microblog analysis. In this paper, we propose an unsupervised incremental machine learning and event detection technique to address these challenges. The proposed technique separates a microblog discussion into topics to address the key problem of diversity. It maintains a record of the evolution of each topic over time. Brevity, time‐sensitivity and unstructured nature are addressed by these individual topic pathways which contribute to generate a temporal, topic‐driven structure of a microblog discussion. The proposed event detection method continuously monitors these topic pathways using multiple domain‐independent event indicators for events of significance. The autonomous nature of topic separation, topic pathway generation, new topic identification and event detection, appropriates the proposed technique for extensive applications in microblog analysis. We demonstrate these capabilities on tweets containing #microsoft and tweets containing #obama.

텍스트마이닝을 활용한 미국 대통령 취임 연설문의 트렌드 연구

Identification of meaningful patterns and trends in large volumes of text data is an important task in various research areas. In the present study, we propose a procedure to find meaningful tendencies based on a combination of text mining, cluster analysis, and low-dimensional embedding. To demonstrate applicability and effectiveness of the proposed procedure, we analyzed the inaugural addresses of the presidents of the United States from 1789 to 2009. The main results of this study show that trends in the national policy agenda can be discovered based on clustering and visualization algorithms.

Towards Interactive Construction of Topical Hierarchy: A Recursive Tensor Decomposition Approach.

Automatic construction of user-desired topical hierarchies over large volumes of text data is a highly desirable but challenging task. This study proposes to give users freedom to construct topical hierarchies via interactive operations such as expanding a branch and merging several branches. Existing hierarchical topic modeling techniques are inadequate for this purpose because (1) they cannot consistently preserve the topics when the hierarchy structure is modified; and (2) the slow inference prevents swift response to user requests. In this study, we propose a novel method, called STROD, that allows efficient and consistent modification of topic hierarchies, based on a recursive generative model and a scalable tensor decomposition inference algorithm with theoretical performance guarantee. Empirical evaluation shows that STROD reduces the runtime of construction by several orders of magnitude, while generating consistent and quality hierarchies.

Association Rule Mining and Network Analysis in Oriental Medicine

Extracting useful and meaningful patterns from large volumes of text data is of growing importance. In the present study we analyze vast amounts of prescription data, generated from the book of oriental medicine to identify the relationships between the symptoms and the associated medicines used to treat these symptoms. The oriental medicine book used in this study (called Bangyakhappyeon) contains a large number of prescriptions to treat about 54 categorized symptoms and lists the corresponding herbal materials. We used an association rule algorithm combined with network analysis and found useful and informative relationships between the symptoms and medicines.