Text Mining Processing Research Articles

An Extended Pattern Based Comprehensive Stemmer for the Urdu Language

The Urdu language is used by approximately 200 million people for spoken and written communications on a daily basis. There is a substantial amount of unstructured Urdu textual data that is available worldwide. Data mining techniques can be used to extract meaningful knowledge from such a large, potentially informative source of data. There are many text processing systems available to process unstructured textual data. However, these systems are mostly language specific and developed for a variety of languages such as English, Spanish, Chinese, etc. Unfortunately, there are not as many language processing resources available for Urdu. Stemming is one of the most important preprocessing steps in the text mining process and its goal is to reduce grammatical words form, e.g., parts of speech, gender, tense, and so on, to their root form. In this work, we have extended the stemming capabilities of our existing pattern-based comprehensive stemming system for Urdu text. In addition to the existing stemming rules in previous work, we introduce novel stemming rules for prefix, and infix stemming. We also optimize the existing suffix removal rules and extend the add character lists for word normalization. These stemming rules are generic and have the ability to generate the stem of Urdu words as well as loan words (words belonging to other languages i.e. Arabic, Persian, Turkish, etc). In the experimental evaluation, we have observed a significant improvement in the overall stemming accuracy of our proposed pattern-based Urud stemmer, which demonstrates the adoptability of the proposed stemming approach for a variety of text-processing applications.

Statistical Approaches Used in Studies Evaluating the Reliability of Autonomous Vehicles Based on Disengagements and Reaction Times

The rapid development of autonomous vehicle (AV) technology highlights the critical importance of enhancing the reliability of these vehicles. Due to the need to test the reliability of AVs, since 2014, the California Department of Motor Vehicles has permitted autonomous vehicle manufacturers to establish an AV Testing program, enabling them to test automated systems on the transportation network. With this, studies on the reliability of AVs have increased rapidly. The most emphasized issues regarding the reliability of AVs have been disengagements, accidents, and reaction times. In this study, disengagements and reaction times are categorized and explained in detail according to the data type, company, period, and statistical method. The data used in the studies cover the years 2014-2020. When examining studies on the reliability of AVs, until 2018, inferences were generally made using real data and descriptive statistics, particularly with methods such as correlation analysis and calculation of disengagements per mile, which investigates the relationship between distance traveled and disengagements. However, since 2018, machine learning has gained importance in evaluating AV reliability. It has been observed that regression, classification, and decision trees were frequently used during this period. Techniques such as deep transfer learning, text mining, and natural language processing also stand out. Furthermore, Software Reliability Growth Models are used to measure software reliability, playing an essential role in evaluating, analyzing, and improving the performance and reliability of AVs. This study aims to reveal the development and diversity of the statistical methods used to determine AV reliability. Additionally, this study aims to guide and provide insights to researchers in the field about the statistical approaches they can utilize.

Applications of satellite platforms and machine learning for mapping and monitoring grasslands and pastures: A systematic and comprehensive review

Grasslands and pastures are critical ecosystems globally, essential for their agricultural and environmental roles. Extensive research from 2000 to 2022, comprising 504 articles, has explored the integration of remote sensing and statistical modelling for mapping and monitoring grassland and pasture ecosystems. These articles were sourced from the SCOPUS database and analysed using text mining and natural language processing techniques to investigate the evolution of publication trends over the past twenty-two years. The number of publications per year on this topic has grown consistently in the considered period, from 3 in 2000 to 93 in 2022, doubling their weight compared to the total number of Scopus publications. The quantitative analysis of satellite platform utilisation revealed the increasing importance of Sentinel-2, even though MODIS remained the most utilised satellite platform throughout the study period. The increasing availability of big data has helped spread the utilisation of machine learning algorithms, mostly in the last ten years. Among these, random forest appeared to be the most widespread for grassland and pasture studies. Researchers' primary interest in this field centres on the technologies and their applications. This is evidenced by cluster analysis, which reveals a dominant focus on terms related to the 'Instruments' (25.8 %) and 'Parameters' (24.9 %) categories. This analysis aims to outline the progression of research, offering insights that could be useful in forecasting future trends and facilitating stakeholder engagement in this sector.

NAKNN: An Efficient Classification of Indonesian News Texts with Nazief-Adriani and KNN

Internet usage in Indonesia has seen a significant increase, reaching 215.63 million users in 2022-2023, or 78.19% of the population. With the ease of internet access, digital news portals like Narasi TV have become a primary source of information for many people. However, the large number of news articles makes manual categorizing challenging. This study aims to classify Indonesian-language news documents from Narasi TV using the Nazief-Adriani algorithm for stemming and the K-Nearest Neighbor (KNN) method for classification. The text mining process begins with preprocessing, which includes case folding, tokenizing, stop-word filtering, and stemming. Using a dataset of 500 news documents, the study demonstrated that with a 90:10 data split, the average accuracy reached 93%, with the highest value being 100%. For the 80:20 data split, the average accuracy was 89%, with the highest value being 93%, and for a 70:30 data split, the average accuracy was 87%, with the highest value being 89%. In conclusion, the combination of the Nazief-Adriani algorithm and the KNN method with optimal k selection and random states obtained high accuracy, obtaining an average accuracy of 93%) in classifying Indonesian-language news documents. These results demonstrate the significant potential of text mining and classification techniques to manage digital news.

Automated data-driven condition assessment method for concrete bridges

This paper addresses the challenge of automating data-driven condition assessments for concrete bridges, hindered by limited data availability. The developed method consists of (1) data integration and standardization, and (2) condition assessment modules. The first module captures, structures, and integrates bridge data from diverse sources, including inspection reports, using web scraping and rule-based data extraction. It standardizes inspection data through text mining and natural language processing. The second module employs Bayesian belief networks to assess bridge deck conditions, leveraging standardized data. The method was validated on a set of bridges in Québec, Canada, resulting in a structured, integrated data repository with standardized data from 2255 inspection reports. This repository supports the development of advanced asset management tools for this class of bridges. The method achieved 94.6% accuracy, 95.0% precision, 94.7% recall, and 94.1% F1 score, demonstrating its potential to help transportation agencies improve bridge data and asset management efficiency.

Application of Natural Language Processing to the Development of Sports Biomechanics in China: A Literature Review of Journal Abstracts in Chinese Between 1980 and 2022

This paper aims to explore the field of sports biomechanics in China between 1980 and 2022 in terms of key developments, hot research topics, integration with other disciplines in kinesiology, and future trends by using text mining and natural language processing to analyze abstracts published in Chinese journals. Over 1,400 research paper abstracts were selected and processed, focusing on specific terms, significance, word-cloud analysis, co-occurrence, and network analysis. Results showed that sports biomechanics research focused on sports technical analysis, application of sports biomechanical principles to athletic training, and sport-injury prevention and rehabilitation. The research areas are multidimensional but well balanced with other disciplines such as physical education, sports training, and motor skill acquisition. Integration with fields like biomedical engineering, computer software and applications, and medical aspects of specific environments suggesting sports biomechanics has a promising future as it continues to develop as a discipline interwoven with other disciplines.

Enhanced Text Classification with Label-Aware Graph Convolutional Networks

Text classification is an important research field in text mining and natural language processing, gaining momentum with the growth of social networks. Despite the accuracy advancements made by deep learning models, existing graph neural network-based methods often overlook the implicit class information within texts. To address this gap, we propose a graph neural network model named LaGCN to improve classification accuracy. LaGCN utilizes the latent class information in texts, treating it as explicit class labels. It refines the graph convolution process by adding label-aware nodes to capture document–word, word–word, and word–class correlations for text classification. Comparing LaGCN with leading-edge models like HDGCN and BERT, our experiments on Ohsumed, Movie Review, 20 Newsgroups, and R8 datasets demonstrate its superiority. LaGCN outperformed existing methods, showing average accuracy improvements of 19.47%, 10%, 4.67%, and 0.4%, respectively. This advancement underscores the importance of integrating class information into graph neural networks, setting a new benchmark for text classification tasks.

Sustainability Strategic Framing in Corporate Communication: Contextual Semantics of Twitter in the Energy Sector

Sustainability has become the prevailing paradigm, a model of conduct, and a central theme in social and media discourses within the past decade. However, it is still a marginal development concept in the interdisciplinary research field dealing with sustainability communication, especially with meaning-making processes in relevant communicative, for example, in corporate and industry contexts. This paper focuses on how the idea of sustainability is framed, anchored, and disseminated in energy companies' communication via social media channels and how the linguistic methods used by energy corporations concerning sustainability co -create theirsymbolic value. Specifically, this study has aimed to understand how the sustainability topic was discussed in the Twitter sphere of the most prominent energy companies in 2020. This study makes a three -fold contribution to sustainable research and practice. First, the study identifies the framing structure of sustainability content in social media communication in the energy sector. Secondly, the study identifies social media data as a viable source of sustainability conceptualization and its understanding through communication. Finally, this study illustrates how text mining and natural language processing (NLP) can be used as a research method for managing big text data and a tool for discovering latent communication structures. Sustainability is present and visible in the communication of the energy industry identified by (1) the semantic construction of phrases in communication that show a future, green, and clean orientation, but it is also strongly declarative, referring to what will be and can be done, and with positive emotions related to the communicated messages; and (2) decoupling from climate change and thus avoiding the industry's responsibility for the impact of global warming but promising clean, sustainable energy transformation and to avoid negative emotional connotation. However, one gets the impression that it takes on the character of a rhetorical art form related to the concept of decorum. In this view, the idea of sustainability is interpreted and shaped through the needs of the industry and as a rhetorical tool in building a positive narrative about companies and is used to gain legitimacy.

Forecasting Business Status of Organizations by Analyzing Historic Earnings Call Transcripts with the Aid of Text Refinement Framework

Objectives: The research work focuses on providing an effective framework for automated text refinement that aids in financial condition projections for the company based on prior transcripts of earnings calls. The proposed framework captures the ad hoc advancements of the organizations described in the earnings call as sentiments and computes a score based on the captured sentiments. The sentiment score is then used as prime parameter to predict the stock values of the organizations. Methods: The framework is equipped with sentiment analysis or opinion mining to identify and extract the subjective content using text mining and Natural Language Processing (NLP). The extracted sentiments help in yielding a sentiment score to aid in the process of stock projection. The research also illustrates how the sentiment score-based stock prediction enhances in projections of stock compared to existing ML frameworks like LSTM, Random Forest, ARIMA and Regression models. Findings: The proposed work has an accuracy score of 93%, precision 96% and recall 95% which is comparatively better than existing ML frameworks framed on LSTM, Random Forest, ARIMA and Regression models. Novelty: The research framework overcomes the influence of regular features and test data in stock prediction by using the computed sentiment prediction score from the extracted sentiment phase to aid in prediction stock values and determine the financial status of organizations. The existing frameworks project the stock price based on trained model from previous stock price repository, which tend to fail capturing ad hoc changes incurring in the organization such as change of management or any economic disaster which can poses a high impact on stock projections, the proposed research work captures these organizational changes from the earnings call transcripts as sentiments and build a score to yield the stock projection framework. Keywords: Natural Language Processing, Text Refinement, Loughran McDonald Sentiment Classifier, Term Frequency Inverse Document Frequency, Stock Price

Deciphering Tourism’s Role in Antarctica’s Geosocial Concerns through Data Mining Techniques

This study explores the changing dynamics of tourism in Antarctica, focusing on the impact of digitalisation and User-Generated Content on platforms like Tripadvisor. It aims to understand how online reviews influence perceptions and decisions to visit Antarctica, a region known for its pristine environment and status as ‘the last frontier’. Utilising Environmental Perception and Behaviour Geography (EPBG) principles, this research conducts a quantitative analysis of reviews from potential and current travellers. Through text mining, topic modelling, sentiment analysis, and Natural Language Processing (NLP), it investigates the emotional and perceptual discourse surrounding Antarctic tourism and its alignment with Agenda 2030 and Sustainable Development Goals. The findings reveal a detailed narrative of sustainability challenges and the emotional geography related to tourism in Antarctica, highlighting emotions such as happiness, anger, surprise, fear, disgust, and sadness among visitors. This study uncovers differences in perception based on visitors’ backgrounds, noting that individuals from nature-focused cities display strong environmental concerns, whereas those from advanced urban centres show a more positive attitude. This research contributes to the understanding of EPBG, text mining, and NLP, offering insights into sustainable tourism practices in Antarctica.

Exploring user reactions to luxury brand videos on YouTube: a comparative study of influencers and brand-official channels

This study offers empirical evidence about individual reactions to brand-related videos posted on YouTube in the luxury context. We use methods of text mining and natural language processing to analyze the characteristics of the posts by people commenting on videos provided by influencers vs. brands’ official YouTube channels. We mined emotions and sentiments and evaluated the complexity and richness of the language. Data on 282 videos and more than 160,000 comments suggest that users display an impulsive reaction to a brand’s official video and a thoughtful reaction to an influencer’s video posted. In fact, in the former case, comments are short, simple, and enthusiastic, while in the latter comments are longer, more complex, and more moderate in terms of positive emotions. This study participates to the debate on the efficacy of influencer marketing to engage customers, offers empirical evidence in the online communication arena of luxury brands, and methodologically suggests a new method to accurately capture users’ engagement in the discussion spurred by publishing a brand-related video.

Building Urban Resilience through Mega-Events: A Systematic Review using Text Mining and Natural Language Processing (NLP)

This research aims to investigate the relationship between urban resilience and urban mega-events and to gain a deeper understanding of the ways in which effective planning, sustainable development, and social capital can be utilized to enhance urban resilience. The research will seek to explore how urban mega-events can be planned in ways that not only have significant economic impacts but also promote strategies for enhancing resilience in cities. To investigate the relationship between urban resilience and mega-events, a systematic review of the existing literature (2500 articles) was carried out. To investigate the relationship between urban resilience and mega-events, a systematic review of 2,500 articles was conducted. However, it should be noted that the limitations of qualitatively reviewing such a large number of publications must be acknowledged. Therefore, the review process involved differentiating between keyword analysis and systematic literature analysis for identifying success factors and good practices. Subsequently, eleven articles on different aspects of urban resilience and mega-events were subjected to lexical analysis (text mining) in order to measure and quantify the sentiment expressed in the articles. It is acknowledged that the number of articles included in the corpus is limited to 11. However, efforts have been made to ensure that the sample is representative of the wider corpus, with articles selected according to the following criteria: theme, keywords, and bibliometric search results. A hierarchical analysis was also employed to group the articles into comparable groups. The results of this analysis provided a more comprehensive understanding of the prevailing views on urban resilience, the implications of mega-events on urban areas, and potential strategies for enhancing the resilience of cities.

Leveraging text mining and analytic hierarchy process for the automatic evaluation of online courses

This study introduced a multi-criteria decision-making methodology leveraging text mining and analytic hierarchy process (AHP) for online course quality evaluation based on students’ feedback texts. First, a hierarchical structure of online course evaluation criteria was formulated by integrating topics (sub-criteria) identified through topic modeling and interpreted based on transactional distance and technology acceptance theories. Second, the weights of the criteria in the hierarchical structure were determined based on topic proportions. Third, the AHP was employed to determine the overall relative advantage of online courses and their relative advantage within each criterion based on the hierarchical framework and criterion weights. The proposed approach was implemented on the datasets of 6940 reviews for knowledge-seeking courses in Art, Design, and Humanities (D1) and 44,697 reviews for skill-seeking courses in Computer Science, Engineering, and Programming (D2) from Class Central to determine ranking positions of nine courses from both D1 and D2 as alternatives. Results revealed common concerns among knowledge and skill-seeking course learners, encompassing “assessment”, “content”, “effort”, “usefulness”, “enjoyment”, “faculty”, “interaction”, and “structure”. The article provides valuable insights into the online course evaluation and selection processes for learners in D1 and D2 groups. Notably, both groups prioritize “effort” and “faculty”, while D2 learners value “assessment” and “enjoyment”, and D1 learners value “usefulness” more. This study demonstrates the efficacy of leveraging online learner reviews and topic modeling for automating MOOC evaluation and informing learners’ decision-making processes.

Species Name Identification for Essential Oils from Biomedical Abstracts Using Text Mining and Natural Language Processing

Species Name Identification for Essential Oils from Biomedical Abstracts Using Text Mining and Natural Language Processing

Unraveling Tourists’ Experience of a Cremation Site: A Text Mining Approach

This article investigates the experiences of tourists visiting a cremation site, an unconventional tourist destination. By analyzing online reviews collected from TripAdvisor, using text mining and natural language processing (NLP) techniques, the study aims to fill the gaps in the literature on tourists’ experiences at cremation sites. The study explores the differences in the experiences of Indian domestic tourists and those from other cultural backgrounds. This study applies the views of Urry’s gaze theory and Stone’s dark tourism classification theory to discuss the tourists’ experiences. The results reveal significant differences in the experiences of Indian and international tourists. The emotion of foreign tourists lies with the existential and philosophical dimension, and Indian tourists are more connected with religious and mythological dimensions. The results validate Urry’s gaze theory and disagree with Stone’s dark tourism classification. This research contributes to the existing knowledge of dark tourism by providing valuable insights into tourists’ experiences at unconventional sites. Furthermore, the study highlights the importance of location managers understanding the significance of tourists’ experiences at sites to create enjoyable tours for domestic and international visitors. The article will benefit casual vacationers, academics, and scholars specializing in dark tourism, travelers??? narratives, online reviews, NLP, and trip organizers.

Building Urban Resilience through Mega-Events: A Systematic Review using Text Mining and Natural Language Processing (NLP)

This research aims to explore the link between urban resilience and urban mega-events and to better understand how effective planning, sustainable development, and social capital can be used to enhance urban resilience. The research will seek to explore how urban mega-events can be planned in ways that not only explode economic impacts but also promote strategies for enhancing resilience in cities. To investigate the relationship between urban resilience and mega-events, a systematic review of the existing literature (2500 articles) was carried out. To investigate the relationship between urban resilience and mega-events, we conducted a systematic review of 2500 articles, but we acknowledge the limitations of qualitatively reviewing such a large number of publications. Therefore, we differentiated between keyword analysis and systematic literature analysis for identifying success factors and good practices. After that, eleven articles on different aspects of urban resilience and mega-events were analyzed using lexical analysis (text mining) to measure and quantify the sentiment of the articles. We are conscious of the fact that 11 articles are not enough for the corpus analyzed, but we have tried to make a sample and a selection according to the theme, the keywords, and the most cited in the bibliometric search so that this selection is as representative as possible. A hierarchical analysis was also used to group the articles into comparable groups. The results of this analysis provided a better understanding of common views on urban resilience, the implications of mega-events on urban areas, and ways to improve the resilience of cities. According to the findings, adequate organization and action planning are essential for the successful development of cities hosting mega-events. In particular, the implementation of sustainability initiatives is imperative to ensure long-term resilience. In conclusion, future research must focus on the current understanding of mega-events and their influence on urban resilience and look at their long-term impacts. It should be conducted in different contexts, taking into account the various aspects of equity involved in these events.

A Flexible Big Data System for Credibility-Based Filtering of Social Media Information According to Expertise

Nowadays, social networks have taken on an irreplaceable role as sources of information. Millions of people use them daily to find out about the issues of the moment. This success has meant that the amount of content present in social networks is unmanageable and, in many cases, fake or non-credible. Therefore, a correct pre-processing of the data is necessary if we want to obtain knowledge and value from these data sets. In this paper, we propose a new data pre-processing technique based on Big Data that seeks to solve two of the key concepts of the Big Data paradigm, data validity and credibility of the data and volume. The system is a Spark-based filter that allows us to flexibly select credible users related to a given topic under analysis, reducing the volume of data and keeping only valid data for the problem under study. The proposed system uses the power of word embeddings in conjunction with other text mining and natural language processing techniques. The system has been validated using three real-world use cases.

A systematic literature review of hate speech identification on Arabic Twitter data: research challenges and future directions.

The automatic speech identification in Arabic tweets has generated substantial attention among academics in the fields of text mining and natural language processing (NLP). The quantity of studies done on this subject has experienced significant growth. This study aims to provide an overview of this field by conducting a systematic review of literature that focuses on automatic hate speech identification, particularly in the Arabic language. The goal is to examine the research trends in Arabic hate speech identification and offer guidance to researchers by highlighting the most significant studies published between 2018 and 2023. This systematic study addresses five specific research questions concerning the types of the Arabic language used, hate speech categories, classification techniques, feature engineering techniques, performance metrics, validation methods, existing challenges faced by researchers, and potential future research directions. Through a comprehensive search across nine academic databases, 24 studies that met the predefined inclusion criteria and quality assessment were identified. The review findings revealed the existence of many Arabic linguistic varieties used in hate speech on Twitter, with modern standard Arabic (MSA) being the most prominent. In identification techniques, machine learning categories are the most used technique for Arabic hate speech identification. The result also shows different feature engineering techniques used and indicates that N-gram and CBOW are the most used techniques. F1-score, precision, recall, and accuracy were also identified as the most used performance metric. The review also shows that the most used validation method is the train/test split method. Therefore, the findings of this study can serve as valuable guidance for researchers in enhancing the efficacy of their models in future investigations. Besides, algorithm development, policy rule regulation, community management, and legal and ethical consideration are other real-world applications that can be reaped from this research.

Agendas of Reform: Continuity and Change in Australia’s National Disability Insurance Scheme (NDIS)

This article used text mining processes to map continuity and change in policy principles of the National Disability Insurance Scheme (NDIS) in Australia and reflect on the underlying agendas of reform. Specifically, this research aimed to: (1) examine the substantive content of the NDIS legislative, operational and reform documentation, (2) assess changes in objects, principles and issues over time, and (3) discuss the implications of these shifting logics and agendas. To achieve this, text mining approaches were applied to ten key NDIS documents from 2011 to 2019. The findings included a low prevalence of ‘rights’-based terminology, sustained attention to Scheme ‘costs’ but limited latterly attention to ‘sustainability’, and increasing prominence of interpersonal (e.g., ‘family’, ‘community’) and decision-making terminology (e.g., ‘decision’, ‘review’). How these shifts have influenced subsequent proposed reforms is explored and ongoing dilemmas about designing policy that ensures rights and entitlements, while balancing cost, sustainability and consistency are identified.

Copyright, text & data mining and the innovation dimension of generative AI

Abstract The rise of Generative AI has raised many questions from the perspective of copyright. From the lens of copyright and database rights, issues revolve not only around the authorship of AI-generated outputs, but also the very process that leads to the generation of these outputs, namely the process of text and data mining (TDM). Does unauthorized TDM process infringe the economic rights of the rightholders? How does the TDM-debate transform and transmute in the age of Generative AI? Generative AI tools create works that substitute the content creators whose very work that they learn from, and successively improvise themselves with every iteration. Generative AI, thus, also presents larger policy question as they substitute the romanticized human author that sits at the centre of copyright. In addition, as Generative AI tools, such as ChatGPT, can now also crawl the web, questions thus transcend the frontiers of copyright, and touch upon innovation and competition in the market for web browsers. This research article contemplates on the foregoing issues, and makes some recommendations to create a balanced framework, whereby incentives to innovate are preserved, and the interests of the human author are suitably safeguarded in the age of TDM and Generative AI.