Field Of Text Mining Research Articles

Review of Survey Research in Fuzzy Approach for Text Mining

Text mining has been common research field, with the emergence of Web 2.0 and the development of social software, the amount of text generated every day has increased dramatically. The texts contain a lot of valuable information, how to analysis for the information from text is very important. Therefore, many research have explored related methods and various fields of text mining, such as sentiment analysis, text clustering, text summarization, etc., However, unlike other numerical data that can be directly calculated in terms of character performance, the calculation must be performed after vector conversion, and the words may have polysemy, which provides more challenges for natural language processing. Due to the above challenges, various techniques are used for data preprocessing before analysis. In addition to the common statistical and discrete methods for text data, methods based on fuzzy logic provide another option for effective natural language analysis, Therefore, in recent years, more and more studies have added Fuzzy logic to additionally capture the context semantics of individual words to help more accurate natural language processing. This survey research discusses multiple text mining methods, The survey paper discusses multiple text mining methods, subfields and application fields, covering the literature published between 2010 and 2022, It is organized to the subtasks to be performed, the methods and natural language processing techniques used, and the application scenarios. At the end of this study, the research has provided the key point discussion and relevant suggestions for text mining combined with fuzzy logic.

Topic Modelling and Artificial Intelligence based Method Using Online Employee Assessments to Analyse Job Satisfaction

In this study, the performance of the proposed sample selection method was evaluated on some basic classifiers by conducting a basic literature review on the use of topic modelling methods by considering the online evaluations of the employees in order to determine and analyse the job satisfaction factors. In addition, the effectiveness of different representation structures is evaluated in order to represent the data sets effectively and the main results are obtained regarding the use of classification ensemble methods in the field of text mining. In this work it was emphasized that machine learning methods can achieve high performance in classification and work effectively and scalable with large data sets. The dataset used in this study was obtained from www.kaggle.com. A total of 67529 comments collected from people working at Google, Amazon, Netflix, Facebook, Apple, and Microsoft were evaluated. Within the scope of this study, a text mining and artificial intelligence-based method will be developed, and a solution will be brought to text mining with artificial intelligence methods.

Intention-guided deep semi-supervised document clustering via metric learning

The intention expresses the user’s preference for document structure division. Intention-guided document structure division is an important task in the field of text mining. To achieve this goal, deep semi-supervised document clustering provides a promising solution to personalized document clustering. However, traditional deep semi-supervised clustering models suffer from the problem of the limited number of constraints which is insufficient for intention-guided document clustering. Moreover, documents normally have various emphases on their representations to reflect different structural opinions. In this paper, we proposed an intention-guided deep semi-supervised document clustering model, namely IGSC, to divide document structure based on a small amount of user-provided supervised information. IGSC designs a deep metric learning network to solve the above problems. The deep metric learner explores the user’s global intention and outputs an intention matrix. The intention is explored from the small amount user provided pairwise constraints and is used to guide the representation learning. Moreover, IGSC uses the intention matrix to guide the clustering process, to get the clustering results that best meet the user’s intention. This paper compares IGSC with a number of document clustering models on four real-world text datasets, namely Reu-10k, BBC, ACM, and Abstract. The results show that IGSC evidently improves the clustering performance and outperforms the best result of benchmark models with 7% on average. The comparison with other models and the visualization results can demonstrate that IGSC is effective.

Topic Modeling with Transformers for Sentence-Level Using Coronavirus Corpus

A Topic Model is a class of generative probabilistic models which has gained widespread use in computer science in recent years, especially in the field of text mining and information retrieval. Since it was first proposed, it has received a large amount of attention and general interest among scientists in many research areas. It allows us to discover the mix of hidden or "latent" subjects that differs from one document to another in a given corpus. But since topic modeling usually requires the prior definition of some parameters - above all the number of topics k to be discovered -, model evaluation is decisive to identify an "optimal" set of parameters for the specific data. Latent Dirichlet allocation (LDA) and Bidirectional Encoder Representations from Transformers Topic (BerTopic) are the two most popular topic modeling techniques. LDA uses a probabilistic approach whereas BerTopic uses transformers (BERT embeddings) and class-based TF-IDF to create dense clusters.

BertSRC: transformer-based semantic relation classification

The relationship between biomedical entities is complex, and many of them have not yet been identified. For many biomedical research areas including drug discovery, it is of paramount importance to identify the relationships that have already been established through a comprehensive literature survey. However, manually searching through literature is difficult as the amount of biomedical publications continues to increase. Therefore, the relation classification task, which automatically mines meaningful relations from the literature, is spotlighted in the field of biomedical text mining. By applying relation classification techniques to the accumulated biomedical literature, existing semantic relations between biomedical entities that can help to infer previously unknown relationships are efficiently grasped. To develop semantic relation classification models, which is a type of supervised machine learning, it is essential to construct a training dataset that is manually annotated by biomedical experts with semantic relations among biomedical entities. Any advanced model must be trained on a dataset with reliable quality and meaningful scale to be deployed in the real world and can assist biologists in their research. In addition, as the number of such public datasets increases, the performance of machine learning algorithms can be accurately revealed and compared by using those datasets as a benchmark for model development and improvement. In this paper, we aim to build such a dataset. Along with that, to validate the usability of the dataset as training data for relation classification models and to improve the performance of the relation extraction task, we built a relation classification model based on Bidirectional Encoder Representations from Transformers (BERT) trained on our dataset, applying our newly proposed fine-tuning methodology. In experiments comparing performance among several models based on different deep learning algorithms, our model with the proposed fine-tuning methodology showed the best performance. The experimental results show that the constructed training dataset is an important information resource for the development and evaluation of semantic relation extraction models. Furthermore, relation extraction performance can be improved by integrating our proposed fine-tuning methodology. Therefore, this can lead to the promotion of future text mining research in the biomedical field.

Analysis of Big Data Methodology for Pattern Recognition

Abstract: Sentiment Analysis (SA) is an ongoing field of research in text mining field. SA is the computational treatment of opinions, sentiments and subjectivity of text. This survey paper tackles a comprehensive overview of the last update in this field. Many recently proposed algorithms' enhancements and various SA applications are investigated and presented briefly in this survey. These articles are categorized according to their contributions in the various SA techniques. The related fields to SA (transfer learning, emotion detection, and building resources) that attracted researchers recently are discussed.

Target-specific sentiment analysis method combining word-masking data enhancement and adversarial learning

Abstract Target-specific sentiment analysis is an emerging topic in the field of text mining but current approaches to deriving the polarity of a sentence suffer from two main drawbacks: on one hand, we lack of a large and well-curated corpus, and on the other hand, current solutions based on deep learning are particularly vulnerable to the attack of adversarial samples. A novel target-specific sentiment classification method is proposed. Firstly, the method of masking target entities is applied to replace synonyms and insert words randomly; secondly, the target-specific sentiment classification model of adversarial learning is constructed with six baseline models; finally, we combine data enhancement and adversarial learning to construct target-specific sentiment classification model. Experimental results show that Macro-F1 values are improved by 0.30–2.91, 0.88–2.42 and 0.13–1.94% compared to the six baseline models by using Laptop14, Restaurant14 and Twitter original datasets, respectively, using Adversarial learning. Using word-masking data enhancement samples and Adversarial learning from Laptop14, Restaurant14 and Twitter shows that Macro-F1 values are improved by 0.9–2.64, 1.59–3.09 and 0.18–1.71% compared to the six baseline (SC), respectively. Our method can effectively improve the quality of samples, it improves the classification performance and the capability of adversarial samples defense.

Comparison of Sentiment Analysis Algorithms Using Twitter and Review Dataset

Abstract: Sentiment Analysis or opinion mining is the computational examination of sentiments, perspectives and emotions conveyed in written language. In recent years, sentiment analysis has become an active study in fields of natural language processing and text mining. The internet is full of textual data in the form of papers, articles and blogs. Sentiment analysis is the best way to extract key information from this data. In this paper we compare classification of tweets using rule based methods, Machine Learning method and Deep Learning method. Keywords: Sentiment Analysis, Opinion Mining, Twitter Dataset, Classification, Natural Language Processing.

An Improved Self Attention Mechanism Based on Optimized BERT-BiLSTM Model for Accurate Polarity Prediction

Abstract Polarity prediction is the field of study that discovers people’s opinions, feelings, assessments, perspectives and feelings about associations and their attributes as communicated in written text. It is one of the most active research areas in the field of text mining. Nowadays online reviews play an important role by giving a helping hand to the customers to know about other customer’s opinions about the product they are going to purchase. This also guides the organizations and government sectors to increase their quality of product and services. Pre-trained BERT (Bidirectional Encoder Representations from Transformers) is used for word embedding in this model. The fine-tuned BERT is used for better word representation which in turn improves the sentimental analysis classification accuracy. Bidirectional Long Short-Term Memory classifier is utilized for polarity prediction. To enhance the performance of Bidirectional Long Short-Term Memory, the weight parameters of Bi-directional LSTM are optimally selected by using APSO algorithm. Improved self-attention mechanism is added with BiLSTM for focusing on significant words in the context. For performance analysis, four bench mark datasets are used for experiments.

Domain-Specific Keyword Extraction Using Joint Modeling of Local and Global Contextual Semantics

Domain-specific keyword extraction is a vital task in the field of text mining. There are various research tasks, such as spam e-mail classification, abusive language detection, sentiment analysis, and emotion mining, where a set of domain-specific keywords (aka lexicon) is highly effective. Existing works for keyword extraction list all keywords rather than domain-specific keywords from a document corpus. Moreover, most of the existing approaches perform well on formal document corpuses but fail on noisy and informal user-generated content in online social media. In this article, we present a hybrid approach by jointly modeling the local and global contextual semantics of words, utilizing the strength of distributional word representation and contrasting-domain corpus for domain-specific keyword extraction. Starting with a seed set of a few domain-specific keywords, we model the text corpus as a weighted word-graph. In this graph, the initial weight of a node (word) represents its semantic association with the target domain calculated as a linear combination of three semantic association metrics, and the weight of an edge connecting a pair of nodes represents the co-occurrence count of the respective words. Thereafter, a modified PageRank method is applied to the word-graph to identify the most relevant words for expanding the initial set of domain-specific keywords. We evaluate our method over both formal and informal text corpuses (comprising six datasets), and show that it performs significantly better in comparison to state-of-the-art methods. Furthermore, we generalize our approach to handle the language-agnostic case, and show that it outperforms existing language-agnostic approaches.

Machine learning in exploration the decoupling paradigm in transport

The article deals with one of the paradigms of sustainable development, i.e. decoupling. The article focuses on machine learning in the field of text mining on decoupling in transport. The research problem was formulated in the form of a question: does machine learning improve the process of exploring information contained in scientific publications on decoupling in transport? The article aims to explore the decoupling paradigm in transport in tasks of natural language processing with the use of machine learning. In the article, research was conducted in the field of text mining and emotion profiler (by Plutchnik emotions), using lemmatization and techniques such as word cloud, a bag of words, Latent Dirichlet Allocation, RAKE (rapid automatic keyword extraction) algorithm, and hierarchical clustering. Important keywords, phrases for the description of the decoupling paradigm, methods of its research, and words that induce increased trust in this concept were found. The authors proposed an original framework for the research procedure.

Concept for the identification of product innovation potentials by the application of text mining

Within the product innovation process, companies are required to design their product according to diverse external influence dimensions from the product environment. By the analysis of these influence dimensions, companies gain insight into the urgency or possibility to innovate their product accordingly. As the majority of data within the business context available is text data, there is a need to formulate a method that enables companies to evaluate the data relevant for the product innovation accordingly. As the manual evaluation of this data is not feasible due to the high data amount, especially for small and medium sized companies, a concept for an automated evaluation method is required. This concept uses approaches from the field of text mining and applies them to the innovation management in order to gain insight from diverse texts about innovation potentials. This concept includes the definition of a suitable preprocessing, a topic modeling approach for this use case and a collection of options for the topic exploration. The preprocessing defines how usually occurring text documents can be converted into a format that is manageable for the text mining approach. The topic modeling is based on a constrained TF-IDF- weighted Latent Dirichlet Allocation to identify preferably new and unique topics within the considered dataset. Afterwards, the identified topics can be explored in differentiated ways in order to gain a better insight into the product environment. The validation of this method on three sample datasets tests its limitations but indicates also the potential of an automated text analysis for innovation management.

A Comprehensive Guideline for Bengali Sentiment Annotation

Sentiment Analysis (SA) is a Natural Language Processing (NLP) and an Information Extraction (IE) task that primarily aims to obtain the writer’s feelings expressed in positive or negative by analyzing a large number of documents. SA is also widely studied in the fields of data mining, web mining, text mining, and information retrieval. The fundamental task in sentiment analysis is to classify the polarity of a given content as Positive, Negative, or Neutral . Although extensive research has been conducted in this area of computational linguistics, most of the research work has been carried out in the context of English language. However, Bengali sentiment expression has varying degree of sentiment labels, which can be plausibly distinct from English language. Therefore, sentiment assessment of Bengali language is undeniably important to be developed and executed properly. In sentiment analysis, the prediction potential of an automatic modeling is completely dependent on the quality of dataset annotation. Bengali sentiment annotation is a challenging task due to diversified structures (syntax) of the language and its different degrees of innate sentiments (i.e., weakly and strongly positive/negative sentiments). Thus, in this article, we propose a novel and precise guideline for the researchers, linguistic experts, and referees to annotate Bengali sentences immaculately with a view to building effective datasets for automatic sentiment prediction efficiently.

Performance Analysis of Hybrid Deep Learning Models with Attention Mechanism Positioning and Focal Loss for Text Classification

Over the past few decades, text classification problems have been widely utilized in many real time applications. Leveraging the text classification methods by means of developing new applications in the field of text mining and Natural Language Processing (NLP) is very important. In order to accurately classify tasks in many applications, a deeper insight into deep learning methods is required as there is an exponential growth in the number of complex documents. The success of any deep learning algorithm depends on its capacity to understand the nonlinear relationships of the complex models within data. Thus, a huge challenge for researchers lies in the development of suitable techniques, architectures, and models for text classification. In this paper, hybrid deep learning models, with an emphasis on positioning of attention mechanism analysis, are considered and analyzed well for text classification. The first hybrid model proposed is called convolutional Bidirectional Long Short-Term Memory (Bi-LSTM) with attention mechanism and output (CBAO) model, and the second hybrid model is called convolutional attention mechanism with Bi-LSTM and output (CABO) model. In the first hybrid model, the attention mechanism is placed after the Bi-LSTM, and then the output Softmax layer is constructed. In the second hybrid model, the attention mechanism is placed after convolutional layer and followed by Bi-LSTM and the output Softmax layer. The proposed hybrid models are tested on three datasets, and the results show that when the proposed CBAO model is implemented for IMDB dataset, a high classification accuracy of 92.72% is obtained and when the proposed CABO model is implemented on the same dataset, a high classification accuracy of 90.51% is obtained.

A mathematical comparison of non-negative matrix factorization related methods with practical implications for the analysis of mass spectrometry imaging data.

RationaleNon‐negative matrix factorization (NMF) has been used extensively for the analysis of mass spectrometry imaging (MSI) data, visualizing simultaneously the spatial and spectral distributions present in a slice of tissue. The statistical framework offers two related NMF methods: probabilistic latent semantic analysis (PLSA) and latent Dirichlet allocation (LDA), which is a generative model. This work offers a mathematical comparison between NMF, PLSA, and LDA, and includes a detailed evaluation of Kullback–Leibler NMF (KL‐NMF) for MSI for the first time. We will inspect the results for MSI data analysis as these different mathematical approaches impose different characteristics on the data and the resulting decomposition.MethodsThe four methods (NMF, KL‐NMF, PLSA, and LDA) are compared on seven different samples: three originated from mice pancreas and four from human‐lymph‐node tissues, all obtained using matrix‐assisted laser desorption/ionization time‐of‐flight mass spectrometry (MALDI‐TOF MS).ResultsWhere matrix factorization methods are often used for the analysis of MSI data, we find that each method has different implications on the exactness and interpretability of the results. We have discovered promising results using KL‐NMF, which has only rarely been used for MSI so far, improving both NMF and PLSA, and have shown that the hitherto stated equivalent KL‐NMF and PLSA algorithms do differ in the case of MSI data analysis. LDA, assumed to be the better method in the field of text mining, is shown to be outperformed by PLSA in the setting of MALDI‐MSI. Additionally, the molecular results of the human‐lymph‐node data have been thoroughly analyzed for better assessment of the methods under investigation.ConclusionsWe present an in‐depth comparison of multiple NMF‐related factorization methods for MSI. We aim to provide fellow researchers in the field of MSI a clear understanding of the mathematical implications using each of these analytical techniques, which might affect the exactness and interpretation of the results.

Learning from High-Dimensional and Class-Imbalanced Datasets Using Random Forests

Class imbalance and high dimensionality are two major issues in several real-life applications, e.g., in the fields of bioinformatics, text mining and image classification. However, while both issues have been extensively studied in the machine learning community, they have mostly been treated separately, and little research has been thus far conducted on which approaches might be best suited to deal with datasets that are class-imbalanced and high-dimensional at the same time (i.e., with a large number of features). This work attempts to give a contribution to this challenging research area by studying the effectiveness of hybrid learning strategies that involve the integration of feature selection techniques, to reduce the data dimensionality, with proper methods that cope with the adverse effects of class imbalance (in particular, data balancing and cost-sensitive methods are considered). Extensive experiments have been carried out across datasets from different domains, leveraging a well-known classifier, the Random Forest, which has proven to be effective in high-dimensional spaces and has also been successfully applied to imbalanced tasks. Our results give evidence of the benefits of such a hybrid approach, when compared to using only feature selection or imbalance learning methods alone.

Modern Clinical Text Mining: A Guide and Review.

Electronic health records (EHRs) are becoming a vital source of data for healthcare quality improvement, research, and operations. However, much of the most valuable information contained in EHRs remains buried in unstructured text. The field of clinical text mining has advanced rapidly in recent years, transitioning from rule-based approaches to machine learning and, more recently, deep learning. With new methods come new challenges, however, especially for those new to the field. This review provides an overview of clinical text mining for those who are encountering it for the first time (e.g., physician researchers, operational analytics teams, machine learning scientists from other domains). While not a comprehensive survey, this review describes the state of the art, with a particular focus on new tasks and methods developed over the past few years. It also identifies key barriers between these remarkable technical advances and the practical realities of implementation in health systems and in industry.

Predicting real estate market trends and value using pre-processing and sentiment text mining analysis

Ubiquitous growth in the text mining field is unprecedented, where social media mining is playing a significant role. Gigantic growth of text mining is becoming a potential source of crowd wisdom extraction and analysis especially in terms of text pre-processing and sentiment analysis. The analysis of a potential influence of sentiment on real estate markets controversially discussed by scholars of finance, valuation and market efficiency supporters. Therefore, it’s a significant task of current research purview which not only provide an appropriate platform for the contributors but also for active real estate market information seekers. Text mining has gained the widespread attention of real estate market information users which is almost on explosion level. Accessibility of data on such behemoth scale mandates regular and critical analysis of this information for various perspectives’ plausibility. Rich patterns of online social text can be exploited to extract the relevant real estate information effectively. As text mining plays a significant and crucial role in discovery of these insights therefore its challenges and contribution in social media analysis must be explored extensively. In this paper, we provide a brief about the current summary of the modern state of text mining in pre-processing and sentiment for the real estate market analysis. Empha-sis is placed on the resources and learning mechanism available to real estate researchers and practitioners, as well as the major text mining tasks of interest to the community. Thus, the main aim of this chapter is to expound and intellectualize the domains of social media which are accessible on an extraordinary range in the field of text mining real estate for predicting real estate market trends and value.

An Empirical Analysis on Various Techniques Used to Detect the Polarity of Customer Satisfaction in Sentiment Analysis

Sentiment analysis is an emerging application of NLP (Natural Language Processing). This is also called opinion mining or attitude detection. In the text mining field, Sentiment Analysis is continuous area of research. It is a procedural treatment of attitude, feelings and textual content. The fundamental thought is to discover text polarity and order it as positive, negative, or neutral. It supports human to take a good judgment. This survey paper gives an extensive summary of the previous updates in this area. Several currently proposed algorithms and numerous upgrades to different SA applications have been investigated and summed up in this review. These articles are classified by their commitment to different SA techniques. Areas identified with SA (business monitoring, polarity observation, social media monitoring) that has recently attracted researchers is discussed. The fundamental objective of this survey is to provide a complete picture of SA practices and associated fields with brief explanation. The significant role of this study includes a refined classification of current papers and a depiction of ongoing patterns in sentiment analysis and research in its associated fields.

A new neutrosophic TF-IDF term weighting for text mining tasks: text classification use case

PurposeThis paper aims to present a new term weighting approach for text classification as a text mining task. The original method, neutrosophic term frequency – inverse term frequency (NTF-IDF), is an extended version of the popular fuzzy TF-IDF (FTF-IDF) and uses the neutrosophic reasoning to analyze and generate weights for terms in natural languages. The paper also propose a comparative study between the popular FTF-IDF and NTF-IDF and their impacts on different machine learning (ML) classifiers for document categorization goals.Design/methodology/approachAfter preprocessing textual data, the original Neutrosophic TF-IDF applies the neutrosophic inference system (NIS) to produce weights for terms representing a document. Using the local frequency TF, global frequency IDF and text N's length as NIS inputs, this study generate two neutrosophic weights for a given term. The first measure provides information on the relevance degree for a word, and the second one represents their ambiguity degree. Next, the Zhang combination function is applied to combine neutrosophic weights outputs and present the final term weight, inserted in the document's representative vector. To analyze the NTF-IDF impact on the classification phase, this study uses a set of ML algorithms.FindingsPracticing the neutrosophic logic (NL) characteristics, the authors have been able to study the ambiguity of the terms and their degree of relevance to represent a document. NL's choice has proven its effectiveness in defining significant text vectorization weights, especially for text classification tasks. The experimentation part demonstrates that the new method positively impacts the categorization. Moreover, the adopted system's recognition rate is higher than 91%, an accuracy score not attained using the FTF-IDF. Also, using benchmarked data sets, in different text mining fields, and many ML classifiers, i.e. SVM and Feed-Forward Network, and applying the proposed term scores NTF-IDF improves the accuracy by 10%.Originality/valueThe novelty of this paper lies in two aspects. First, a new term weighting method, which uses the term frequencies as components to define the relevance and the ambiguity of term; second, the application of NL to infer weights is considered as an original model in this paper, which also aims to correct the shortcomings of the FTF-IDF which uses fuzzy logic and its drawbacks. The introduced technique was combined with different ML models to improve the accuracy and relevance of the obtained feature vectors to fed the classification mechanism.