Unstructured Text Data Research Articles

Text Mining Based on the Lexicon-Constrained Network in the Context of Big Data

Unstructured textual news data is produced every day; analyzing them using an abstractive summarization algorithm provides advanced analytics to decision-makers. Deep learning network with copy mechanism is finding increasing use in abstractive summarization, because copy mechanism allows sequence-to-sequence models to choose words from the input and put them directly into the output. However, since there is no explicit delimiter in Chinese sentences, most existing models for Chinese abstractive summarization can only perform character copy, resulting in inefficiency. To solve this problem, we propose a lexicon-constrained copying network that models multigranularity in both encoder and decoder. On the source side, words and characters are aggregated into the same input memory using a Transformer-based encoder. On the target side, the decoder can copy either a character or a multicharacter word at each time step, and the decoding process is guided by a word-enhanced search algorithm which facilitates the parallel computation and encourages the model to copy more words. Moreover, we adopt a word selector to integrate keyword information. Experiment results on a Chinese social media dataset show that our model can work standalone or with the word selector. Both forms can outperform previous character-based models and achieve competitive performances.

Clinical Text Data Categorization and Feature Extraction Using Medical-Fissure Algorithm and Neg-Seq Algorithm.

A large amount of patient information has been gathered in Electronic Health Records (EHRs) concerning their conditions. An EHR, as an unstructured text document, serves to maintain health by identifying, treating, and curing illnesses. In this research, the technical complexities in extracting the clinical text data are removed by using machine learning and natural language processing techniques, in which an unstructured clinical text data with low data quality is recognized by Halve Progression, which uses Medical-Fissure Algorithm which provides better data quality and makes diagnosis easier by using a cross-validation approach. Moreover, to enhance the accuracy in extracting and mapping clinical text data, Clinical Data Progression uses Neg-Seq Algorithm in which the redundancy in clinical text data is removed. Finally, the extracted clinical text data is stored in the cloud with a secret key to enhance security. The proposed technique improves the data quality and provides an efficient data extraction with high accuracy of 99.6%.

Analyzing genderless fashion trends of consumers\u2019 perceptions on social media: using unstructured big data analysis through Latent Dirichlet Allocation-based topic modeling

After the development of Web 2.0 and social networks, analyzing consumers’ responses and opinions in real-time became profoundly important to gain business insights. This study aims to identify consumers’ preferences and perceptions of genderless fashion trends by text-mining, Latent Dirichlet Allocation-based topic modeling, and time-series linear regression analysis. Unstructured text data from consumer-posted sources, such as blogs and online communities, were collected from January 1, 2018 to December 31, 2020. We examined 9722 posts that included the keyword “genderless fashion” with Python 3.7 software. Results showed that consumers were interested in fragrances, fashion, and beauty brands and products. In particular, 18 topics were extracted: 13 were classified as fashion categories and 5 were derived from beauty and fragrance sectors. Examining the genderless fashion trend development among consumers from 2018 to 2020, “perfume and scent” was revealed as the hot topic, whereas “bags,” “all-in-one skin care,” and “set-up suit” were cold topics, declining in popularity among consumers. The findings contribute to contemporary fashion trends and provide in-depth knowledge about consumers’ perceptions using big data analysis methods and offer insights into product development strategies.

How We Failed in Context: A Text-Mining Approach to Understanding Hotel Service Failures

Service failure is inevitable. Although empirical studies on the outcomes and processes of service failures have been conducted in the hotel industry, the findings need more exploration to understand how different segments perceive service failures and the associated emotions differently. This approach enables hotel managers to develop more effective strategies to prevent service failures and implement more specific service-recovery actions. For analysis, we obtained a nine-year (2010–2018) longitudinal dataset containing 1224 valid respondents with 73,622 words of textual content from a property affiliated with an international hotel brand in Canada. A series of text-mining and natural language processing (NLP) analyses, including frequency analysis and word cloud, sentiment analysis, word correlation, and TF–IDF analysis, were conducted to explore the information hidden in the massive amount of unstructured text data. The results revealed the similarities and differences between groups (i.e., men vs. women and leisure vs. business) in reporting service failures. We also carefully examined different meanings of words that emerged from the text-mining results to ensure a more comprehensive understanding of the guest experience.

Discovering research topics, trends, and perspectives in COVID-19-related transportation journal articles

Research interest in COVID-19 in the transportation field has increased sporadically since its outbreak in 2019. This has led to an unprecedented increase in the number of publications in academic journals, rendering it difficult to clearly capture and understand the themes being discussed in the entire literature. This study employs a Structural Topic Model, a robust probabilistic topic model that incorporates document-level metadata to extract hidden topics in unstructured textual big data that focuses on COVID-19 and transportation. To understand the topics identified, the study examined the topical trends over time and compared them to provide insights into authors’ perspectives based on their country’s economic status. In total, abstracts from 421 research articles published in top transportation/transportation science journals were collected and analysed. The results reveal that the major academic concerns in the area of COVID-19 and transportation are related to the changing travel behaviour, airport financial performance, and supply chain optimisation. Overall, research trends seem to be shifting towards shipping emissions, air transport recovery, travel behaviour, and the performance of airports. In addition, authors from both high-income and middle-and low-income countries were found to have different perspectives regarding the topics identified. The findings from this study contribute to understanding topical trends and perspectives in the literature on COVID-19 and transportation and can be used by researchers, policymakers, and fund providers to recognise current research issues to guide future research direction and for making more informed policy decisions.

SenDemonNet: sentiment analysis for demonetization tweets using heuristic deep neural network.

Sentiment analysis is one of the efficient models for extracting opinion mining with identification and classification from unstructured text data such as product reviews or microblogs. It is used to gain feedback from political campaigns, brand reviews, marketing analysis, and customers. The sentiment analysis on Twitter data is a recent research field in the natural processing. The dataset is gathered from the “Twitter” package in R along with Twitter API. The main intent of this paper is to understand the public opinion on the recently implemented demonetization policy using the proposed SenDemonNet. Initially, the tweet preprocessing was done, which is intended for cleaning the text data. Then, the feature extraction is performed by Bag of n-grams, TF-IDF, and the word2vec algorithm. The main objective of this work is a weighted feature selection that is developed by the hybrid Forest–Whale Optimization Algorithm (F-WOA) to get the best classification outcome. With these features, the Heuristic Deep Neural Network (HDNN) is adopted for classification, where the proposed FOA and WOA tune the parameter of DNN for reaching the maximum accuracy rate. From the statistical analysis, the performance of the designed F-WOA-DNN is 1.8%, 1.9%, 1.86%, and 2% enhanced than PSO-DNN, GWO-DNN, WOA-DNN, FOA-DNN, SVM, CNN, LSTM, and DNN respectively. Extensive experimental results show that SenDemonNet outperforms its competitors, producing an impressive increase in the classification accuracy on the benchmark dataset.Graphical abstract Graphical architectural diagram of Sentiment analysis model on demonetization policy in India

데이터마이닝 기법을 활용한 관광목적지 이미지와 온라인 관광정보의 지각된 유용성과의 관계분석

This study seeks to analyze the relationship of tourism destination images and perceived usefulness of potential tourists in English reviews published on TripAdvisor, a post-travel reviews website belonging to online tourism information. The research method utilizes R programs, an open-source statistical software, to apply text mining techniques to quantify unstructured text data. First, a total of 25,016 reviews from 20 Seoul tourism destinations of TripAdvisor were collected with data mining techniques to extract the words, and a simple regression analysis was conducted with each extracted word. To test the hypothesis, a negative binary regression was conducted. The results of this study are as follows. First, it was confirmed that the tourism destination images affect perceived usefulness. Second, we were able to see the relationship among the tourism destination images. Finally, based on the results of these empirical analyses, we present academic and practical implications.

Improving Diabetes-Related Biomedical Literature Exploration in the Clinical Decision-making Process via Interactive Classification and Topic Discovery: Methodology Development Study.

BackgroundThe amount of available textual health data such as scientific and biomedical literature is constantly growing and becoming more and more challenging for health professionals to properly summarize those data and practice evidence-based clinical decision making. Moreover, the exploration of unstructured health text data is challenging for professionals without computer science knowledge due to limited time, resources, and skills. Current tools to explore text data lack ease of use, require high computational efforts, and incorporate domain knowledge and focus on topics of interest with difficulty.ObjectiveWe developed a methodology able to explore and target topics of interest via an interactive user interface for health professionals with limited computer science knowledge. We aim to reach near state-of-the-art performance while reducing memory consumption, increasing scalability, and minimizing user interaction effort to improve the clinical decision-making process. The performance was evaluated on diabetes-related abstracts from PubMed.MethodsThe methodology consists of 4 parts: (1) a novel interpretable hierarchical clustering of documents where each node is defined by headwords (words that best represent the documents in the node), (2) an efficient classification system to target topics, (3) minimized user interaction effort through active learning, and (4) a visual user interface. We evaluated our approach on 50,911 diabetes-related abstracts providing a hierarchical Medical Subject Headings (MeSH) structure, a unique identifier for a topic. Hierarchical clustering performance was compared against the implementation in the machine learning library scikit-learn. On a subset of 2000 randomly chosen diabetes abstracts, our active learning strategy was compared against 3 other strategies: random selection of training instances, uncertainty sampling that chooses instances about which the model is most uncertain, and an expected gradient length strategy based on convolutional neural networks (CNNs).ResultsFor the hierarchical clustering performance, we achieved an F1 score of 0.73 compared to 0.76 achieved by scikit-learn. Concerning active learning performance, after 200 chosen training samples based on these strategies, the weighted F1 score of all MeSH codes resulted in a satisfying 0.62 F1 score using our approach, 0.61 using the uncertainty strategy, 0.63 using the CNN, and 0.45 using the random strategy. Moreover, our methodology showed a constant low memory use with increased number of documents.ConclusionsWe proposed an easy-to-use tool for health professionals with limited computer science knowledge who combine their domain knowledge with topic exploration and target specific topics of interest while improving transparency. Furthermore, our approach is memory efficient and highly parallelizable, making it interesting for large Big Data sets. This approach can be used by health professionals to gain deep insights into biomedical literature to ultimately improve the evidence-based clinical decision making process.

Named entity recognition using neural language model and CRF for Hindi language

Named Entity Recognition (NER) plays an important role in various Natural Language Processing (NLP) applications to extract the key information from a huge amount of unstructured text data. NER is a task of identifying and classifying the named entities into predefined categories for a given text. Recently, language models are highly appreciable in several NLP tasks as these state-of-the-art models result better even in resource scarcity. In this paper, we perform NER task on the Hindi language by incorporating the recently released multilingual language model MuRIL which stands for Multilingual Representation for Indian Languages. MuRIL is specially trained for 16 Indian languages. We develop a Hindi NER system using MuRIL with a conditional random field (CRF) layer and fine-tune the model on the ICON 2013 Hindi NER dataset. Further, in the proposed approach, we compute the addition of the last 4 layers representations of the MuRIL model instead of just using the last layer's representation and fine-tune the whole model. Several variants of this model are presented by applying different computations on token representations provided by different layers of 12-layered MuRIL architecture. The proposed model achieves state-of-the-art results as 87.89% precision, 83.74% recall and 85.77% F1-score and outperforms all other existing Hindi NER systems developed on the ICON 2013 dataset. Additionally, we develop a similar Hindi NER system by replacing the MuRIL language model with another state-of-the-art language model, called multilingual Bidirectional Encoder Representations from Transformers (mBERT) to analyze the efficiency of both language models over the Hindi NER task.

MarkerGenie: an NLP-enabled text-mining system for biomedical entity relation extraction.

Natural language processing (NLP) tasks aim to convert unstructured text data (e.g. articles or dialogues) to structured information. In recent years, we have witnessed fundamental advances of NLP technique, which has been widely used in many applications such as financial text mining, news recommendation and machine translation. However, its application in the biomedical space remains challenging due to a lack of labeled data, ambiguities and inconsistencies of biological terminology. In biomedical marker discovery studies, tools that rely on NLP models to automatically and accurately extract relations of biomedical entities are valuable as they can provide a more thorough survey of all available literature, hence providing a less biased result compared to manual curation. In addition, the fast speed of machine reader helps quickly orient research and development. To address the aforementioned needs, we developed automatic training data labeling, rule-based biological terminology cleaning and a more accurate NLP model for binary associative and multi-relation prediction into the MarkerGenie program. We demonstrated the effectiveness of the proposed methods in identifying relations between biomedical entities on various benchmark datasets and case studies. MarkerGenie is available at https://www.genegeniedx.com/markergenie/. Data for model training and evaluation, term lists of biomedical entities, details of the case studies and all trained models are provided at https://drive.google.com/drive/folders/14RypiIfIr3W_K-mNIAx9BNtObHSZoAyn?usp=sharing. Supplementary data are available at Bioinformatics Advances online.

Classification and pattern extraction of incidents: a deep learning-based approach

Classifying or predicting occupational incidents using both structured and unstructured (text) data are an unexplored area of research. Unstructured texts, i.e., incident narratives are often unutilized or underutilized. Besides the explicit information, there exist a large amount of hidden information present in a dataset, which cannot be explored by the traditional machine learning (ML) algorithms. There is a scarcity of studies that reveal the use of deep neural networks (DNNs) in the domain of incident prediction, and its parameter optimization for achieving better prediction power. To address these issues, initially, key terms are extracted from the unstructured texts using LDA-based topic modeling. Then, these key terms are added with the predictor categories to form the feature vector, which is further processed for noise reduction and fed to the adaptive moment estimation (ADAM)-based DNN (i.e., ADNN) for classification, as ADAM is superior to GD, SGD, and RMSProp. To evaluate the effectiveness of our proposed method, a comparative study has been conducted using some state-of-the-arts on five benchmark datasets. Moreover, a case study of an integrated steel plant in India has been demonstrated for the validation of the proposed model. Experimental results reveal that ADNN produces superior performance than others in terms of accuracy. Therefore, the present study offers a robust methodological guide that enables us to handle the issues of unstructured data and hidden information for developing a predictive model.

STonKGs: a sophisticated transformer trained on biomedical text and knowledge graphs.

MotivationThe majority of biomedical knowledge is stored in structured databases or as unstructured text in scientific publications. This vast amount of information has led to numerous machine learning-based biological applications using either text through natural language processing (NLP) or structured data through knowledge graph embedding models. However, representations based on a single modality are inherently limited.ResultsTo generate better representations of biological knowledge, we propose STonKGs, a Sophisticated Transformer trained on biomedical text and Knowledge Graphs (KGs). This multimodal Transformer uses combined input sequences of structured information from KGs and unstructured text data from biomedical literature to learn joint representations in a shared embedding space. First, we pre-trained STonKGs on a knowledge base assembled by the Integrated Network and Dynamical Reasoning Assembler consisting of millions of text-triple pairs extracted from biomedical literature by multiple NLP systems. Then, we benchmarked STonKGs against three baseline models trained on either one of the modalities (i.e. text or KG) across eight different classification tasks, each corresponding to a different biological application. Our results demonstrate that STonKGs outperforms both baselines, especially on the more challenging tasks with respect to the number of classes, improving upon the F1-score of the best baseline by up to 0.084 (i.e. from 0.881 to 0.965). Finally, our pre-trained model as well as the model architecture can be adapted to various other transfer learning applications.Availability and implementationWe make the source code and the Python package of STonKGs available at GitHub (https://github.com/stonkgs/stonkgs) and PyPI (https://pypi.org/project/stonkgs/). The pre-trained STonKGs models and the task-specific classification models are respectively available at https://huggingface.co/stonkgs/stonkgs-150k and https://zenodo.org/communities/stonkgs.Supplementary information Supplementary data are available at Bioinformatics online.

Constituent vs Dependency Parsing-Based RDF Model Generation from Dengue Patients’ Case Sheets

Electronic Health Record (EHR) systems in healthcare organisations are primarily maintained in isolation from each other that makes interoperability of unstructured(text) data stored in these EHR systems challenging in the healthcare domain. Similar information may be described using different terminologies by different applications that can be evaded by transforming the content into the Resource Description Framework (RDF) model that is interoperable amongst organisations. RDF requires a document’s contents to be translated into a repository of triplets (subject, predicate, object) known as RDF statements. Natural Language Processing (NLP) techniques can help get actionable insights from these text data and create triplets for RDF model generation. This paper discusses two NLP-based approaches to generate the RDF models from unstructured patients’ documents, namely dependency structure-based and constituent(phrase) structure-based parser. Models generated by both approaches are evaluated in two aspects: exhaustiveness of the represented knowledge and the model generation time. The precision measure is used to compute the models’ exhaustiveness in terms of the number of facts that are transformed into RDF representations.

Trend Prediction Of Stock Industry Index Based On Financial Text

The deep learning model has achieved good results in index prediction, but most models only use structured data and cannot effectively predict mutation data. In order to better predict the sudden change data effectively, this paper proposes a new trend prediction model SEN_LSTM, which takes unstructured text data as a supplementary input and improves the performance of the model.

Blood Donation Narratives on Social Media: A Topic Modeling Study.

Social media have shown great potential for producing significant changes in behavior and have become the cornerstone for many public health and agency efforts. The nonprofit sector -including blood collection agencies- has adopted social media to aid their cause and reach their goals. However, despite the tremendous impact of social media on society and its promising role for donor recruitment and retention, it has been overlooked in donor research. This study therefore sought to map the social media landscape around blood donation. We showcase an inductive computational method to make sense of vast amounts of dynamic unstructured blood donation text data that exists on social media. With this method, we display what is discussed about blood donation on social media, how these topics are distributed on Facebook and Twitter, and how the prevalence of these topics changes over time. We applied structural topic modeling on 7 years of Dutch blood donation Facebook and Twitter data by the general public. We found 25 topics clustered in 6 distinct clusters. Over time, there is a substantial reduction of messages in which donors announce their donations. Topics that emphasize the positives of blood donation, including donor identity-related topics, are rising. In addition, the findings show a clear social media platform contrast. Topics related to campaigns and controversial policies were found more on Twitter and positive donation topics and topics related to the donation process more on Facebook. To make optimal use of social media for recruitment and retention campaigns and efforts, blood collection agencies should recognize the turbulent environment in which they take place. Monitoring public opinions about blood donation will help blood collection agencies make strategic choices and utilize social media more effectively.

RTN-GNNR: Fusing Review Text Features and Node Features for Graph Neural Network Recommendation

Recent recommendation systems have achieved good results by applying Graph Neural Network (GNN) to user-item interaction graphs. However, these recommendation systems can only handle structured interaction data and cannot handle unstructured review text data well. Based on the user-item interaction graph, combining review text can effectively solve the problem of data sparsity and improve recommendation quality. Most of the current recommendation methods combining review texts stitch the data from different modalities, leading to insufficient interactions and degrading the recommendations’ performance. A model called RTN-GNNR to fuse Review Text feature and Node feature for Graph Neural Network Recommendation is proposed to solve these problems and get better item recommendations. RTN-GNNR consists of four modules. The review text feature extraction module proposes a Bi-directional Gated Recurrent Unit (Bi-GRU) text analysis method that combines Bidirectional Encoder Representation from Transformers (BERT) and attention mechanism to enable the model to focus on more valuable reviews. The node feature extraction module proposes a GNN combined with the attention mechanism for the interactive node extraction method, which enables the model to have better higher-order feature extraction capability. The feature fusion module proposes the method of tandem Factorization Machine (FM) and Multilayer Perceptron (MLP) to realize interactive learning among multi-source features. The prediction module inner-products the fused higher-order features to achieve recommendation effect. We conducted experiments on five publicly available datasets from Amazon, showing that RTN-GNNR outperforms state-of-the-art personalized recommendation methods in both RMSE and MSE, especially in the sparser two datasets. The effectiveness of each module of the model is also demonstrated by a comparison of the ablation experiments.

Artificial intelligence guided predicting the length of hospital-stay in a neurosurgical hospital based on the text data of electronic medical records

Rational use of internal resources of hospitals including bed fund turnover is important objective in high-tech medicine. Machine learning technologies can improve neurosurgical care and contribute to patient-oriented approach. To evaluate the quality of AI-guided predicting the length of hospital-stay in a neurosurgical hospital based on the text data of electronic medical records in comparison with expectations of patients and physicians. AI-guided prediction was based on analysis of unstructured text records of the electronic medical history (preoperative examination and surgical protocol). Predictive models were learned on the data of the Burdenko Neurosurgery Center accumulated for the period 2000-2017 (90.688 cases). Model testing was performed on 111 completed neurosurgical cases in a prospective study. We compared the accuracy of prediction models compared to expectations of patients and physicians regarding hospital-stay. The median absolute error of machine prediction in the final test was 2.00 days. This value was comparable with the doctor's prediction error. This study demonstrated the possibility of using unstructured textual data to predict the length of hospital-stay in a neurosurgical hospital.

Defining characteristics of the most innovative companies

In this study, a text analytics approach was employed to analyse the mission statements of the most innovative companies to help uncover emerging trends and unlock the value of unstructured textual data. The tools that were used to conduct this analysis were R and SAS Enterprise Miner. The dataset that is used for this analysis contains the mission statements of the top 50 most innovative companies ranked by Boston Consulting Group. Valuable information was extracted by classifying, clustering, and visualising the most frequently appearing and significant terms found across the mission statements of the most innovative companies.

Sentiment analysis of facebook posts on Kolkata police page and public sentiment

This paper is an analysis of the comments posted on the facebook page of Kolkata Police through sentiment analysis tools to understand people's attitudes towards Kolkata Police during lockdown I & II in the state of West Bengal in India. These platforms keep people digitally connected. Social media also has given consumers a virtual soapbox, from where they can convey their opinions. Every day huge volumes of text data are getting generated on social platforms. While humans instantly understand the sentiment behind a single comment, how should we cope with the explosion of content shared on social platforms? It is extremely challenging to analyse the massive amount of unstructured text data manually. Here we need tools to help us. AI-powered sentiment analysis tools use natural language processing techniques to analyse the online conversation and underlying context - positive, neutral, or negative. Sentiment analysis tools mimic human brains and allow us to get the sentiment behind online content. In this study, researcher has used rule-based sentiment analysis techniques (TextBlob and Valence Aware Dictionary and Sentiment Reasoner or VADER) to detect the public sentiment towards the service of Kolkata police during COVID lockdown phase I & II (March 22, 2020, to May 3, 2020)

Cluster Analysis in Online Learning Communities: A Text Mining Approach

This paper presents a theory-informed blueprint for mining unstructured text data using mixed- and multi-methods to improve understanding of collaboration in asynchronous online discussions (AOD). Grounded in a community of inquiry theoretical framework to systematically combine established research techniques, we investigated how AOD topics and individual reflections on those topics affect the formation of clusters or groups in a community. The data for the investigation came from 54 participants and 470 messages. Data analysis combined the analytical efficiency and scalability of topic modeling, social network analysis, and cluster analysis with qualitative content analysis. The cluster analysis found three clusters and that members of the intermediate cluster (i.e., middle of three clusters) played a pivotal role in this community by expressing uncertainty statements, which facilitated a collective sense-making process to resolve misunderstandings. Furthermore, we found that participants’ selected discussion topics and how they discussed those topics influenced cluster formations. Theoretical, practical, and methodological implications are discussed in depth.