Unstructured Textual Information Research Articles

Application of Text Mining in Social Media.

Text mining or information discovery is that sub manner of information mining that is extensively being used to find out hidden styles and huge records from the massive amount of unstructured written fabric. Text mining allows accelerate know-how discovery by way of notably growing the amount records that can be analyzed. Rapid development in digital facts acquisition strategies have caused huge extent of facts. More than 80 percentages of these day’s statistics is composed of unstructured or semi-based information. This sort of data cannot be used until or unless particular records or pattern is determined. The discovery of appropriate patterns and trends to research the text documents from large quantity of records is a big issue. Social community applications create possibilities to set up interaction amongst people main reciprocal studying and exchanging of relevant understanding, chat, feedback, and discussion forums. Information in social media web sites is disorganized and fuzzy in character. In normal lifestyles conversations, people does not care about the spellings and correct grammatical creation of a sentence that could result in exceptional styles of ambivalence, such as syntactic, lexical and semantic. Large quantities of unstructured text information generated on the Internet, text mining is thought to have excessive business fee. We describe how textual content mining might also expand modern organizational studies with the aid of permitting the testing of present or new research questions with information which are likely to be rich.

Method of searching similar documents based on the definition of the semantic core of unstructured textual information

The problem of searching for documents similar to the one specified by the user is considered. The results of the analysis of existing methods of searching for similar documents in content are presented. The basic provisions and results of approbation of the developed method for searching for similar documents based on the definition of the semantic core of unstructured textual information are outlined.

Methods for semantic processing of unstructured textual information

Scientific and methodological approaches to the semantic processing of unstructured textual information based on heuristic and neuromathematical approaches are considered. The results of a conceptual analysis of heuristic and neuromathematical methods for the semantic processing of unstructured textual information are presented. The basic provisions for the implementation of a complete linguistic analysis of unstructured textual information based on the complex application of heuristic and neuromathematical approaches are outlined.

Global Multi-Source Information Fusion Management and Deep Learning Optimization for Tourism

The purpose is to solve the problems of sparse data information, low recommendation precision and recall rate and cold start of the current tourism personalized recommendation system. First, a context based personalized recommendation model (CPRM) is established by using the labeled-LDA (Labeled Latent Dirichlet Allocation) algorithm. The precision and recall of interest point recommendation are improved by mining the context information in unstructured text. Then, the interest point recommendation framework based on convolutional neural network (IPRC) is established. The semantic and emotional information in the comment text is extracted to identify user preferences, and the score of interest points in the target location is predicted combined with the influence factors of geographical location. Finally, real datasets are adopted to evaluate the recommendation precision and recall of the above two models and their performance of solving the cold start problem.

A Brief Survey on Text Analytics Methods and Applications

About 80% organizational data are present in the unstructured (Text) format. E-mails, Social media, notes, and wide variety of different types of documents in text formats are present, but all these data are not got importance and analyzed in meaningful ways. It has been observed that information workers spend their significant time (up to one third) to locating this information and trying to make sense of it. Text analytics (TA) is the process which analyzed all these available unstructured text information and converts it into useful information which helps the organization significantly in their business processes. In this paper we have discussed the business values, methods of text analytics, and business application of text analytics.

Text Analytics: An Application of Text Mining

About 80% organizational data are present in the unstructured (Text) format. E-mails, social media, notes, and wide variety of different types of documents in text formats are present, but all these data are not get importance and analyzed in meaningful ways. It has been observed that information workers spend their significant time (up to one third) to locating this information and trying to make sense of it. Text analytics is the process which analyzed all these available unstructured text information and converts it into useful information which helps the organization significantly in their business processes. In this paper, we have highlighted the business values, some of the methods, and business application of text analytics.

Hybrid attention-based transformer block model for distant supervision relation extraction

With an exponential explosive growth of various digital text information, it is challenging to efficiently obtain specific knowledge from massive unstructured text information. As one basic task for natural language processing (NLP), relation extraction (RE) aims to extract semantic relations between entity pairs based on the given text. To avoid manual labeling of datasets, distant supervision relation extraction (DSRE) has been widely used, aiming to utilize knowledge base to automatically annotate datasets. Unfortunately, this method heavily suffers from wrong labelling due to its underlying strong assumptions. To address this issue, we propose a new framework using hybrid attention-based Transformer block with multi-instance learning for DSRE. More specifically, the Transformer block is, for the first time, used as a sentence encoder, which mainly utilizes multi-head self-attention to capture syntactic information at the word level. Then, a novel sentence-level attention mechanism is proposed to calculate the bag representation, aiming to exploit all useful information in each sentence. Experimental results on the public dataset New York Times (NYT) demonstrate that the proposed approach can outperform the state-of-the-art algorithms on the adopted dataset, which verifies the effectiveness of our model on the DSRE task.

Social analysis of young Basque-speaking communities in twitter

ABSTRACT In this paper, we take into account both social and linguistic aspects to perform demographic analysis by processing a large amount of tweets in Basque language. The study of demographic characteristics and social relationships are approached by applying machine learning and modern deep-learning Natural Language Processing (NLP) techniques, combining social sciences with automatic text processing. More specifically, our main objective is to combine demographic inference and social analysis in order to detect young Basque Twitter users and to identify the communities that arise from their relationships or shared content. This social and demographic analysis will be entirely based on the automatically collected tweets using NLP to convert unstructured textual information into interpretable knowledge.

A Custom Word Embedding Model for Clustering of Maintenance Records

Maintenance records of industrial equipment contain rich descriptive information in free-text format, such as involved parts, failure mechanisms, operating conditions, etc. Our objective is to leverage this unstructured textual information to identify groups of similar maintenance jobs. In this article, we use a natural language based approach and propose a novel custom word embedding model, which utilizes two sources of information, first, maintenance records collected from in-field operations and second, industrial taxonomy, to effectively identify clusters. The advantages of our model include combined use of semantic and taxonomic sources of information for clustering, one step/simultaneous training, which enables knowledge sharing between the two information sources and reduces hyperparameters, and no dependence on third-party data. We demonstrate the efficacy of our model for cluster identification using a real-world dataset. The results show that simultaneous incorporation of semantic and taxonomic information enables accurate extraction of contextual insights for improving maintenance decision-making and equipment reliability.

Automatic Extraction of Adverse Drug Reactions from Summary of Product Characteristics

The summary of product characteristics from the European Medicines Agency is a reference document on medicines in the EU. It contains textual information for clinical experts on how to safely use medicines, including adverse drug reactions. Using natural language processing (NLP) techniques to automatically extract adverse drug reactions from such unstructured textual information helps clinical experts to effectively and efficiently use them in daily practices. Such techniques have been developed for Structured Product Labels from the Food and Drug Administration (FDA), but there is no research focusing on extracting from the Summary of Product Characteristics. In this work, we built a natural language processing pipeline that automatically scrapes the summary of product characteristics online and then extracts adverse drug reactions from them. Besides, we have made the method and its output publicly available so that it can be reused and further evaluated in clinical practices. In total, we extracted 32,797 common adverse drug reactions for 647 common medicines scraped from the Electronic Medicines Compendium. A manual review of 37 commonly used medicines has indicated a good performance, with a recall and precision of 0.99 and 0.934, respectively.

Компараторная идентификация таксономии автоматизированной системы управления морскими портами

The article considers a corporate taxonomy concept as a hierarchical classification of organization or administration, a form of organizing data (documents, digital assets, etc.). There is given the idea of the corporate taxonomy of seaports, branched hierarchy of concepts, objects and subjects of ports, extracted from the hidden knowledge of unstructured textual information and studied for the classification of electronic documents and other sources of information. It has been stated that one of the highest forms of human intellectual activity is understanding and extracting semantic units with their organization into a hierarchical structure. The most convenient and effective method of indirect identification is the method of comparative identification, which deductively uses the basic provisions of the theory of intelligence. There are considered processes of data processing in organizational information systems, surjective imaging, where each element of the value domain has at least one prototype. A system of logical equations showing the regularities of signal processing is given. The logical scheme of using the method of comparative identification for extracting non-obvious knowledge is illustrated. To compile the corporate taxonomy it is necessary to identify classes of conditional equivalence (in addition to classes of unconditional equivalence) that contain groups of close semantic units and have common features that will be grouped into one taxon. The descriptors of lexical units are presented. It has been inferred that the method of comparative identification used to create the corporate taxonomy allows to move from the subjective perception of meanings to the objective relationship between the documents in the automated system of seaports and the semantic units represented by keywords, headings, etc.

Rating prediction based on combination of review mining and user preference analysis

Review websites allow users to share their reviews of products or businesses, give ratings to products or businesses, and interact with other users. Due to the rapid growth of online review data, users face the problem of information overload. To resolve this problem, many researches have proposed various recommendation methods based on the analysis of users’ ratings. Besides user ratings, the review websites contain unstructured textual data and information of different aspects which has different impact and importance to both users and businesses. It may lead to inaccurate rating predictions, as it is difficult to know users’ preferring aspects and their corresponding importance by analyzing users’ rating data.To resolve the above-stated problems, this research proposed aspect-based rating prediction methods, i.e., ARPM and ARPM-Social, which integrate aspect detection and sentiment analysis to generate user preference and business performance, combined with the results of social behavior analysis to predict the ratings of the businesses that users will be interested in the future. Based on the experimental results, the proposed methods perform better than other traditional rating-based prediction methods. They can effectively analyze various aspects and sentiments from the reviews of users and businesses, as well as improve the accuracy of rating predictions.

Модели машинного обучения для классификации отзывов о банках

Using the banking products and services review corpus, analysis is conducted to establish different text classification models. The paper explores different approaches to the processing of unstructured textual information. Based on the selected approaches, the review corpus on banking products and services received during the COVID-19 pandemic is analyzed. An automatic Internet resources parser has been developed to obtain the required training sample. Software has been developed that implemens basic methods for the classification models construction. This model can be used to create system for monitoring people’s attitudes to banking processes.

Mathematical model of the process of text information semantic processing

A formalized description of the process of unstructured text information complete linguistic analysis is considered. The results of the development of a mathematical model of semantic processing of unstructured texts based on data arrays and rules of complete linguistic analysis are presented. The data on the implementation of the developed scientific provisions are given.

Fault Diagnosis of Signal Equipment on the Lanzhou‐Xinjiang High‐Speed Railway Using Machine Learning for Natural Language Processing

The Lanzhou‐Xinjiang (Lan‐Xin) high‐speed railway is one of the principal sections of the railway network in western China, and signal equipment is of great importance in ensuring the safe and efficient operation of the high‐speed railway. Over a long period, in the railway operation and maintenance process, the railway signaling and communications department has recorded a large amount of unstructured text information about equipment faults in the form of natural language. However, due to irregularities in the recording methods of these data, it is difficult to use directly. In this paper, a method based on natural language processing (NLP) was adopted to analyze and classify this information. First, the Latent Dirichlet Allocation (LDA) topic model was used to extract the semantic features of the text, which were then expressed in the corresponding topic feature space. Next, the Support Vector Machine (SVM) algorithm was used to construct a signal equipment fault diagnostic model that reduced the impact of sample data imbalance on the classification accuracy. This was compared and analyzed with the traditional Naive Bayes (NB), Logistic Regression (LR), Random Forest (RF), and K‐Nearest Neighbor (KNN) algorithms. This study used signal equipment failure text data from the Lan‐Xin high‐speed railway to conduct experimental analysis and verify the effectiveness of the proposed method. Experiments showed that the accuracy of the SVM classification algorithm could reach 0.84 after being combined with the LDA topic model, which verifies that the natural language processing method can effectively realize the fault diagnosis of signal equipment and has certain guiding significance for the maintenance of field signal equipment.

Method of syntactic-semantik analysis of unstructured texts

The methodology of computer linguistic analysis of unstructured text information is considered. The basic provisions of various approaches to the automation of automatic processing of unstructured texts in computer systems are presented. A formalized description of the method of syntactic-semantic computer analysis of texts is given.

Review of Natural Language Processing in Radiology.

Natural language processing (NLP) is an interdisciplinary field, combining linguistics, computer science, and artificial intelligence to enable machines to read and understand human language for meaningful purposes. Recent advancements in deep learning have begun to offer significant improvements in NLP task performance. These techniques have the potential to create new automated tools that could improve clinical workflows and unlock unstructured textual information contained in radiology and clinical reports for the development of radiology and clinical artificial intelligence applications. These applications will combine the appropriate application of classic linguistic and NLP preprocessing techniques, modern NLP techniques, and modern deep learning techniques.

Networks from archives: Reconstructing networks of official correspondence in the early modern Portuguese empire

Historical archives provide invaluable insights into societies of the past, including social networks. However, the required amount of traditional archival work makes historical network studies usually small-scaled. We consider the problem of processing a large corpus of unstructured textual information to extract network data. The corpus consists of almost 170,000 documents of administrative correspondence of the Portuguese Empire, from 1610 to 1833, catalogued in the Portuguese Overseas Archives of Lisbon. Our contribution is twofold: the method and the result. Firstly, grounded in the review of manual, semi-manual and automatic methods of network data extraction from natural language corpora, we propose and demonstrate an approach using modern natural language processing algorithms. This approach tries to mimic traditional archivist’s coding practices and is applicable to large corpora of texts, for which manual coding is infeasible because of scale. We believe our approach is generic and adaptable to other substantive contexts, languages, and types of historical archives. Secondly, the dataset created is rich in additional information such as occupation, administrative affiliation, and geographical location of senders and recipients. We provide a preliminary network analysis suggesting that the dataset is an attractive material for historians and social network researchers for addressing research questions about the political and social evolution of the early modern Portuguese Empire, spanning the reign of seven Portuguese monarchs.

Critical risk factors in ship fire accidents

ABSTRACT Risk management of ship fire accidents (SFA) is a vital issue in maritime transportation systems since ship fires are instantaneous and developing events-small errors can cause serious accidents. The traditional approach for SFA analysis is on a coarse-grained level, using accident data from one or several case studies or expert knowledge; such data acquisition method substantially relies on subjective expert experience and brings about epistemic uncertainties. This study proposes a new framework for identifying critical risk factors in SFA. First, grounded theory (GT) is applied to obtain risk factors on a fine-grained level by converting unstructured text information to structured accident data. Then, Fault tree analysis (FTA) is used to develop a risk model and make a quantitative analysis of the SFA. Bayesian network (BN) is also integrated to verify and identify the critical risk factors in SFA with sensitivity analysis. Results indicate that sealing off the ship cabin timely, effective ventilation system, as well as prompt emergency response by crews and passengers on board are the dominant factors. Our findings provide useful insights for maritime safety administration and shipping companies to take actions for fire prevention.

Eras: Improving the quality control in the annotation process for Natural Language Processing tasks

The increasing amount of valuable, unstructured textual information poses a major challenge to extract value from those texts. We need to use NLP (Natural Language Processing) techniques, most of which rely on manually annotating a large corpus of text for its development and evaluation. Creating a large annotated corpus is laborious and requires suitable computational support. There are many annotation tools available, but their main weaknesses are the absence of data management features for quality control and the need for a commercial license. As the quality of the data used to train an NLP model directly affects the quality of the results, the quality control of the annotations is essential. In this paper, we introduce ERAS, a novel web-based text annotation tool developed to facilitate and manage the process of text annotation. ERAS includes not only the key features of current mainstream annotation systems but also other features necessary to improve the curation process, such as the inter-annotator agreement, self-agreement and annotation log visualization, for annotation quality control. ERAS also implements a series of features to improve the customization of the user’s annotation workflow, such as: random document selection, re-annotation stages, and warm-up annotations. We conducted two empirical studies to evaluate the tool’s support to text annotation, and the results suggest that the tool not only meets the basic needs of the annotation task but also has some important advantages over the other tools evaluated in the studies. ERAS is freely available at https://github.com/grosmanjs/eras.