Real Text Data Research Articles

Analysis and Synthesis of Technology for Textual Information Classification

The task of developing effective text information classification systems requires the thoughtful analysis and synthesis of variable components of technology. These components strongly affect the practical efficiency and the requirements to the data. For this purpose, a typical technology was discussed, comparing the regular “learning from features” approach versus the more advanced “deep learning” approach, that studies from data. In order to implement the technology, the first approach was tested, which included the means (methods, algorithms) for analysis of the features of the source text, by applying the dimensionality transformation, and building model solutions that allow the correct classification of data by a set of features. As a result, all the steps of the technology are described, which allowed to determine the way of presenting data in terms of hidden features in data, their presentation in a standard visual form and evaluate the solution, as well as its practical efficiency, based on this set of features. In a depth study, the informational core of the document was studied, using the regression and T-stochastic grouping of features for dimensionality reduction.The separate results contain estimation of practical efficiency of the algorithms in terms of time and relative performance for each step of the proposed technology. This estimation gives a possibility to obtain the best algorithm of intelligent data processing that is useful for a given dataset and application. In order to estimate the best suited algorithm for separation in reduced dimension an experiment was carried out which allowed the selection of the best range of data classification algorithms, in particular boosting methods. As a result of the analysis of the technology, the necessary steps of this technology were discussed and the classification on real text data was conducted, which allowed to identify the most important stages of the technology for text classification.

A mathematical modification of the WNTM associative algorithm for cognitive analysis and efficient detection of dualisms in text and video streaming data

We report on a modification of the WNTM (Word Network Topic Model) algorithm for efficient modelling of bitermes – pairs of words that frequently occur together in texts of different topics. The modified algorithm is an extension of the classical topic model and allows efficient detection and extraction of semantic relations between pairs of words. The paper presents formalized mathematical equations describing the process of modelling biterms, and also presents the results of experiments on real text data, confirming the effectiveness of the proposed approach.

Information flow estimation: A study of news on Twitter

News media has long been an ecosystem of creation, reproduction, and critique, where news outlets report on current events and add commentary to ongoing stories. Understanding the dynamics of news information creation and dispersion is important to accurately ascribe credit to influential work and understand how societal narratives develop. These dynamics can be modelled through a combination of information-theoretic natural language processing and networks; and can be parameterised using large quantities of textual data. However, it is challenging to see “the wood for the trees”, i.e., to detect small but important flows of information in a sea of noise. Here we develop new comparative techniques to estimate temporal information flow between pairs of text producers. Using both simulated and real text data we compare the reliability and sensitivity of methods for estimating textual information flow, showing that a metric that normalises by local neighbourhood structure provides a robust estimate of information flow in large networks. We apply this metric to a large corpus of news organisations on Twitter and demonstrate its usefulness in identifying influence within an information ecosystem, finding that average information contribution to the network is not correlated with the number of followers or the number of tweets. This suggests that small local organisations and right-wing organisations which have lower average follower counts still contribute significant information to the ecosystem. Further, the methods are applied to smaller full-text datasets of specific news events across news sites and Russian troll accounts on Twitter. The information flow estimation reveals and quantifies features of how these events develop and the role of groups of trolls in setting disinformation narratives. In summary, this work provides a new methodology for examining the information transmitted between content producers in any connected system of natural language, a toolkit with applications to the many networked discourses of our online world.

A Novel Text Ensemble Clustering Based on Weighted Entropy Filtering Model

Text clustering is one of the important technical bases of natural language processing, and ensemble clustering improves the robustness of text clustering. According to the existing research of scholars and experts, the quality and diversity of basic clustering have a great influence on consensus clustering, and it has a particularly significant effect on text clustering. However, there are a few pieces of research aiming at reducing the number of low-quality clustering in ensembles. This paper proposes a novel clustering filtering model based on entropy criteria. The entropy criterion is used to evaluate the uncertainty of each cluster w.r.t. the ensemble. Two indexes are proposed on the basis of the uncertainty of cluster, namely, Clustering Trend Index (CTI) which indicates the contribution of each cluster w.r.t. basic clustering, and Cluster Consistency Index(CCI) which indicates the degree of cluster dispersion in the basic clustering. The proposed clustering filtering model is built on the basis of new weight using two proposed indexes. Thereby, by dropping the low-quality clustering, the percentage of high-quality clustering will increase. A large number of experiments on various real text data sets using optimal thresholds show that the proposed method has greatly improved accuracy and robustness, and is superior to existing ensemble clustering algorithms.

INFORMATION INTELLIGENT TECHNOLOGY OF THE AUTOMATIC GEOREFERENCING OF THE ECOLOGICAL TEXT NATURAL-LANGUAGE INFORMATION

Research is devoted to the development of the information intelligent technology of the automated georeferencing of text information-language information by means of the NER (named entity recognition) technology and NLP (natural language processing) technology with the reference to the geographical objects of the vector maps. The study kit is formed by means of partition of the labeled entities-locations and entities-organizations into separate samples, which contain combined in a certain way entities, that characterize the planar objects of larger surface and, separately, those that characterize smaller planar objects, linear and point objects. Such division of the data enables to organize multistage revision of the identification results and models used, this allows to provide simultaneously the increase of the completeness, accuracy and speed of georeferencing of the set ecological text information. Recommendations, regarding the application of this technology for Ukrainian, English and other languages as well as  for the algorithm for the  preparation of the input cartographic data, using GIS-package of ArcGIS programs are developed. The examples of the application of the separate elements of the suggested technology to real text data about the state of water arrays of the South Bug basin are given.

Comparative Study of Clustering Algorithms in Text Mining Context

The spectacular increasing of Data is due to the appearance of networks and smartphones. Amount 42% of world population using internet [1]; have created a problem related of the processing of the data exchanged, which is rising exponentially and that should be automatically treated. This paper presents a classical process of knowledge discovery databases, in order to treat textual data. This process is divided into three parts: preprocessing, processing and post-processing. In the processing step, we present a comparative study between several clustering algorithms such as KMeans, Global KMeans, Fast Global KMeans, Two Level KMeans and FWKmeans. The comparison between these algorithms is made on real textual data from the web using RSS feeds. Experimental results identified two problems: the first one quality results which remain for algorithms, which rapidly converge. The second problem is due to the execution time that needs to decrease for some algorithms.

A multi-stage method for content classification and opinion mining on weblog comments

In this paper, we illustrate how to combine supervised machine learning algorithms and unsupervised learning techniques for sentiment analysis and opinion mining purposes. To this end, we describe a multi-stage method for the automatic detection of different opinion trends. The proposal has been tested on real textual data available from comments introduced in a weblog, connected to organizational and administrative affairs in a public educational institution. The use of the described tool, given its potential impact to obtain valuable knowledge from opinion streams created by commenters, may be straightforwardly extended, for example, to the detection of opinion trends concerning policy decision making or electoral campaigns.

A novel semi-supervised learning framework with simultaneous text representing

Text representation has received extensive attention in text mining tasks. There are various text representation models. Among them, vector space model is the most commonly used one. For vector space model, the core technique is term weighting. To date, a great deal of different term-weighting methods have been proposed, which can be divided into supervised group and unsupervised group. However, it is not advisable to use these two groups of methods directly in semi-supervised applications. In semi-supervised applications, the majority of the supervised term-weighting methods are not applicable as the label information is insufficient; meanwhile, the unsupervised term-weighting methods cannot make use of the provided category labels. Thus, a semi-supervised learning framework for iteratively revising the text representation by an EM-like strategy is proposed in this paper. Furthermore, a new supervised term-weighting method t f.sd f is proposed. T f.sd f has the ability to emphasize the importance of terms that are unevenly distributed among all the classes and weaken the importance of terms that are uniformly distributed. Experimental results on real text data show that the proposed semi-supervised learning framework with the aid of t f.sd f performs well. Also, t f.sd f is shown to be efficient for supervised learning.

Cleaning data: guess the olympian

AbstractThis article tackles the problem of what should be done with real textual data that are contaminated by errors of recording, particularly when the data contain words that are misspelt, unintentionally or otherwise.

A subspace decision cluster classifier for text classification

In this paper, a new classification method (SDCC) for high dimensional text data with multiple classes is proposed. In this method, a subspace decision cluster classification (SDCC) model consists of a set of disjoint subspace decision clusters, each labeled with a dominant class to determine the class of new objects falling in the cluster. A cluster tree is first generated from a training data set by recursively calling a subspace clustering algorithm Entropy Weighting k-Means algorithm. Then, the SDCC model is extracted from the subspace decision cluster tree. Various tests including Anderson–Darling test are used to determine the stopping condition of the tree growing. A series of experiments on real text data sets have been conducted. Their results show that the new classification method (SDCC) outperforms the existing methods like decision tree and SVM. SDCC is particularly suitable for large, high dimensional sparse text data with many classes.

Teaching Qualitative Methods in Management Classrooms —

This article describes an innovative approach to teaching qualitative research methods for organizational studies using researchers who publicly display in real-time their approaches to analyzing real textual data. The authors developed this approach to help students appreciate the many ways in which qualitative research can generate uniquely meaningful interpretations of text. Students reported that the qualitative module was worthwhile in helping them to demystify the process of data analysis, particularly demonstrated through a joint panel session of three researchers. The authors also hope that this article inspires others to use similar approaches in their own teaching of qualitative methods.