Huge Text Research Articles

Maximum Frequent Item Set based Clustering Algorithm for Big Text Data

Due to fast growth of internet and continuous expansion of World Wide Web like digital libraries, online news contributes to massive amount of electronic unstructured text documents on the web. Although lot traditional techniques are available to extract the knowledge from large collection of text documents, still to improve precision of the web search retrieval and to find most appropriate documents from huge text collections proficiently is a big challenge. Clustering techniques helps the search engine to retrieve the documents. The proposed system overcomes existing problems using bivariate n-gram frequent item clustering algorithm by concept of maximum frequent set which maintain the sequence and meaning of sentence in order to reduce huge dimension and and frequent item sets finds similarity. Then based on maximum document occurrence we cluster the documents. Thus our method obtains quality of clusters when compared with existing methodologies and improves the efficiency. The experiment is shown for sample Newsgroup dataset for existing K-Mean and FICMDO (Frequent item clustering method based on maximum document occurrence) and proved the f-measure is higher for our algorithm. Since the f-measure increases, obtains efficient clusters. Hence it is faster and efficient big data method which improves the performance when compared with vector space model like K-Means algorithm.

Extractive Text Summarization Using Graph Based Ranking Algorithm And Mean Shift Clustering

Text summarization is the process of converting a huge text file into a summarized version preserving its meaning and context. The major aspect of any extractive text summarization technique is to provide precise and accurate summary by using any sentence ranking algorithms. The text is first tokenized and preprocessed. The sentences in the text are ranked according to the corresponding graph based ranking algorithm and finally the top K-scored sentences are included in the summary. Combining the graph based ranking algorithm along with mean shift clustering enables us to efficiently summarize the articles increasing the precision of the system generated summary and also reduces the execution time. The efficiency of these approaches are evaluated using rouge metrics and execution time.

Construction of Financial News Sentiment Indices Using Deep Neural Networks

With the progress in natural language processing and machine learning, using deep learning techniques to extract valuable information from financial texts has gained popularity among researchers. However, as deep learning models require large amounts of labelled data, current models for financial text mining are sensitive to noise and prone to overfitting due to the lack of labelled data in financial fields. We address this issue by using pretrained word representations and BERT model to leverage knowledge acquired from huge text corpus. In particular, we design a deep neural network model that combines CNN, Att-BLSTM and BERT for predicting financial news sentiment polarity. Then, the news sentiment indices are constructed based on the predictive model. In terms of predicting news sentiment polarity, we show that our model outperforms state-of-the-art competitors. Furthermore, we show that the news sentiment exhibits a significant relationship with stock market returns.

To Build Corpus of Sindhi Language

The present day state of Sindhi corpus construction is elaborated in detail in this paper. The issues like corpus acquisition, tokenization and preprocessing have been analyzed and discussed minutely for Sindhi corpus enhancement. Initial observations and results are included for letter unigram, bigram and trigram frequencies. There has been discussed the present status of Sindhi corpus in perspective of restriction and future work. Orthography and script were also explored in this paper with reference to corpus development. Basically the word corpus was used first time by German Scholar (Das Corpus). The plural of corpus is corpora, which is used for huge text data consists of millions and billions of text data. The task of Natural Language Processing was very challenging because there was the scarcity of resources for computational linguistics and research. Different text corpora have been made in different languages of different countries, after reviewing the corpora of different languages of various countries, we are trying to make the corpus for Sindhi language.

Trends in Contemporary Art Discourse: Using Topic Models to Analyze 25 years of Professional Art Criticism

In this article, we use topic modeling to systematically explore topics discussed in contemporary art criticism. Analyzing 6965 articles published between 1991 and 2015 in Frieze, a leading art magazine, we find a plurality of topics characterizing professional discourse on contemporary art. Not surprisingly, media- or genre-specific topics such as film/cinema, photography, sculpture/installations, etc. emerge. Interestingly, extra-artistic topics also characterize contemporary art criticism: there is room for articles on new digital technology and on art and philosophy; there is also growing interest in the relationship between art and society. Our analysis shows that despite evolutions in the field of contemporary art – such as the ‘social turn’, in which contemporary art starts paying more attention to social forms and content – the prevalence of certain topics in contemporary art criticism has barely changed over the past 25 years. With this article, we demonstrate the unique value of topic modeling for cultural sociology: it is both a powerful computational technique to generate a bird’s-eye view of a huge text corpus and a heuristic device that locates key texts for further close reading.

Partial Greedy Algorithm to Extract a Minimum Phonetically-and-Prosodically Rich Sentence Set

A phonetically-and-prosodically rich sentence set is so important in collecting a read-speech corpus for developing phoneme-based speech recognition. The sentence set is usually searched from a huge text corpus of million sentences using the optimization methods. One of the commonly used optimization methods for this case is a Least-to-Most Greedy (LTMG) algo-rithm. It is effective in minimizing the number of phoneme-units. Unfortunately, it does not distribute their frequencies. In this paper, a new method called Partial LTMG algorithm (PLTMG) is proposed to search an optimum set containing triphones and prosodies those are distributed in a near-uniform fashion. Testing on an Indonesian text corpus of ten million sentences crawled from some websites of newspapers and novels shows that the proposed method is not only capable of minimizing both phoneme-units and prosodies but also effective in distributing their frequencies.

Integrating on-premises data with customer relationship management application on cloud: A hybrid IT infrastructure support service

Customer relationship management (CRM) is a group of data-driven, integrated solution that enhances how an organization interacts and does business with the customers. University support service centres provide hone support for the university stakeholders’ queries and requests related to academic information. The support centre uses retrieval of information from huge text documents, records, and log files from disparate data sources residing on on-premises. Handling data has become a critical task and moving all the data to the cloud is not feasible. The objective of designed hybrid solution is to serve a resolving customer service request and a scalable solution to a large set of data for the organizations concerned with security, efficiency, and to use the advantages of the cloud service. In the proposed hybrid solution, the data resides on the on-premises is connected to the customer relationship management application deployed on the cloud, this reduces the IT infrastructure tasks at a university support service. We describe the system’s architecture and the technical challenges we have faced. The CRM application is deployed on Microsoft Azure cloud test instance and the Azure service bus relay is used to connect the on-premises data.

Legal Transplants: A conflict of statutory law and customary Law in Papua New Guinea

The state of Papua New Guinea adopted the common law system of government in 1975 during independence. The genesis of most if not all its legislation can be traced back to the United Kingdom, Australia, New Zealand and other commonwealth countries. The tendency for legal transplants of legislative texts from these common law jurisdictions to sections of Papua New Guinean laws has been a constant reoccurrence. With huge texts of laws transplanted it begs the question whether these laws are coherent with existing laws and appropriate for Papua New Guinea. This paper analyses the existing Mining Act 1992 and Oil and Gas Act 1998 vesting ownership of minerals and petroleum in the State although these resources are located on customary land. I will use the said acts to establish the hypothesis that, in the rush to transplant legislation from Australia, this transplanted provision fails the functionality test and is not effective in Papua New Guinea. I will also try and point out the effects and solutions to redress this situation. Keywords: Customary law, alienated land, legal transplants, functionality test

The exact probability law for the approximated similarity from the Minhashing method

We propose a probabilistic setting in which we study the probability law of the Rajaraman and Ullman \textit{RU} algorithm and a modified version of it denoted by \textit{RUM}. These algorithms aim at estimating the similarity index between huge texts in the context of the web. We give a foundation of this method by showing, in the ideal case of carefully chosen probability laws, the exact similarity is the mathematical expectation of the random similarity provided by the algorithm. Some extensions are given. \noindent \textbf{R\'{e}sum\'{e}.} Nous proposons un cadre probabilistique dans lequel nous \'{e}tudions la loi de probabilit\'{e} de l'algorithme de Rajaraman et Ullman \textit{RU} ainsi qu'une version modifi\'{e}e de cet algorithme not\'{e}e \textit{RUM}. Ces alogrithmes visent \`{a} estimer l'indice de la similarit\'{e} entre des textes de grandes tailles dans le contexte du Web. Nous donnons une base de validit\'e de cette m\'{e}thode en montrant que pour des lois de probabilit\'{e}s minutieusement choisies, la similarit\'{e} exacte est l'esp\'{e}rance math\'{e}matique de la similarit\'{e} al\'{e}atoire donn\'{e}e par l'algorithme \textit{RUM}. Des g\'en\'eralisations sont abord\'ees.

Research and Implementation of Finding Duplicate Science Project Based on Dimension Filtering of Q-gram Index

Because of the huge number and the dimensions sparseness of the scientific research text set, a new similarity search algorithm for the scientific research text set is proposed, which is based on the weights of the q-gram. The algorithm can greatly reduce the number of dimensions, and then quickly find the similar text. Experiments show that the time consumption and the space consumption are decreased largely for the huge text set, and it has a high accuracy rate.

From Writing Dictionaries to Weaving Lexical Networks

Dictionaries are designed as huge texts made up of a collection of much smaller texts, i.e. lexicographic articles. To put it differently, dictionaries are two-dimensional textual models of natural language lexicons. Lexicographers, however, are well aware of the fact that their task is to account for a truly multidimensional entity: a gigantic graph of lexical units connected by various paradigmatic and syntagmatic relations. The most significant advance that computer science will bring to the future of lexicography is therefore not the ability to better store, search and manipulate textual lexicographic data; it will be to allow lexicographers to bypass the text as a formal representation of lexicons and to directly work on lexical networks. Such networks are more suitable to the lexicographic endeavour because they are better formal metaphors of the “natural” structure we are trying to account for. This paper presents lexical systems as graph models of lexicons and introduces the corresponding lexicography of virtual dictionaries. It is based on extensive lexicographic work that is being conducted on the French Lexical Network, a lexical database built according to theoretical and methodological principles borrowed from Explanatory Combinatorial Lexicology.

A Survey of Various Machine Learning Techniques for Text Classification

Sentiments are expressions of one’s words in a sentence. Hence understanding the meaning of text in the sentence is of utmost importance to people of various fields like customer reviews in companies, movie reviews in movies, etc. It may involve huge text data to analyze and it becomes totally unviable for manually understanding the meaning of sentences. Classifier algorithms should be used to classify the various meaning of the sentences. By using pre-defined data to train our classifier and three different algorithms namely Naive Bayes, Support Vector Machines, Decision Trees, we can simplify the task of text classification. Using relevant results and examples we will prove that SVM is one of the better algorithms in providing higher accuracy over the other two algorithms i.e. Naive Bayes and Decision Tree.

Quantifying the language of British politics, 1880-1910

Historians – influenced by the ‘linguistic turn’ – currently study political language almost exclusively qualitatively, through sharply focused case studies. This approach has many strengths but has limited power to examine national long-term political change over such vast discourses as general election campaigns. This article contends that electronic quantitative text analysis methods (inspired by corpus linguistics) could potentially reshape historians' understanding of the language of British electoral politics by providing a powerful ability holistically to assess the typicality, scope and power of key issues, ideologies, personalities and ideas across huge texts. It presents three supporting case studies – on Irish Home Rule, imperialism and William Gladstone – where a multi-million word ‘corpus’ of election speeches from this period is interrogated by computer.

Quantifying the Semantic Representations of Adolescents’ Memories of Positive and Negative Life Events

We quantified the semantic content in adolescents’ descriptions of positive and negative life events and studied how these descriptions are related to the assessment Subjective Well-Being (SWB) at two points in time. The semantic content of the descriptions was quantified by Latent Semantic Analysis (LSA). LSA is a computational method based on algorithms stemming from computational linguistics, where a high dimensional semantic representation of words can be generated from co-occurrence of words in huge text corpora. We investigated if the semantic content of written autobiographical memories of positive and negative life events predicted traditionally ranked measures of SWB, i.e., self-reports of Positive and Negative Affect, and thus created semantic measures of SWB. Such measures can be used to investigate the relationship between semantic content and SWB, which could only indirectly be accomplished by the ranked data. Pupils wrote down positive or negative life events during the last three months and self-reported SWB. Four weeks later, participants were presented with their own description and asked to report current SWB. The results showed that the semantic representation predicted SWB and experimental conditions. The agreement between semantic and ranked measures supports the validity of the semantic scores. We argue that our proposed method for studying SWB provides new and essential information about well-being by the quantification of a richer set of information from adolescents’ own memories.

Write‐in comments in employee surveys

PurposeItems that ask the respondent for a comment in his/her own words are becoming increasingly popular in online employee surveys, but research on such comments is scarce. The purpose of this paper is to analyze, theoretically and empirically, what kind of comments are generated in employee surveys by respondents who differ in terms of job satisfaction and commitment.Design/methodology/approachThe data studied here are from an online employee survey conducted in 2004 in a multinational IT organization with a German headquarter. Some 24,000 employees generated about 75,000 comments focusing on 15 topic fields. The comments were additionally coded on their tone, using a computerized dictionary approach especially developed for this purpose. Frequencies, wordiness, and the tone of different types of comments are measured. The statistical relationship of comments to job satisfaction and to organizational commitment is analyzed.FindingsSome 40 per cent of the respondents provide comments. Most comments have a negative tone. Negative comments are wordier than positive ones. The likelihood of writing comments is inversely related to the respondents’ job satisfaction and to their organizational commitment. Dissatisfied employees and employees with low commitment also write more negative and wordier comments.Practical implicationsThe study sets benchmarks on what to expect when using open‐ended comment fields in employee surveys. A methodology for analyzing huge text files with respect to their tone is presented.Originality/valueThis is the first paper that investigates a realistically large data set of comments. It also shows how to use computer methods to relate frequencies, wordiness, and tone of comments to standard variables such as job satisfaction or commitment.

Special Issue on Innovation over Multimedia Processing

With broadband networks, audio/video coding, and processing techniques and user devices swiftly advancing, multimedia streaming over networks is now a reality. Distributed interactive multimedia applications ? one of the fastest growing market sectors demanding innovations ? are covered in this special issue on multimedia processing (MP). This special issue focuses on ambitious and intriguing papers from experts on a wide variety of multimedia areas. These articles address innovations and offer effective solutions to MP problems. Y. Fushio et al. propose a shadow generation system based on shape recognition, while M. Suzuki et al. develop musical expression generation reflecting the userfs impression in kansei space and fuzzy rules. K. Ishizuka et al. evaluate an operetta songs generation system based on the impressions given by story scenes, which are strongly related to the integrative research projects of kansei engineering, music, text, and visual perception. T. Toyota et al. develop a web news visualization system based on an efficient self-organizing map using a restricted region search and dimensionality reduction. S. Kawauchi et al. propose a knowledge expansion support system using related search keyword generation based on Wikipedia categories and point-wise mutual information, involving that are aggressive looking progressive research projects covering huge text information. T. Tanabata et al. present an interactive data mining tool for microarray data analysis using formal concept analysis, which adds a new dimension to the bio-informatics field. T. Kawamura et al. analyze keywords input to a book search engine. T. Sugimoto et al. propose a recommendation system involving the use of comprehensive trend indications based on weighted complete graphs, which are related to topical discussions in the information retrieval and recommendation field. Readers are sure to find new and valuable information and ideas in their own fields in these excellent articles.

Database construction and data retrieval for optimal casting design

The existing optimal casting design data retrieval system enables a user to easily retrieve only the needed text data from a huge text database. While the necessary data can be found quickly through search terms, however, it is difficult to access know how gained in the field and the shape data on casting design models. To solve this problem, an optimal system has been developed to retrieve all the shape data generated in the field. The features and keywords of comparative optimal casting designs are extracted and stored in the shape feature keyword database. Through the user interface, the two-dimensional and three-dimensional shape data and shape keywords of a product model are compared with those of casting designs stored in the database. In this way, the optimal casting design most similar to a good model is chosen.

Constructing and maintaining knowledge organization tools: a symbolic approach

Purpose - To propose a comprehensive and semi-automatic method for constructing or updating knowledge organization tools such as thesauri. Design/methodology/approach - The paper proposes a comprehensive methodology for thesaurus construction and maintenance combining shallow NLP with a clustering algorithm and an information visualization interface. The resulting system TermWatch, extracts terms from a text collection, mines semantic relations between them using complementary linguistic approaches and clusters terms using these semantic relations. The clusters are mapped onto a 2D using an integrated visualization tool. Findings - The clusters formed exhibit the different relations necessary to populate a thesaurus or ontology: synonymy, generic/specific and relatedness. The clusters represent, for a given term, its closest neighbours in terms of semantic relations. Practical implications - This could change the way in which information professionals (librarians and documentalists) undertake knowledge organization tasks. TermWatch can be useful either as a starting point for grasping the conceptual organization of knowledge in a huge text collection without having to read the texts, then actually serving as a suggestive tool for populating different hierarchies of a thesaurus or an ontology because its clusters are based on semantic relations. Originality/value - This lies in several points: combined use of linguistic relations with an adapted clustering algorithm, which is scalable and can handle sparse data. The paper proposes a comprehensive approach to semantic relations acquisition whereas existing studies often use one or two approaches. The domain knowledge maps produced by the system represents an added advantage over existing approaches to automatic thesaurus construction in that clusters are formed using semantic relations between domain terms. Thus while offering a meaningful synthesis of the information contained in the original corpus through clustering, the results can be used for knowledge organization tasks (thesaurus building and ontology population) The system also constitutes a platform for performing several knowledge-oriented tasks like science and technology watch, textmining, query refinement.

L-Tree Match: A New Data Extraction Model and Algorithm for Huge Text Stream with Noises

In this paper, a new method, named as L-tree match, is presented for extracting data from complex data sources. Firstly, based on data extraction logic presented in this work, a new data extraction model is constructed in which model components are structurally correlated via a generalized template. Secondly, a database-populating mechanism is built, along with some object-manipulating operations needed for flexible database design, to support data extraction from huge text stream. Thirdly, top-down and bottom-up strategies are combined to design a new extraction algorithm that can extract data from data sources with optional, unordered, nested, and/or noisy components. Lastly, this method is applied to extract accurate data from biological documents amounting to 100GB for the first online integrated biological data warehouse of China.

Collective Violence: Effective Strategies for Assessing and Interviewing in Fatal Group and Institutional Aggression

Back to table of contents Previous article Next article Book ReviewFull AccessCollective Violence: Effective Strategies for Assessing and Interviewing in Fatal Group and Institutional AggressionAbraham L. Halpern, M.D.Abraham L. HalpernSearch for more papers by this author, M.D.Published Online:1 Mar 2000https://doi.org/10.1176/appi.ps.51.3.398-aAboutSectionsView EPUB ToolsAdd to favoritesDownload CitationsTrack Citations ShareShare onFacebookTwitterLinked InEmail This massive book is a product of 32 contributors, including two chief editors, two associate editors, and ten consulting editors. The book is divided into three distinct parts. The first part, Foundational Issues, contains six chapters covering violent groups and institutions in the United States, juvenile delinquency and violent crime, and women and aggression. Part 2, Collective Violence by Private Groups and Institutions, contains five chapters whose topics include cults, hate crimes, profiling and criminal investigative analysis of violent crimes, and dealing with large-scale hostage and barricade incidents. Part 3, Collective Violence by Government Institutions, comprises nine chapters providing detailed information on subjects ranging from capital punishment and deadly use of police force to chemical and biological violence and "oppression by science."In addition, appendixes totaling more than 100 pages include chapters discussing misuse of psychological techniques under U.S. government auspices, atrocities that took place during the Vietnam War, and assessment of dangerousness using handwriting characteristics. A unique and valuable feature of this book is that almost every chapter has an excellent annotated bibliography summarizing the most significant works on the subject of violence.The chapters on hate crimes, deadly cults, and antigovernment terrorists are especially thorough and fascinating. The book will surely be of interest to persons wishing to understand institutional and government-initiated violence. However, I cannot see many individuals purchasing a personal copy of this astronomically priced book. Nor would a library permit anyone, even a speed reader, to borrow it for the extended period necessary to read it in its entirety.An important objective of this huge text, not hinted at in the title, is to propose principles and methods useful for inhibiting group and institutional lethal aggression. In this, the volume succeeds admirably. But even so, with the possible exception of the chapter on cults, most mental health professionals would not find the book of value in their clinical practice.In sum, Collective Violence is an absorbing book, but not a useful one for busy mental health professionals seeking to improve their therapy skills.Dr. Halpern is professor emeritus of psychiatry at New York Medical College in Valhalla, New York.edited by Harold V. Hall and Leighton C. Whitaker; Boca Raton, Florida, CRC Press, 1998, 721 pages, $134.95 FiguresReferencesCited byDetailsCited ByNone Volume 51Issue 3 March 2000Pages 398-a-399 Metrics History Published online 1 March 2000 Published in print 1 March 2000